2024-09-19T11:12:47.758 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T11:12:47.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T11:12:47.861 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434 branch: wip-yuval-test description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} email: null first_in_suite: false job_id: '7912434' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuval-test ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: 08af21e0d16078b88ddf808fe9ba2ab84dbe6754 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08af21e0d16078b88ddf808fe9ba2ab84dbe6754 rgw: ec-data-pool: true frontend: beast s3tests: slow_backend: true thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuval-test sha1: 08af21e0d16078b88ddf808fe9ba2ab84dbe6754 owner: scheduled_yuvalif@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 seed: 4251 sha1: 08af21e0d16078b88ddf808fe9ba2ab84dbe6754 sleep_before_teardown: 0 subset: 1/3 suite: rgw suite_branch: wip-yuval-test suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_08af21e0d16078b88ddf808fe9ba2ab84dbe6754/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 08af21e0d16078b88ddf808fe9ba2ab84dbe6754 targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBImOkcQMyTTsxZiTVVPdRrkNVGDdXuiUSCHaj35l0O1g/LEZt8LQeLO0yDuqCM2XEh54Yi6Np2VyPaIhVs1bRcc= smithi028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBMGD8ZDswDSqOs0qmNxS6fFMcpqZhrjRjpde0qAFVfctzs4Q03rGA9Ia2s82d6gFMC/XQRgqx1XCIvBG27Icg= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-19_08:49:34 tube: smithi user: yuvalif verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T11:12:47.861 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_08af21e0d16078b88ddf808fe9ba2ab84dbe6754/qa; will attempt to use it 2024-09-19T11:12:47.861 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_08af21e0d16078b88ddf808fe9ba2ab84dbe6754/qa/tasks 2024-09-19T11:12:47.862 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T11:12:47.863 INFO:teuthology.task.internal:Checking packages... 2024-09-19T11:12:47.883 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '08af21e0d16078b88ddf808fe9ba2ab84dbe6754' 2024-09-19T11:12:47.883 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T11:12:47.883 INFO:teuthology.packaging:ref: None 2024-09-19T11:12:47.883 INFO:teuthology.packaging:tag: None 2024-09-19T11:12:47.884 INFO:teuthology.packaging:branch: wip-yuval-test 2024-09-19T11:12:47.884 INFO:teuthology.packaging:sha1: 08af21e0d16078b88ddf808fe9ba2ab84dbe6754 2024-09-19T11:12:47.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuval-test 2024-09-19T11:12:48.030 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5012.g08af21e0 2024-09-19T11:12:48.031 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T11:12:48.042 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T11:12:48.042 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T11:12:48.046 INFO:teuthology.task.internal:Saving configuration 2024-09-19T11:12:48.059 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T11:12:48.065 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T11:12:48.092 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 11:08:07.395588', 'locked_by': 'scheduled_yuvalif@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBImOkcQMyTTsxZiTVVPdRrkNVGDdXuiUSCHaj35l0O1g/LEZt8LQeLO0yDuqCM2XEh54Yi6Np2VyPaIhVs1bRcc='} 2024-09-19T11:12:48.123 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 11:08:07.397196', 'locked_by': 'scheduled_yuvalif@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBMGD8ZDswDSqOs0qmNxS6fFMcpqZhrjRjpde0qAFVfctzs4Q03rGA9Ia2s82d6gFMC/XQRgqx1XCIvBG27Icg='} 2024-09-19T11:12:48.123 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T11:12:48.129 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-09-19T11:12:48.129 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-09-19T11:12:48.129 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T11:12:48.203 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb264c1bac0>, signals=[15]) 2024-09-19T11:12:48.203 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T11:12:48.209 INFO:teuthology.task.internal:Opening connections... 2024-09-19T11:12:48.209 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T11:12:48.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:12:48.291 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:12:48.292 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:12:48.375 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T11:12:48.381 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-09-19T11:12:48.400 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-09-19T11:12:48.400 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-09-19T11:12:48.457 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-09-19T11:12:48.457 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-09-19T11:12:48.457 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-09-19T11:12:48.457 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T11:12:48.458 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T11:12:48.459 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-09-19T11:12:48.490 DEBUG:teuthology.orchestra.run.smithi028:> uname -m 2024-09-19T11:12:48.508 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2024-09-19T11:12:48.508 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="9" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="9" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el9" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:LOGO="fedora-logo-icon" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/" 2024-09-19T11:12:48.566 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T11:12:48.567 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T11:12:48.567 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T11:12:48.567 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2024-09-19T11:12:48.598 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T11:12:48.606 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T11:12:48.610 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T11:12:48.611 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-09-19T11:12:48.613 DEBUG:teuthology.orchestra.run.smithi028:> test '!' -e /home/ubuntu/cephtest 2024-09-19T11:12:48.630 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T11:12:48.636 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T11:12:48.637 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-09-19T11:12:48.670 DEBUG:teuthology.orchestra.run.smithi028:> test -z $(ls -A /var/lib/ceph) 2024-09-19T11:12:48.696 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T11:12:48.769 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T11:12:48.782 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T11:12:48.782 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T11:12:48.782 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T11:12:48.783 DEBUG:teuthology.orchestra.run.smithi028:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T11:12:48.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:12:48.800 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-19T11:12:48.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:12:48.801 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-09-19T11:12:48.857 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-503.el9.x86_64 2024-09-19T11:12:48.857 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.14.0-503.el9.x86_64 2024-09-19T11:12:48.858 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2024-09-19T11:12:48.860 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-503.el9.x86_64 2024-09-19T11:12:48.860 INFO:teuthology.task.kernel:Running kernel on smithi028: 5.14.0-503.el9.x86_64 2024-09-19T11:12:48.860 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum install -y kernel 2024-09-19T11:12:51.503 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.3 MB 00:00 2024-09-19T11:12:51.998 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.3 MB 00:01 2024-09-19T11:12:53.304 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-19T11:12:54.541 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 14 MB/s | 20 MB 00:01 2024-09-19T11:13:00.025 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-19T11:13:01.700 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-19T11:13:02.880 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-19T11:13:08.577 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-19T11:13:10.525 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T11:13:10.526 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T11:13:10.623 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:13:10.628 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:13:10.628 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-09-19T11:13:10.628 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:13:10.628 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:13:10.628 INFO:teuthology.orchestra.run.smithi028.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T11:13:10.628 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout:Install 4 Packages 2024-09-19T11:13:10.629 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:13:10.630 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 79 M 2024-09-19T11:13:10.630 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 126 M 2024-09-19T11:13:10.630 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:13:10.837 INFO:teuthology.orchestra.run.smithi028.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 238 kB/s | 36 kB 00:00 2024-09-19T11:13:11.161 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 479 kB/s | 6.5 MB 00:13 2024-09-19T11:13:11.471 INFO:teuthology.orchestra.run.smithi028.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-19T11:13:12.079 INFO:teuthology.orchestra.run.smithi028.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-19T11:13:12.396 INFO:teuthology.orchestra.run.smithi028.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-19T11:13:12.396 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:13:12.397 INFO:teuthology.orchestra.run.smithi028.stdout:Total 45 MB/s | 79 MB 00:01 2024-09-19T11:13:13.109 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:13:13.445 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-19T11:13:13.490 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:13:13.491 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:13:14.382 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-19T11:13:14.471 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:13:14.472 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:13:17.064 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:13:17.732 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T11:13:17.814 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T11:13:20.564 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T11:13:20.690 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T11:13:21.754 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 959 B/s | 1.7 kB 00:01 2024-09-19T11:13:23.769 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T11:13:23.769 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T11:13:23.878 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T11:13:23.888 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:13:23.889 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:13:23.889 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:13:23.889 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-09-19T11:13:23.889 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:13:23.889 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 79 M 2024-09-19T11:13:23.889 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 126 M 2024-09-19T11:13:23.889 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:13:24.114 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 341 kB/s | 36 kB 00:00 2024-09-19T11:13:24.498 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-19T11:13:24.923 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-19T11:13:25.240 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-19T11:13:25.241 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:13:25.241 INFO:teuthology.orchestra.run.smithi022.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-19T11:13:25.814 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T11:13:25.858 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:13:25.948 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:13:26.318 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:13:26.318 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:13:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:13:27.290 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:13:29.822 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:13:30.495 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T11:13:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T11:13:30.994 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:13:33.331 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T11:13:33.454 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T11:13:38.588 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T11:13:38.629 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:13:43.763 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:09.906 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:09.947 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:10.278 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:10.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T11:14:10.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T11:14:10.278 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T11:14:11.133 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:11.133 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:14:11.133 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:14:11.134 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T11:14:11.134 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T11:14:11.134 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T11:14:11.134 INFO:teuthology.orchestra.run.smithi028.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T11:14:11.134 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:14:11.134 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:14:11.491 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T11:14:11.586 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T11:14:11.586 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T11:14:11.586 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T11:14:11.586 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T11:14:11.587 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T11:14:11.587 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-19T11:14:11.587 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi028.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T11:14:11.587 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T11:14:11.646 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T11:14:11.646 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T11:14:11.646 DEBUG:teuthology.orchestra.run.smithi028:> sudo rpm -qi grub2-tools 2024-09-19T11:14:11.686 INFO:teuthology.orchestra.run.smithi028.stdout:Name : grub2-tools 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Epoch : 1 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Version : 2.06 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Release : 82.el9 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Architecture: x86_64 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Group : Unspecified 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Size : 8274670 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:License : GPLv3+ 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Packager : builder@centos.org 2024-09-19T11:14:11.687 INFO:teuthology.orchestra.run.smithi028.stdout:Vendor : CentOS 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:Summary : Support tools for GRUB. 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:Description : 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:hardware devices. 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:14:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T11:14:11.689 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T11:14:11.689 INFO:teuthology.task.kernel:Updating grub on smithi028 to boot 5.14.0-508.el9.x86_64 2024-09-19T11:14:11.689 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T11:14:12.382 INFO:teuthology.orchestra.run.smithi028.stderr:Generating grub configuration file ... 2024-09-19T11:14:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T11:14:13.754 INFO:teuthology.orchestra.run.smithi028.stderr:done 2024-09-19T11:14:13.756 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-09-19T11:14:13.771 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.LRLBqIEHtz 2024-09-19T11:14:13.771 DEBUG:teuthology.orchestra.run.smithi028:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LRLBqIEHtz 2024-09-19T11:14:13.838 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.LRLBqIEHtz 2024-09-19T11:14:13.980 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.LRLBqIEHtz is 6KB 2024-09-19T11:14:14.028 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.LRLBqIEHtz 2024-09-19T11:14:14.043 DEBUG:teuthology.orchestra.run.smithi028:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T11:14:14.104 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T11:14:14.105 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T11:14:14.105 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T11:14:14.105 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T11:14:14.106 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T11:14:14.260 DEBUG:teuthology.orchestra.run.smithi028:> sudo shutdown -r now 2024-09-19T11:14:22.531 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:22.571 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:22.928 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:22.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T11:14:22.928 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T11:14:22.929 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T11:14:23.808 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T11:14:23.808 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:14:23.808 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:14:23.808 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T11:14:23.808 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T11:14:23.809 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T11:14:23.809 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T11:14:23.809 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:14:23.809 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:14:24.154 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T11:14:24.236 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T11:14:24.236 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T11:14:24.236 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T11:14:24.236 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T11:14:24.236 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T11:14:24.237 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T11:14:24.237 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T11:14:24.237 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T11:14:24.296 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T11:14:24.296 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T11:14:24.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-09-19T11:14:24.327 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-09-19T11:14:24.327 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 82.el9 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8274670 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-09-19T11:14:24.328 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:14:24.329 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T11:14:24.330 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T11:14:24.330 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 5.14.0-508.el9.x86_64 2024-09-19T11:14:24.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T11:14:24.959 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-09-19T11:14:26.199 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T11:14:26.240 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-09-19T11:14:26.242 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-09-19T11:14:26.257 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.0LKOAP3jPN 2024-09-19T11:14:26.258 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0LKOAP3jPN 2024-09-19T11:14:26.323 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.0LKOAP3jPN 2024-09-19T11:14:26.460 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.0LKOAP3jPN is 6KB 2024-09-19T11:14:26.508 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.0LKOAP3jPN 2024-09-19T11:14:26.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T11:14:26.594 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T11:14:26.594 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T11:14:26.594 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T11:14:26.594 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T11:14:26.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T11:14:26.752 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-09-19T11:14:44.279 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-19T11:14:44.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T11:14:44.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:14:56.765 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T11:14:56.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T11:14:56.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:15:44.337 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T11:15:53.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T11:15:53.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:15:53.731 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-09-19T11:15:54.088 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T11:15:54.088 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-508.el9.x86_64"... 2024-09-19T11:15:54.088 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-09-19T11:15:54.105 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-508.el9.x86_64 2024-09-19T11:15:54.105 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T11:15:54.105 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T11:15:54.105 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T11:15:55.106 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-19T11:15:55.107 DEBUG:teuthology.orchestra.run.smithi028:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T11:15:55.165 INFO:teuthology.orchestra.run.smithi028.stdout:ttyS1 2024-09-19T11:15:55.190 DEBUG:teuthology.parallel:result is None 2024-09-19T11:15:56.778 DEBUG:teuthology.orchestra.remote:timed out 2024-09-19T11:16:05.779 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T11:16:05.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:16:06.119 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-19T11:16:06.475 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T11:16:06.475 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-19T11:16:06.476 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-19T11:16:06.490 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-508.el9.x86_64 2024-09-19T11:16:06.491 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T11:16:06.491 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T11:16:06.491 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T11:16:07.492 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T11:16:07.492 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T11:16:07.553 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2024-09-19T11:16:07.580 DEBUG:teuthology.parallel:result is None 2024-09-19T11:16:07.581 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T11:16:07.588 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T11:16:07.588 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T11:16:07.590 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T11:16:07.610 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T11:16:07.616 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T11:16:07.627 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T11:16:07.628 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T11:16:07.647 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T11:16:07.681 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T11:16:07.688 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T11:16:07.688 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T11:16:07.707 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T11:16:07.735 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T11:16:07.746 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T11:16:07.754 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T11:16:07.766 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T11:16:07.768 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T11:16:07.775 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T11:16:07.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T11:16:07.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T11:16:07.836 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T11:16:07.845 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T11:16:07.845 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T11:16:07.850 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T11:16:07.894 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T11:16:07.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T11:16:07.989 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T11:16:08.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T11:16:08.106 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:16:08.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T11:16:08.168 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T11:16:08.220 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T11:16:08.267 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T11:16:08.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T11:16:08.397 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:16:08.397 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T11:16:08.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-09-19T11:16:08.468 DEBUG:teuthology.orchestra.run.smithi028:> sudo service rsyslog restart 2024-09-19T11:16:08.519 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T11:16:08.556 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T11:16:09.081 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T11:16:09.089 INFO:teuthology.task.internal:Starting timer... 2024-09-19T11:16:09.089 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T11:16:09.099 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T11:16:09.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-09-19T11:16:09.197 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-09-19T11:16:09.199 DEBUG:teuthology.orchestra.run.smithi028:> sudo service auditd rotate 2024-09-19T11:16:09.246 INFO:teuthology.orchestra.run.smithi028.stdout:Rotating logs: 2024-09-19T11:16:09.248 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T11:16:09.249 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-09-19T11:16:09.269 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-09-19T11:16:09.269 DEBUG:teuthology.orchestra.run.smithi028:> /usr/sbin/getenforce 2024-09-19T11:16:09.294 INFO:teuthology.orchestra.run.smithi028.stdout:Permissive 2024-09-19T11:16:09.294 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi028.front.sepia.ceph.com': 'permissive'} 2024-09-19T11:16:09.294 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-19T11:16:09.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:16:09.333 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-19T11:16:09.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:16:09.363 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T11:16:09.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-09-19T11:16:09.400 DEBUG:teuthology.orchestra.run.smithi028:> sudo /usr/sbin/setenforce permissive 2024-09-19T11:16:09.437 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T11:16:09.446 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T11:16:09.458 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T11:16:09.459 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi028.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T11:19:59.148 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi028.front.sepia.ceph.com')] 2024-09-19T11:19:59.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T11:19:59.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:19:59.238 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-19T11:19:59.316 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-19T11:19:59.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T11:19:59.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T11:19:59.404 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-09-19T11:19:59.445 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-19T11:19:59.446 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T11:19:59.457 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T11:19:59.457 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T11:19:59.457 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T11:19:59.459 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T11:19:59.460 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T11:19:59.488 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T11:19:59.503 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T11:19:59.516 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T11:19:59.527 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-09-19T11:19:59.532 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T11:19:59.539 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-09-19T11:19:59.554 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T11:19:59.558 INFO:teuthology.orchestra.run.smithi028.stderr:sudo: ntpd: command not found 2024-09-19T11:19:59.568 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T11:19:59.570 INFO:teuthology.orchestra.run.smithi028.stdout:506 Cannot talk to daemon 2024-09-19T11:19:59.585 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T11:19:59.600 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T11:19:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-19T11:19:59.632 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T11:19:59.632 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-19T11:19:59.632 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.632 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.633 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.633 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-09-19T11:19:59.661 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T11:19:59.661 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-09-19T11:19:59.661 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.661 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.661 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.661 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:19:59.661 INFO:teuthology.run_tasks:Running task install... 2024-09-19T11:19:59.670 DEBUG:teuthology.task.install:project ceph 2024-09-19T11:19:59.670 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754'}} 2024-09-19T11:19:59.670 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754'} 2024-09-19T11:19:59.670 INFO:teuthology.task.install:Using flavor: default 2024-09-19T11:19:59.676 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T11:19:59.676 INFO:teuthology.task.install:extra packages: [] 2024-09-19T11:19:59.676 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754', 'tag': None, 'wait_for_package': False} 2024-09-19T11:19:59.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08af21e0d16078b88ddf808fe9ba2ab84dbe6754 2024-09-19T11:19:59.679 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754', 'tag': None, 'wait_for_package': False} 2024-09-19T11:19:59.679 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08af21e0d16078b88ddf808fe9ba2ab84dbe6754 2024-09-19T11:19:59.835 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/ 2024-09-19T11:19:59.835 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5012.g08af21e0 2024-09-19T11:19:59.848 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/ 2024-09-19T11:19:59.849 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5012.g08af21e0 2024-09-19T11:19:59.955 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T11:19:59.955 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:19:59.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T11:19:59.969 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuval-test/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T11:19:59.969 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:19:59.969 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T11:19:59.981 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-19T11:19:59.982 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T11:19:59.997 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-19T11:19:59.997 DEBUG:teuthology.orchestra.run.smithi028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08af21e0d16078b88ddf808fe9ba2ab84dbe6754/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T11:20:00.050 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T11:20:00.065 DEBUG:teuthology.orchestra.run.smithi028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T11:20:00.128 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T11:20:00.143 DEBUG:teuthology.orchestra.run.smithi028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T11:20:00.195 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-09-19T11:20:00.197 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-09-19T11:20:00.209 INFO:teuthology.orchestra.run.smithi028.stdout:check_obsoletes = 1 2024-09-19T11:20:00.211 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum clean all 2024-09-19T11:20:00.438 INFO:teuthology.orchestra.run.smithi022.stdout:45 files removed 2024-09-19T11:20:00.449 INFO:teuthology.orchestra.run.smithi028.stdout:45 files removed 2024-09-19T11:20:00.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw 2024-09-19T11:20:00.476 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-radosgw 2024-09-19T11:20:01.036 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-19T11:20:01.045 INFO:teuthology.orchestra.run.smithi028.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-19T11:20:01.328 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-19T11:20:01.345 INFO:teuthology.orchestra.run.smithi028.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-19T11:20:01.612 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T11:20:01.629 INFO:teuthology.orchestra.run.smithi028.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T11:20:02.062 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-19T11:20:02.221 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-19T11:20:03.771 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-19T11:20:05.356 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-19T11:20:09.914 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-19T11:20:09.965 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-19T11:20:11.507 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-19T11:20:11.639 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-19T11:20:13.120 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-19T11:20:13.436 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-19T11:20:18.751 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-19T11:20:19.033 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-19T11:20:21.054 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 11 M 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrading: 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: librados2 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 3.5 M 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 3.2 M 2024-09-19T11:20:21.065 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 5.5 M 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 22 M 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 25 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 755 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 509 k 2024-09-19T11:20:21.066 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 5.5 M 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 45 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 148 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 161 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 321 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 307 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 99 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T11:20:21.067 INFO:teuthology.orchestra.run.smithi028.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout:Installing weak dependencies: 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout:Install 31 Packages 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrade 2 Packages 2024-09-19T11:20:21.068 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:21.069 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 62 M 2024-09-19T11:20:21.069 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:20:21.354 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:20:21.370 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:20:21.370 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:20:21.370 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:20:21.370 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:20:21.370 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 11 M 2024-09-19T11:20:21.370 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 3.5 M 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 3.2 M 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 5.5 M 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 22 M 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 25 k 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 755 k 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-19T11:20:21.371 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 509 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 5.5 M 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 45 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 148 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 161 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T11:20:21.372 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 321 k 2024-09-19T11:20:21.373 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 307 k 2024-09-19T11:20:21.373 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 99 k 2024-09-19T11:20:21.373 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T11:20:21.373 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T11:20:21.373 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T11:20:21.373 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T11:20:21.373 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-09-19T11:20:21.374 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T11:20:21.374 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:21.374 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:20:21.374 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:20:21.374 INFO:teuthology.orchestra.run.smithi022.stdout:Install 31 Packages 2024-09-19T11:20:21.374 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 2 Packages 2024-09-19T11:20:21.374 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:21.375 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 62 M 2024-09-19T11:20:21.375 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:20:22.277 INFO:teuthology.orchestra.run.smithi028.stdout:(1/33): ceph-base-19.3.0-5012.g08af21e0.el9.x86 7.9 MB/s | 5.5 MB 00:00 2024-09-19T11:20:22.336 INFO:teuthology.orchestra.run.smithi028.stdout:(2/33): ceph-selinux-19.3.0-5012.g08af21e0.el9. 433 kB/s | 25 kB 00:00 2024-09-19T11:20:22.485 INFO:teuthology.orchestra.run.smithi028.stdout:(3/33): libcephfs2-19.3.0-5012.g08af21e0.el9.x8 4.9 MB/s | 755 kB 00:00 2024-09-19T11:20:22.586 INFO:teuthology.orchestra.run.smithi028.stdout:(4/33): libradosstriper1-19.3.0-5012.g08af21e0. 5.0 MB/s | 509 kB 00:00 2024-09-19T11:20:22.709 INFO:teuthology.orchestra.run.smithi022.stdout:(1/33): ceph-base-19.3.0-5012.g08af21e0.el9.x86 5.1 MB/s | 5.5 MB 00:01 2024-09-19T11:20:22.751 INFO:teuthology.orchestra.run.smithi022.stdout:(2/33): ceph-selinux-19.3.0-5012.g08af21e0.el9. 606 kB/s | 25 kB 00:00 2024-09-19T11:20:22.786 INFO:teuthology.orchestra.run.smithi028.stdout:(5/33): ceph-common-19.3.0-5012.g08af21e0.el9.x 18 MB/s | 22 MB 00:01 2024-09-19T11:20:22.834 INFO:teuthology.orchestra.run.smithi022.stdout:(3/33): libcephfs2-19.3.0-5012.g08af21e0.el9.x8 8.8 MB/s | 755 kB 00:00 2024-09-19T11:20:22.885 INFO:teuthology.orchestra.run.smithi022.stdout:(4/33): libradosstriper1-19.3.0-5012.g08af21e0. 10 MB/s | 509 kB 00:00 2024-09-19T11:20:22.936 INFO:teuthology.orchestra.run.smithi028.stdout:(6/33): ceph-radosgw-19.3.0-5012.g08af21e0.el9. 8.0 MB/s | 11 MB 00:01 2024-09-19T11:20:22.962 INFO:teuthology.orchestra.run.smithi028.stdout:(7/33): python3-ceph-argparse-19.3.0-5012.g08af 258 kB/s | 45 kB 00:00 2024-09-19T11:20:22.995 INFO:teuthology.orchestra.run.smithi028.stdout:(8/33): python3-ceph-common-19.3.0-5012.g08af21 2.5 MB/s | 148 kB 00:00 2024-09-19T11:20:23.020 INFO:teuthology.orchestra.run.smithi028.stdout:(9/33): python3-cephfs-19.3.0-5012.g08af21e0.el 2.7 MB/s | 161 kB 00:00 2024-09-19T11:20:23.045 INFO:teuthology.orchestra.run.smithi028.stdout:(10/33): python3-rados-19.3.0-5012.g08af21e0.el 6.3 MB/s | 321 kB 00:00 2024-09-19T11:20:23.071 INFO:teuthology.orchestra.run.smithi028.stdout:(11/33): python3-rbd-19.3.0-5012.g08af21e0.el9. 5.9 MB/s | 307 kB 00:00 2024-09-19T11:20:23.146 INFO:teuthology.orchestra.run.smithi028.stdout:(12/33): librgw2-19.3.0-5012.g08af21e0.el9.x86_ 9.8 MB/s | 5.5 MB 00:00 2024-09-19T11:20:23.180 INFO:teuthology.orchestra.run.smithi028.stdout:(13/33): python3-rgw-19.3.0-5012.g08af21e0.el9. 741 kB/s | 99 kB 00:00 2024-09-19T11:20:23.193 INFO:teuthology.orchestra.run.smithi022.stdout:(5/33): librgw2-19.3.0-5012.g08af21e0.el9.x86_6 18 MB/s | 5.5 MB 00:00 2024-09-19T11:20:23.235 INFO:teuthology.orchestra.run.smithi022.stdout:(6/33): python3-ceph-argparse-19.3.0-5012.g08af 1.1 MB/s | 45 kB 00:00 2024-09-19T11:20:23.272 INFO:teuthology.orchestra.run.smithi028.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 532 kB/s | 107 kB 00:00 2024-09-19T11:20:23.322 INFO:teuthology.orchestra.run.smithi028.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 970 kB/s | 171 kB 00:00 2024-09-19T11:20:23.347 INFO:teuthology.orchestra.run.smithi028.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-19T11:20:23.352 INFO:teuthology.orchestra.run.smithi022.stdout:(7/33): ceph-radosgw-19.3.0-5012.g08af21e0.el9. 6.3 MB/s | 11 MB 00:01 2024-09-19T11:20:23.373 INFO:teuthology.orchestra.run.smithi028.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 833 kB/s | 160 kB 00:00 2024-09-19T11:20:23.377 INFO:teuthology.orchestra.run.smithi022.stdout:(8/33): python3-ceph-common-19.3.0-5012.g08af21 1.0 MB/s | 148 kB 00:00 2024-09-19T11:20:23.403 INFO:teuthology.orchestra.run.smithi022.stdout:(9/33): python3-cephfs-19.3.0-5012.g08af21e0.el 3.2 MB/s | 161 kB 00:00 2024-09-19T11:20:23.423 INFO:teuthology.orchestra.run.smithi028.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-19T11:20:23.428 INFO:teuthology.orchestra.run.smithi022.stdout:(10/33): python3-rados-19.3.0-5012.g08af21e0.el 6.2 MB/s | 321 kB 00:00 2024-09-19T11:20:23.448 INFO:teuthology.orchestra.run.smithi028.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 561 kB/s | 42 kB 00:00 2024-09-19T11:20:23.453 INFO:teuthology.orchestra.run.smithi022.stdout:(11/33): python3-rbd-19.3.0-5012.g08af21e0.el9. 6.0 MB/s | 307 kB 00:00 2024-09-19T11:20:23.479 INFO:teuthology.orchestra.run.smithi022.stdout:(12/33): python3-rgw-19.3.0-5012.g08af21e0.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T11:20:23.490 INFO:teuthology.orchestra.run.smithi028.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-19T11:20:23.524 INFO:teuthology.orchestra.run.smithi028.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-09-19T11:20:23.549 INFO:teuthology.orchestra.run.smithi028.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 358 kB/s | 36 kB 00:00 2024-09-19T11:20:23.574 INFO:teuthology.orchestra.run.smithi028.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-19T11:20:23.596 INFO:teuthology.orchestra.run.smithi022.stdout:(13/33): boost-program-options-1.75.0-8.el9.x86 753 kB/s | 107 kB 00:00 2024-09-19T11:20:23.621 INFO:teuthology.orchestra.run.smithi022.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.2 MB/s | 171 kB 00:00 2024-09-19T11:20:23.641 INFO:teuthology.orchestra.run.smithi028.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-19T11:20:23.675 INFO:teuthology.orchestra.run.smithi028.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-19T11:20:23.688 INFO:teuthology.orchestra.run.smithi022.stdout:(15/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-19T11:20:23.691 INFO:teuthology.orchestra.run.smithi028.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 255 kB/s | 26 kB 00:00 2024-09-19T11:20:23.713 INFO:teuthology.orchestra.run.smithi022.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-09-19T11:20:23.717 INFO:teuthology.orchestra.run.smithi028.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-19T11:20:23.751 INFO:teuthology.orchestra.run.smithi028.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-19T11:20:23.788 INFO:teuthology.orchestra.run.smithi022.stdout:(17/33): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-09-19T11:20:23.814 INFO:teuthology.orchestra.run.smithi022.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-19T11:20:23.826 INFO:teuthology.orchestra.run.smithi028.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-19T11:20:23.864 INFO:teuthology.orchestra.run.smithi022.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 560 kB/s | 42 kB 00:00 2024-09-19T11:20:23.876 INFO:teuthology.orchestra.run.smithi028.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-19T11:20:23.889 INFO:teuthology.orchestra.run.smithi022.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-19T11:20:23.924 INFO:teuthology.orchestra.run.smithi022.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 611 kB/s | 36 kB 00:00 2024-09-19T11:20:23.965 INFO:teuthology.orchestra.run.smithi022.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-19T11:20:23.978 INFO:teuthology.orchestra.run.smithi028.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-19T11:20:23.999 INFO:teuthology.orchestra.run.smithi022.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-19T11:20:24.024 INFO:teuthology.orchestra.run.smithi022.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-19T11:20:24.049 INFO:teuthology.orchestra.run.smithi022.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T11:20:24.075 INFO:teuthology.orchestra.run.smithi022.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T11:20:24.110 INFO:teuthology.orchestra.run.smithi028.stdout:(32/33): librados2-19.3.0-5012.g08af21e0.el9.x8 12 MB/s | 3.5 MB 00:00 2024-09-19T11:20:24.150 INFO:teuthology.orchestra.run.smithi022.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-19T11:20:24.169 INFO:teuthology.orchestra.run.smithi028.stdout:(33/33): librbd1-19.3.0-5012.g08af21e0.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-19T11:20:24.173 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:20:24.173 INFO:teuthology.orchestra.run.smithi028.stdout:Total 20 MB/s | 62 MB 00:03 2024-09-19T11:20:24.183 INFO:teuthology.orchestra.run.smithi022.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-19T11:20:24.267 INFO:teuthology.orchestra.run.smithi022.stdout:(29/33): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-19T11:20:24.286 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:20:24.326 INFO:teuthology.orchestra.run.smithi022.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-19T11:20:24.326 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:20:24.327 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:20:24.526 INFO:teuthology.orchestra.run.smithi022.stdout:(31/33): librados2-19.3.0-5012.g08af21e0.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-19T11:20:24.740 INFO:teuthology.orchestra.run.smithi022.stdout:(32/33): librbd1-19.3.0-5012.g08af21e0.el9.x86_ 7.9 MB/s | 3.2 MB 00:00 2024-09-19T11:20:25.052 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:20:25.053 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:20:25.268 INFO:teuthology.orchestra.run.smithi022.stdout:(33/33): ceph-common-19.3.0-5012.g08af21e0.el9. 5.9 MB/s | 22 MB 00:03 2024-09-19T11:20:25.274 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:20:25.274 INFO:teuthology.orchestra.run.smithi022.stdout:Total 16 MB/s | 62 MB 00:03 2024-09-19T11:20:25.391 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:20:25.434 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:20:25.434 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:20:26.159 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:20:26.159 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:20:26.600 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:20:26.686 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-19T11:20:26.715 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-19T11:20:26.741 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-19T11:20:26.765 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-19T11:20:26.795 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-19T11:20:26.995 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-19T11:20:27.055 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-19T11:20:27.455 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-19T11:20:27.519 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 7/35 2024-09-19T11:20:27.595 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 7/35 2024-09-19T11:20:27.685 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 8/35 2024-09-19T11:20:27.712 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 9/35 2024-09-19T11:20:27.816 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 9/35 2024-09-19T11:20:27.845 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-19T11:20:27.858 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:20:27.889 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-argparse-2:19.3.0-5012.g08af21e0.el9. 11/35 2024-09-19T11:20:27.936 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-19T11:20:27.959 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 12/35 2024-09-19T11:20:27.973 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-19T11:20:27.989 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_6 13/35 2024-09-19T11:20:27.998 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-19T11:20:28.021 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-19T11:20:28.050 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-19T11:20:28.053 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_6 13/35 2024-09-19T11:20:28.088 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-19T11:20:28.199 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-19T11:20:28.199 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-19T11:20:28.239 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-19T11:20:28.252 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-19T11:20:28.294 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-19T11:20:28.328 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-19T11:20:28.358 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-19T11:20:28.390 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-19T11:20:28.428 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-19T11:20:28.664 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-19T11:20:28.725 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 7/35 2024-09-19T11:20:28.764 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-19T11:20:28.812 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 23/35 2024-09-19T11:20:28.844 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 7/35 2024-09-19T11:20:28.904 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 23/35 2024-09-19T11:20:28.933 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 24/35 2024-09-19T11:20:28.934 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 8/35 2024-09-19T11:20:28.968 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 9/35 2024-09-19T11:20:28.969 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-19T11:20:29.031 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 9/35 2024-09-19T11:20:29.069 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-19T11:20:29.113 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:19.3.0-5012.g08af21e0.el9. 11/35 2024-09-19T11:20:29.183 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 12/35 2024-09-19T11:20:29.212 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_6 13/35 2024-09-19T11:20:29.277 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_6 13/35 2024-09-19T11:20:29.311 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-19T11:20:29.422 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-19T11:20:29.462 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-19T11:20:29.487 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-19T11:20:29.510 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-19T11:20:29.522 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 27/35 2024-09-19T11:20:29.544 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-19T11:20:29.573 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-19T11:20:29.594 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 27/35 2024-09-19T11:20:29.605 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-19T11:20:29.643 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-19T11:20:29.687 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 28/35 2024-09-19T11:20:29.725 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-common-2:19.3.0-5012.g08af21e0.el9.x8 29/35 2024-09-19T11:20:29.988 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-19T11:20:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 23/35 2024-09-19T11:20:30.128 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 23/35 2024-09-19T11:20:30.157 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 24/35 2024-09-19T11:20:30.194 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-19T11:20:30.712 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-19T11:20:30.755 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 27/35 2024-09-19T11:20:30.818 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 27/35 2024-09-19T11:20:30.905 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 28/35 2024-09-19T11:20:30.933 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:19.3.0-5012.g08af21e0.el9.x8 29/35 2024-09-19T11:20:32.142 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 30/35 2024-09-19T11:20:32.196 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 30/35 2024-09-19T11:20:32.793 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 30/35 2024-09-19T11:20:32.832 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 31/35 2024-09-19T11:20:32.908 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 31/35 2024-09-19T11:20:32.908 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T11:20:32.908 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T11:20:32.908 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:32.950 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 32/35 2024-09-19T11:20:33.369 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 30/35 2024-09-19T11:20:33.404 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 30/35 2024-09-19T11:20:33.994 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 30/35 2024-09-19T11:20:34.033 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 31/35 2024-09-19T11:20:34.134 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 31/35 2024-09-19T11:20:34.134 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T11:20:34.134 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T11:20:34.134 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:34.178 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 32/35 2024-09-19T11:20:43.328 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 32/35 2024-09-19T11:20:43.328 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /sys 2024-09-19T11:20:43.329 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /proc 2024-09-19T11:20:43.329 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /mnt 2024-09-19T11:20:43.329 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /var/tmp 2024-09-19T11:20:43.329 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /home 2024-09-19T11:20:43.329 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /root 2024-09-19T11:20:43.329 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /tmp 2024-09-19T11:20:43.329 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:44.380 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 33/35 2024-09-19T11:20:44.406 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 33/35 2024-09-19T11:20:44.406 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:20:44.406 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T11:20:44.406 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T11:20:44.406 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T11:20:44.407 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:44.408 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-19T11:20:44.482 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-19T11:20:44.482 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-19T11:20:44.824 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 32/35 2024-09-19T11:20:44.824 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-09-19T11:20:44.824 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-09-19T11:20:44.824 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-09-19T11:20:44.824 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-09-19T11:20:44.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-09-19T11:20:44.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-09-19T11:20:44.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-09-19T11:20:44.825 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-19T11:20:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 1/35 2024-09-19T11:20:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 2/35 2024-09-19T11:20:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 3/35 2024-09-19T11:20:45.662 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 4/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 5/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_6 6/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 7/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5012.g08af21e0.el9. 8/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-common-2:19.3.0-5012.g08af21e0.el9.x8 9/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 10/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 11/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 12/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 13/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-19T11:20:45.663 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-19T11:20:45.664 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-19T11:20:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-19T11:20:45.665 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-19T11:20:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 32/35 2024-09-19T11:20:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-19T11:20:45.666 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 34/35 2024-09-19T11:20:45.861 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 33/35 2024-09-19T11:20:45.887 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 33/35 2024-09-19T11:20:45.887 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:20:45.887 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T11:20:45.887 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T11:20:45.888 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T11:20:45.888 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:45.889 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-19T11:20:45.963 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-19T11:20:45.963 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-19T11:20:46.688 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-19T11:20:46.689 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:46.689 INFO:teuthology.orchestra.run.smithi028.stdout:Upgraded: 2024-09-19T11:20:46.689 INFO:teuthology.orchestra.run.smithi028.stdout: librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.689 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.689 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:20:46.690 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T11:20:46.690 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.690 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.690 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.690 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.691 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T11:20:46.691 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T11:20:46.691 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T11:20:46.691 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.691 INFO:teuthology.orchestra.run.smithi028.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-19T11:20:46.691 INFO:teuthology.orchestra.run.smithi028.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T11:20:46.692 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T11:20:46.692 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T11:20:46.692 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.692 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T11:20:46.692 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T11:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T11:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T11:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.693 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T11:20:46.694 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.694 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.694 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:46.694 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T11:20:46.694 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T11:20:46.694 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T11:20:46.695 INFO:teuthology.orchestra.run.smithi028.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T11:20:46.695 INFO:teuthology.orchestra.run.smithi028.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T11:20:46.695 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:46.695 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:20:47.018 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-test 2024-09-19T11:20:47.196 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-19T11:20:47.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 1/35 2024-09-19T11:20:47.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 2/35 2024-09-19T11:20:47.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 3/35 2024-09-19T11:20:47.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 4/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 5/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_6 6/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 7/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5012.g08af21e0.el9. 8/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:19.3.0-5012.g08af21e0.el9.x8 9/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 10/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 11/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 12/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 13/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-19T11:20:47.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-19T11:20:47.198 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-19T11:20:47.199 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-19T11:20:47.199 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 32/35 2024-09-19T11:20:47.199 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-19T11:20:47.199 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 34/35 2024-09-19T11:20:47.627 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:29 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:20:47.778 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 46 M 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 161 k 2024-09-19T11:20:47.779 INFO:teuthology.orchestra.run.smithi028.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T11:20:47.780 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T11:20:47.780 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:47.780 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:20:47.780 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:47.780 INFO:teuthology.orchestra.run.smithi028.stdout:Install 4 Packages 2024-09-19T11:20:47.780 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:47.780 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 46 M 2024-09-19T11:20:47.781 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 201 M 2024-09-19T11:20:47.781 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:20:48.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.129 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-19T11:20:48.130 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.131 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:48.132 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:20:48.270 INFO:teuthology.orchestra.run.smithi028.stdout:(1/4): libcephsqlite-19.3.0-5012.g08af21e0.el9. 773 kB/s | 161 kB 00:00 2024-09-19T11:20:48.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-test 2024-09-19T11:20:48.645 INFO:teuthology.orchestra.run.smithi028.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-19T11:20:48.695 INFO:teuthology.orchestra.run.smithi028.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 480 kB/s | 304 kB 00:00 2024-09-19T11:20:49.087 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:30 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:20:49.224 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 46 M 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 161 k 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:20:49.225 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:20:49.226 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-09-19T11:20:49.226 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:49.226 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 46 M 2024-09-19T11:20:49.226 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 201 M 2024-09-19T11:20:49.226 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:20:49.377 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-09-19T11:20:49.419 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-19T11:20:49.478 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): libcephsqlite-19.3.0-5012.g08af21e0.el9. 704 kB/s | 161 kB 00:00 2024-09-19T11:20:49.529 INFO:teuthology.orchestra.run.smithi028.stdout:(4/4): ceph-test-19.3.0-5012.g08af21e0.el9.x86_ 31 MB/s | 46 MB 00:01 2024-09-19T11:20:49.531 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:20:49.531 INFO:teuthology.orchestra.run.smithi028.stdout:Total 26 MB/s | 46 MB 00:01 2024-09-19T11:20:49.560 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:20:49.582 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:20:49.582 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:20:49.954 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:20:49.954 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:20:50.494 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:20:50.581 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-19T11:20:50.620 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-19T11:20:50.647 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 3/4 2024-09-19T11:20:50.845 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): ceph-test-19.3.0-5012.g08af21e0.el9.x86_ 29 MB/s | 46 MB 00:01 2024-09-19T11:20:50.847 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:20:50.847 INFO:teuthology.orchestra.run.smithi022.stdout:Total 28 MB/s | 46 MB 00:01 2024-09-19T11:20:50.875 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:20:50.899 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:20:50.899 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:20:51.271 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:20:51.272 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:20:51.809 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:20:51.888 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-19T11:20:51.928 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-19T11:20:51.954 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 3/4 2024-09-19T11:20:55.294 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 3/4 2024-09-19T11:20:55.381 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 4/4 2024-09-19T11:20:56.601 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 3/4 2024-09-19T11:20:56.673 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 4/4 2024-09-19T11:20:57.685 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 4/4 2024-09-19T11:20:57.685 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 1/4 2024-09-19T11:20:57.685 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 2/4 2024-09-19T11:20:57.685 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-19T11:20:58.017 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-19T11:20:58.017 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:58.018 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:20:58.018 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:58.018 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:58.018 INFO:teuthology.orchestra.run.smithi028.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T11:20:58.018 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T11:20:58.018 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:58.018 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:20:58.281 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph 2024-09-19T11:20:58.885 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:40 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:20:59.027 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout: ceph x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 6.6 k 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 2.4 M 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 1.5 M 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 292 k 2024-09-19T11:20:59.030 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 4.8 M 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 17 M 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-19T11:20:59.031 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T11:20:59.032 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:20:59.033 INFO:teuthology.orchestra.run.smithi028.stdout:Install 37 Packages 2024-09-19T11:20:59.034 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:20:59.034 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 31 M 2024-09-19T11:20:59.034 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 109 M 2024-09-19T11:20:59.034 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:20:59.050 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 4/4 2024-09-19T11:20:59.050 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 1/4 2024-09-19T11:20:59.050 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 2/4 2024-09-19T11:20:59.050 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:20:59.361 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:20:59.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph 2024-09-19T11:21:00.120 INFO:teuthology.orchestra.run.smithi028.stdout:(1/37): ceph-19.3.0-5012.g08af21e0.el9.x86_64.r 49 kB/s | 6.6 kB 00:00 2024-09-19T11:21:00.237 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:00.312 INFO:teuthology.orchestra.run.smithi028.stdout:(2/37): ceph-mgr-19.3.0-5012.g08af21e0.el9.x86_ 4.7 MB/s | 1.5 MB 00:00 2024-09-19T11:21:00.363 INFO:teuthology.orchestra.run.smithi028.stdout:(3/37): ceph-mds-19.3.0-5012.g08af21e0.el9.x86_ 6.3 MB/s | 2.4 MB 00:00 2024-09-19T11:21:00.386 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:00.389 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:00.389 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:00.389 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:00.389 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:00.389 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 6.6 k 2024-09-19T11:21:00.389 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:21:00.389 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 2.4 M 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 1.5 M 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 292 k 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 4.8 M 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 17 M 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T11:21:00.390 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T11:21:00.391 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T11:21:00.392 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout:Install 37 Packages 2024-09-19T11:21:00.393 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:00.394 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 31 M 2024-09-19T11:21:00.394 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 109 M 2024-09-19T11:21:00.394 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:00.446 INFO:teuthology.orchestra.run.smithi028.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5012.g08af 3.4 MB/s | 292 kB 00:00 2024-09-19T11:21:00.538 INFO:teuthology.orchestra.run.smithi028.stdout:(5/37): ceph-mon-19.3.0-5012.g08af21e0.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-19T11:21:00.863 INFO:teuthology.orchestra.run.smithi028.stdout:(6/37): ceph-osd-19.3.0-5012.g08af21e0.el9.x86_ 31 MB/s | 17 MB 00:00 2024-09-19T11:21:01.059 INFO:teuthology.orchestra.run.smithi022.stdout:(1/37): ceph-19.3.0-5012.g08af21e0.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-19T11:21:01.218 INFO:teuthology.orchestra.run.smithi022.stdout:(2/37): ceph-mgr-19.3.0-5012.g08af21e0.el9.x86_ 5.2 MB/s | 1.5 MB 00:00 2024-09-19T11:21:01.268 INFO:teuthology.orchestra.run.smithi022.stdout:(3/37): ceph-mds-19.3.0-5012.g08af21e0.el9.x86_ 6.9 MB/s | 2.4 MB 00:00 2024-09-19T11:21:01.338 INFO:teuthology.orchestra.run.smithi028.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 284 kB/s | 253 kB 00:00 2024-09-19T11:21:01.360 INFO:teuthology.orchestra.run.smithi022.stdout:(4/37): ceph-mgr-modules-core-19.3.0-5012.g08af 3.1 MB/s | 292 kB 00:00 2024-09-19T11:21:01.555 INFO:teuthology.orchestra.run.smithi028.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-09-19T11:21:01.568 INFO:teuthology.orchestra.run.smithi022.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-19T11:21:01.694 INFO:teuthology.orchestra.run.smithi022.stdout:(6/37): ceph-mon-19.3.0-5012.g08af21e0.el9.x86_ 7.5 MB/s | 4.8 MB 00:00 2024-09-19T11:21:01.697 INFO:teuthology.orchestra.run.smithi028.stdout:(9/37): python3-cryptography-36.0.1-4.el9.x86_6 1.1 MB/s | 1.2 MB 00:01 2024-09-19T11:21:01.861 INFO:teuthology.orchestra.run.smithi022.stdout:(7/37): ceph-osd-19.3.0-5012.g08af21e0.el9.x86_ 27 MB/s | 17 MB 00:00 2024-09-19T11:21:01.895 INFO:teuthology.orchestra.run.smithi022.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 3.8 MB/s | 1.2 MB 00:00 2024-09-19T11:21:01.945 INFO:teuthology.orchestra.run.smithi022.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-19T11:21:01.995 INFO:teuthology.orchestra.run.smithi022.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 354 kB/s | 106 kB 00:00 2024-09-19T11:21:02.039 INFO:teuthology.orchestra.run.smithi028.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 193 kB/s | 135 kB 00:00 2024-09-19T11:21:02.070 INFO:teuthology.orchestra.run.smithi022.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 646 kB/s | 135 kB 00:00 2024-09-19T11:21:02.095 INFO:teuthology.orchestra.run.smithi022.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.4 MB/s | 215 kB 00:00 2024-09-19T11:21:02.165 INFO:teuthology.orchestra.run.smithi028.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-19T11:21:02.190 INFO:teuthology.orchestra.run.smithi022.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 979 kB/s | 188 kB 00:00 2024-09-19T11:21:02.206 INFO:teuthology.orchestra.run.smithi028.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 4.1 MB/s | 172 kB 00:00 2024-09-19T11:21:02.213 INFO:teuthology.orchestra.run.smithi022.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 296 kB/s | 35 kB 00:00 2024-09-19T11:21:02.238 INFO:teuthology.orchestra.run.smithi022.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-19T11:21:02.248 INFO:teuthology.orchestra.run.smithi028.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 834 kB/s | 35 kB 00:00 2024-09-19T11:21:02.263 INFO:teuthology.orchestra.run.smithi022.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 893 kB/s | 172 kB 00:00 2024-09-19T11:21:02.290 INFO:teuthology.orchestra.run.smithi028.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-19T11:21:02.297 INFO:teuthology.orchestra.run.smithi022.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-09-19T11:21:02.316 INFO:teuthology.orchestra.run.smithi028.stdout:(15/37): python3-requests-2.25.1-8.el9.noarch.r 166 kB/s | 125 kB 00:00 2024-09-19T11:21:02.322 INFO:teuthology.orchestra.run.smithi022.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 467 kB/s | 51 kB 00:00 2024-09-19T11:21:02.341 INFO:teuthology.orchestra.run.smithi028.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T11:21:02.364 INFO:teuthology.orchestra.run.smithi022.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-19T11:21:02.407 INFO:teuthology.orchestra.run.smithi022.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 519 kB/s | 43 kB 00:00 2024-09-19T11:21:02.408 INFO:teuthology.orchestra.run.smithi028.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 455 kB/s | 42 kB 00:00 2024-09-19T11:21:02.432 INFO:teuthology.orchestra.run.smithi022.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-19T11:21:02.433 INFO:teuthology.orchestra.run.smithi028.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-19T11:21:02.457 INFO:teuthology.orchestra.run.smithi022.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-19T11:21:02.482 INFO:teuthology.orchestra.run.smithi022.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.8 MB/s | 358 kB 00:00 2024-09-19T11:21:02.500 INFO:teuthology.orchestra.run.smithi028.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-19T11:21:02.508 INFO:teuthology.orchestra.run.smithi022.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-19T11:21:02.525 INFO:teuthology.orchestra.run.smithi022.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 263 kB/s | 18 kB 00:00 2024-09-19T11:21:02.525 INFO:teuthology.orchestra.run.smithi028.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 474 kB/s | 43 kB 00:00 2024-09-19T11:21:02.550 INFO:teuthology.orchestra.run.smithi022.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 344 kB/s | 23 kB 00:00 2024-09-19T11:21:02.550 INFO:teuthology.orchestra.run.smithi028.stdout:(21/37): python3-urllib3-1.26.5-6.el9.noarch.rp 252 kB/s | 215 kB 00:00 2024-09-19T11:21:02.567 INFO:teuthology.orchestra.run.smithi022.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 332 kB/s | 19 kB 00:00 2024-09-19T11:21:02.576 INFO:teuthology.orchestra.run.smithi028.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-19T11:21:02.592 INFO:teuthology.orchestra.run.smithi022.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 294 kB/s | 20 kB 00:00 2024-09-19T11:21:02.609 INFO:teuthology.orchestra.run.smithi028.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 715 kB/s | 18 kB 00:00 2024-09-19T11:21:02.617 INFO:teuthology.orchestra.run.smithi022.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 926 kB/s | 46 kB 00:00 2024-09-19T11:21:02.635 INFO:teuthology.orchestra.run.smithi028.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-09-19T11:21:02.642 INFO:teuthology.orchestra.run.smithi022.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-19T11:21:02.651 INFO:teuthology.orchestra.run.smithi028.stdout:(25/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 106 kB/s | 11 kB 00:00 2024-09-19T11:21:02.669 INFO:teuthology.orchestra.run.smithi028.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 407 kB/s | 23 kB 00:00 2024-09-19T11:21:02.685 INFO:teuthology.orchestra.run.smithi028.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-19T11:21:02.702 INFO:teuthology.orchestra.run.smithi028.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-19T11:21:02.707 INFO:teuthology.orchestra.run.smithi022.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 184 kB/s | 16 kB 00:00 2024-09-19T11:21:02.719 INFO:teuthology.orchestra.run.smithi028.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-19T11:21:02.748 INFO:teuthology.orchestra.run.smithi028.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2024-09-19T11:21:02.781 INFO:teuthology.orchestra.run.smithi028.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 266 kB/s | 16 kB 00:00 2024-09-19T11:21:02.793 INFO:teuthology.orchestra.run.smithi022.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 598 kB/s | 90 kB 00:00 2024-09-19T11:21:02.807 INFO:teuthology.orchestra.run.smithi028.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-19T11:21:02.818 INFO:teuthology.orchestra.run.smithi022.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 322 kB/s | 36 kB 00:00 2024-09-19T11:21:02.843 INFO:teuthology.orchestra.run.smithi022.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-09-19T11:21:02.857 INFO:teuthology.orchestra.run.smithi028.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 477 kB/s | 36 kB 00:00 2024-09-19T11:21:02.868 INFO:teuthology.orchestra.run.smithi022.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T11:21:02.882 INFO:teuthology.orchestra.run.smithi028.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-19T11:21:02.885 INFO:teuthology.orchestra.run.smithi022.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 478 kB/s | 20 kB 00:00 2024-09-19T11:21:02.916 INFO:teuthology.orchestra.run.smithi028.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-19T11:21:02.927 INFO:teuthology.orchestra.run.smithi022.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-19T11:21:02.931 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:02.931 INFO:teuthology.orchestra.run.smithi022.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-19T11:21:02.941 INFO:teuthology.orchestra.run.smithi028.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 342 kB/s | 20 kB 00:00 2024-09-19T11:21:02.974 INFO:teuthology.orchestra.run.smithi028.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-19T11:21:02.979 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:02.979 INFO:teuthology.orchestra.run.smithi028.stdout:Total 7.8 MB/s | 31 MB 00:03 2024-09-19T11:21:03.089 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:03.134 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:03.134 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:03.137 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:03.182 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:03.182 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:03.621 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:03.621 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:03.675 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:03.676 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:05.032 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:05.084 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:05.105 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-19T11:21:05.147 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-19T11:21:05.152 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-19T11:21:05.185 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-19T11:21:05.208 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-19T11:21:05.213 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-19T11:21:05.284 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-19T11:21:05.287 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-19T11:21:05.346 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-19T11:21:05.349 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-19T11:21:05.378 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-19T11:21:05.382 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-19T11:21:05.474 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-19T11:21:05.488 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-19T11:21:05.559 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-19T11:21:05.565 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-19T11:21:05.606 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-19T11:21:05.612 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-19T11:21:05.641 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-19T11:21:05.646 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-19T11:21:05.671 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-19T11:21:05.676 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-19T11:21:05.710 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-19T11:21:05.713 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-19T11:21:05.744 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-19T11:21:05.755 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-19T11:21:05.785 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-19T11:21:05.805 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-19T11:21:05.837 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-19T11:21:05.848 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-19T11:21:05.871 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-19T11:21:05.882 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-19T11:21:05.920 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-19T11:21:05.931 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-19T11:21:05.965 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-19T11:21:05.974 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-19T11:21:06.045 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-19T11:21:06.055 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-19T11:21:06.197 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-19T11:21:06.216 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-19T11:21:06.279 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-19T11:21:06.306 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-19T11:21:06.338 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-19T11:21:06.366 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-19T11:21:06.377 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-19T11:21:06.401 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-19T11:21:06.435 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-19T11:21:06.460 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-19T11:21:06.499 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-19T11:21:06.532 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-19T11:21:06.697 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-19T11:21:06.729 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-19T11:21:06.747 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-19T11:21:06.780 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-19T11:21:06.829 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-19T11:21:06.869 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-19T11:21:06.927 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-19T11:21:06.968 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-19T11:21:06.983 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-19T11:21:07.009 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-19T11:21:07.123 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-19T11:21:07.149 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-19T11:21:07.306 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5012.g08af21e0.el9. 32/37 2024-09-19T11:21:07.332 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5012.g08af21e0.el9. 32/37 2024-09-19T11:21:07.339 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 33/37 2024-09-19T11:21:07.364 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 33/37 2024-09-19T11:21:07.368 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 33/37 2024-09-19T11:21:07.368 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:07.368 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T11:21:07.368 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T11:21:07.368 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T11:21:07.368 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:07.392 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 33/37 2024-09-19T11:21:07.392 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:07.392 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T11:21:07.392 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T11:21:07.392 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T11:21:07.392 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:08.915 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 34/37 2024-09-19T11:21:08.939 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 34/37 2024-09-19T11:21:08.944 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 34/37 2024-09-19T11:21:08.944 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:08.944 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T11:21:08.944 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T11:21:08.945 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T11:21:08.945 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:08.968 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 34/37 2024-09-19T11:21:08.968 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:08.969 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T11:21:08.969 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T11:21:08.969 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T11:21:08.969 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:09.416 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 35/37 2024-09-19T11:21:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 35/37 2024-09-19T11:21:09.441 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:09.441 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T11:21:09.441 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T11:21:09.441 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T11:21:09.441 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:09.448 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 35/37 2024-09-19T11:21:09.475 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 35/37 2024-09-19T11:21:09.475 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:09.475 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T11:21:09.476 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T11:21:09.476 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T11:21:09.476 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:09.675 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 36/37 2024-09-19T11:21:09.699 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 36/37 2024-09-19T11:21:09.700 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:09.700 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T11:21:09.700 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T11:21:09.700 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T11:21:09.700 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:09.716 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 36/37 2024-09-19T11:21:09.741 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 36/37 2024-09-19T11:21:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T11:21:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T11:21:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T11:21:09.741 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:09.776 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 37/37 2024-09-19T11:21:09.807 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 37/37 2024-09-19T11:21:11.454 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 37/37 2024-09-19T11:21:11.454 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 1/37 2024-09-19T11:21:11.454 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 2/37 2024-09-19T11:21:11.454 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 3/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 4/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 5/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5012.g08af21e0.el9. 6/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-19T11:21:11.455 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-19T11:21:11.456 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-19T11:21:11.457 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-19T11:21:11.457 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-19T11:21:11.457 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-19T11:21:11.457 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-19T11:21:11.457 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-19T11:21:11.457 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-19T11:21:11.644 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 37/37 2024-09-19T11:21:11.644 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 1/37 2024-09-19T11:21:11.644 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 2/37 2024-09-19T11:21:11.644 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 3/37 2024-09-19T11:21:11.644 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 4/37 2024-09-19T11:21:11.644 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 5/37 2024-09-19T11:21:11.644 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5012.g08af21e0.el9. 6/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-19T11:21:11.645 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-19T11:21:11.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-19T11:21:11.647 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-19T11:21:11.647 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-19T11:21:11.647 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-19T11:21:11.647 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-19T11:21:11.647 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-19T11:21:11.648 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-19T11:21:11.648 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-19T11:21:11.648 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-19T11:21:11.648 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.597 INFO:teuthology.orchestra.run.smithi022.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T11:21:12.598 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T11:21:12.599 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T11:21:12.600 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T11:21:12.600 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T11:21:12.600 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T11:21:12.600 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T11:21:12.600 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:12.600 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:12.646 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-19T11:21:12.646 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T11:21:12.647 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T11:21:12.648 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:12.649 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:12.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-base 2024-09-19T11:21:13.027 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-base 2024-09-19T11:21:13.595 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:54 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:13.630 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:55 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:13.688 INFO:teuthology.orchestra.run.smithi022.stdout:Package ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:21:13.722 INFO:teuthology.orchestra.run.smithi028.stdout:Package ceph-base-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:21:13.732 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:13.733 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:21:13.733 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:13.766 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:13.768 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:21:13.768 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:13.808 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install cephadm 2024-09-19T11:21:13.834 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install cephadm 2024-09-19T11:21:14.412 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:55 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:14.434 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:56 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:14.549 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 784 k 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:14.550 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 784 k 2024-09-19T11:21:14.551 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 791 k 2024-09-19T11:21:14.551 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:14.571 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:14.572 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:21:14.572 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:14.572 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 784 k 2024-09-19T11:21:14.572 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:14.572 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:14.573 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:14.573 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:21:14.573 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:14.573 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 784 k 2024-09-19T11:21:14.573 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 791 k 2024-09-19T11:21:14.573 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:14.808 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm-19.3.0-5012.g08af21e0.el9.noarch.rpm 3.0 MB/s | 784 kB 00:00 2024-09-19T11:21:14.808 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:14.809 INFO:teuthology.orchestra.run.smithi022.stdout:Total 3.0 MB/s | 784 kB 00:00 2024-09-19T11:21:14.809 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:14.814 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:14.814 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:14.821 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:14.821 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:14.857 INFO:teuthology.orchestra.run.smithi028.stdout:cephadm-19.3.0-5012.g08af21e0.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-19T11:21:14.858 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:14.859 INFO:teuthology.orchestra.run.smithi028.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-19T11:21:14.859 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:14.865 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:14.865 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:14.871 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:14.871 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:14.892 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:14.946 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:15.365 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:15.373 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:15.463 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:15.464 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:15.815 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:15.817 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:16.060 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:16.060 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:16.060 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:16.060 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:16.060 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:16.060 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:16.068 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:16.068 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:16.069 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:16.069 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:16.069 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:16.069 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:16.253 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-immutable-object-cache 2024-09-19T11:21:16.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-immutable-object-cache 2024-09-19T11:21:16.865 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:58 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:16.903 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:57 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:17.004 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:17.004 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 145 k 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 145 k 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 432 k 2024-09-19T11:21:17.005 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:17.047 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:17.048 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-19T11:21:17.048 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:17.048 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 145 k 2024-09-19T11:21:17.048 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:21:17.049 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 145 k 2024-09-19T11:21:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 432 k 2024-09-19T11:21:17.049 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:17.204 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-immutable-object-cache-19.3.0-5012.g08af21 729 kB/s | 145 kB 00:00 2024-09-19T11:21:17.205 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:17.205 INFO:teuthology.orchestra.run.smithi028.stdout:Total 724 kB/s | 145 kB 00:00 2024-09-19T11:21:17.205 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:17.213 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:17.213 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:17.273 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:17.273 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:17.285 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-immutable-object-cache-19.3.0-5012.g08af21 613 kB/s | 145 kB 00:00 2024-09-19T11:21:17.286 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:17.286 INFO:teuthology.orchestra.run.smithi022.stdout:Total 609 kB/s | 145 kB 00:00 2024-09-19T11:21:17.286 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:17.294 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:17.294 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:17.355 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:17.355 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:17.428 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:17.488 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0. 1/1 2024-09-19T11:21:17.507 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:17.515 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0. 1/1 2024-09-19T11:21:17.515 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:17.515 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T11:21:17.515 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:17.583 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0. 1/1 2024-09-19T11:21:17.609 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0. 1/1 2024-09-19T11:21:17.609 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:17.609 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T11:21:17.609 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:18.441 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0. 1/1 2024-09-19T11:21:18.441 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:18.441 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:18.441 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:18.441 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:18.441 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:18.542 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0. 1/1 2024-09-19T11:21:18.542 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:18.542 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:18.542 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:18.542 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:18.542 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:18.657 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr 2024-09-19T11:21:18.804 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr 2024-09-19T11:21:19.266 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:01 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:19.360 INFO:teuthology.orchestra.run.smithi028.stdout:Package ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:21:19.405 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:19.406 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:21:19.406 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:19.411 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:00 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:19.480 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-dashboard 2024-09-19T11:21:19.503 INFO:teuthology.orchestra.run.smithi022.stdout:Package ceph-mgr-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:21:19.548 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:19.549 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:21:19.549 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:19.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-dashboard 2024-09-19T11:21:20.079 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:02 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:20.217 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:01 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:20.217 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:20.218 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 14 M 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 34 k 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 16 k 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T11:21:20.219 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout:Install 12 Packages 2024-09-19T11:21:20.220 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:20.221 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 19 M 2024-09-19T11:21:20.221 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 159 M 2024-09-19T11:21:20.221 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:20.354 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 14 M 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 34 k 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 16 k 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T11:21:20.356 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout:Install 12 Packages 2024-09-19T11:21:20.357 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:20.358 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 19 M 2024-09-19T11:21:20.358 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 159 M 2024-09-19T11:21:20.358 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:20.590 INFO:teuthology.orchestra.run.smithi028.stdout:(1/12): ceph-grafana-dashboards-19.3.0-5012.g08 240 kB/s | 34 kB 00:00 2024-09-19T11:21:20.820 INFO:teuthology.orchestra.run.smithi022.stdout:(1/12): ceph-prometheus-alerts-19.3.0-5012.g08a 101 kB/s | 16 kB 00:00 2024-09-19T11:21:20.845 INFO:teuthology.orchestra.run.smithi022.stdout:(2/12): ceph-grafana-dashboards-19.3.0-5012.g08 182 kB/s | 34 kB 00:00 2024-09-19T11:21:20.979 INFO:teuthology.orchestra.run.smithi022.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-19T11:21:21.096 INFO:teuthology.orchestra.run.smithi022.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.7 MB/s | 1.0 MB 00:00 2024-09-19T11:21:21.137 INFO:teuthology.orchestra.run.smithi022.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.3 MB/s | 863 kB 00:00 2024-09-19T11:21:21.179 INFO:teuthology.orchestra.run.smithi022.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 6.4 MB/s | 548 kB 00:00 2024-09-19T11:21:21.213 INFO:teuthology.orchestra.run.smithi022.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-09-19T11:21:21.255 INFO:teuthology.orchestra.run.smithi022.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-09-19T11:21:21.280 INFO:teuthology.orchestra.run.smithi022.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-19T11:21:21.347 INFO:teuthology.orchestra.run.smithi022.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-19T11:21:21.372 INFO:teuthology.orchestra.run.smithi022.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-09-19T11:21:21.374 INFO:teuthology.orchestra.run.smithi028.stdout:(2/12): ceph-mgr-dashboard-19.3.0-5012.g08af21e 15 MB/s | 14 MB 00:00 2024-09-19T11:21:21.674 INFO:teuthology.orchestra.run.smithi028.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 953 kB/s | 1.0 MB 00:01 2024-09-19T11:21:21.764 INFO:teuthology.orchestra.run.smithi022.stdout:(12/12): ceph-mgr-dashboard-19.3.0-5012.g08af21 13 MB/s | 14 MB 00:01 2024-09-19T11:21:21.769 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:21.769 INFO:teuthology.orchestra.run.smithi022.stdout:Total 14 MB/s | 19 MB 00:01 2024-09-19T11:21:21.822 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:21.833 INFO:teuthology.orchestra.run.smithi028.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.3 MB/s | 863 kB 00:00 2024-09-19T11:21:21.844 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:21.844 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:21.891 INFO:teuthology.orchestra.run.smithi028.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-19T11:21:21.916 INFO:teuthology.orchestra.run.smithi028.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-19T11:21:21.992 INFO:teuthology.orchestra.run.smithi028.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 27 MB/s | 2.0 MB 00:00 2024-09-19T11:21:22.017 INFO:teuthology.orchestra.run.smithi028.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-19T11:21:22.042 INFO:teuthology.orchestra.run.smithi028.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-19T11:21:22.067 INFO:teuthology.orchestra.run.smithi028.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-19T11:21:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:22.125 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:22.176 INFO:teuthology.orchestra.run.smithi028.stdout:(11/12): python3-protobuf-3.14.0-13.el9.noarch. 336 kB/s | 269 kB 00:00 2024-09-19T11:21:22.651 INFO:teuthology.orchestra.run.smithi028.stdout:(12/12): ceph-prometheus-alerts-19.3.0-5012.g08 7.5 kB/s | 16 kB 00:02 2024-09-19T11:21:22.655 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:22.655 INFO:teuthology.orchestra.run.smithi028.stdout:Total 7.9 MB/s | 19 MB 00:02 2024-09-19T11:21:22.708 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:22.728 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:22.729 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:22.729 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:22.808 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-19T11:21:22.843 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-19T11:21:22.899 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-19T11:21:22.936 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-19T11:21:23.021 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:23.022 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:23.034 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-19T11:21:23.169 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-19T11:21:23.260 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-19T11:21:23.309 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-19T11:21:23.336 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-19T11:21:23.369 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5012.g08af21e0.el9 10/12 2024-09-19T11:21:23.622 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:23.711 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-19T11:21:23.754 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-19T11:21:23.802 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-19T11:21:23.830 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-19T11:21:23.943 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-19T11:21:24.072 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-19T11:21:24.163 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-19T11:21:24.212 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-19T11:21:24.240 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-19T11:21:24.273 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5012.g08af21e0.el9 10/12 2024-09-19T11:21:26.777 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5012.g08af21e0.el 11/12 2024-09-19T11:21:26.830 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noa 12/12 2024-09-19T11:21:27.656 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5012.g08af21e0.el 11/12 2024-09-19T11:21:27.709 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noa 12/12 2024-09-19T11:21:28.743 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noa 12/12 2024-09-19T11:21:28.743 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5012.g08af21e0.el 1/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noa 2/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5012.g08af21e0.el9 3/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-19T11:21:28.744 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:29.083 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:29.084 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:29.352 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-19T11:21:29.853 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noa 12/12 2024-09-19T11:21:29.853 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5012.g08af21e0.el 1/12 2024-09-19T11:21:29.853 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noa 2/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5012.g08af21e0.el9 3/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-19T11:21:29.854 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-19T11:21:29.964 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:10 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:30.105 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:30.106 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-19T11:21:30.106 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:30.106 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-19T11:21:30.106 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 7.4 M 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T11:21:30.107 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T11:21:30.108 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T11:21:30.108 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:30.108 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:30.108 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-19T11:21:30.108 INFO:teuthology.orchestra.run.smithi022.stdout:Install 12 Packages 2024-09-19T11:21:30.108 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:30.109 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 38 M 2024-09-19T11:21:30.109 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 212 M 2024-09-19T11:21:30.109 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:30.173 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T11:21:30.174 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T11:21:30.175 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T11:21:30.175 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T11:21:30.175 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:30.175 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:30.581 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-19T11:21:30.613 INFO:teuthology.orchestra.run.smithi022.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 764 kB/s | 191 kB 00:00 2024-09-19T11:21:30.655 INFO:teuthology.orchestra.run.smithi022.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-19T11:21:30.705 INFO:teuthology.orchestra.run.smithi022.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-19T11:21:30.748 INFO:teuthology.orchestra.run.smithi022.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 434 kB/s | 18 kB 00:00 2024-09-19T11:21:30.781 INFO:teuthology.orchestra.run.smithi022.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 1.1 MB/s | 38 kB 00:00 2024-09-19T11:21:30.923 INFO:teuthology.orchestra.run.smithi022.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-50 13 MB/s | 7.4 MB 00:00 2024-09-19T11:21:31.074 INFO:teuthology.orchestra.run.smithi022.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-19T11:21:31.195 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:13 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:31.338 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:31.338 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 7.4 M 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T11:21:31.339 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout:Install 12 Packages 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:31.340 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 38 M 2024-09-19T11:21:31.341 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 212 M 2024-09-19T11:21:31.341 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:31.449 INFO:teuthology.orchestra.run.smithi022.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.8 MB/s | 3.0 MB 00:00 2024-09-19T11:21:31.533 INFO:teuthology.orchestra.run.smithi022.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-19T11:21:31.833 INFO:teuthology.orchestra.run.smithi028.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-19T11:21:31.875 INFO:teuthology.orchestra.run.smithi028.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-19T11:21:31.933 INFO:teuthology.orchestra.run.smithi028.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-19T11:21:32.017 INFO:teuthology.orchestra.run.smithi028.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-09-19T11:21:32.041 INFO:teuthology.orchestra.run.smithi022.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.9 MB/s | 4.9 MB 00:01 2024-09-19T11:21:32.067 INFO:teuthology.orchestra.run.smithi028.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-09-19T11:21:32.109 INFO:teuthology.orchestra.run.smithi028.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 419 kB/s | 38 kB 00:00 2024-09-19T11:21:32.226 INFO:teuthology.orchestra.run.smithi028.stdout:(7/12): ceph-mgr-diskprediction-local-19.3.0-50 13 MB/s | 7.4 MB 00:00 2024-09-19T11:21:32.251 INFO:teuthology.orchestra.run.smithi028.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-19T11:21:32.275 INFO:teuthology.orchestra.run.smithi022.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-19T11:21:32.318 INFO:teuthology.orchestra.run.smithi028.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 20 MB/s | 4.9 MB 00:00 2024-09-19T11:21:32.344 INFO:teuthology.orchestra.run.smithi028.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-09-19T11:21:32.483 INFO:teuthology.orchestra.run.smithi022.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-19T11:21:32.484 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:32.484 INFO:teuthology.orchestra.run.smithi022.stdout:Total 16 MB/s | 38 MB 00:02 2024-09-19T11:21:32.502 INFO:teuthology.orchestra.run.smithi028.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-19T11:21:32.677 INFO:teuthology.orchestra.run.smithi028.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-09-19T11:21:32.678 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:32.679 INFO:teuthology.orchestra.run.smithi028.stdout:Total 29 MB/s | 38 MB 00:01 2024-09-19T11:21:32.851 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:32.878 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:32.878 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:33.044 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:33.071 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:33.071 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:33.332 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:33.332 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:33.533 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:33.533 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:34.155 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:34.223 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-19T11:21:34.268 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-19T11:21:34.324 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:34.379 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-19T11:21:34.380 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-19T11:21:34.429 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-19T11:21:34.433 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-19T11:21:34.531 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-19T11:21:34.568 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-19T11:21:34.635 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-19T11:21:34.673 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-19T11:21:34.753 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-19T11:21:34.762 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-19T11:21:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-19T11:21:34.871 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-19T11:21:35.444 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-19T11:21:35.562 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-19T11:21:35.566 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-19T11:21:35.675 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-19T11:21:36.904 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T11:21:37.006 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T11:21:38.004 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-19T11:21:38.031 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af2 12/12 2024-09-19T11:21:38.082 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-19T11:21:38.115 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af2 12/12 2024-09-19T11:21:40.628 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af2 12/12 2024-09-19T11:21:40.628 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af2 1/12 2024-09-19T11:21:40.628 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-19T11:21:40.628 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-19T11:21:40.629 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-19T11:21:40.632 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-19T11:21:40.632 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-19T11:21:40.632 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-19T11:21:40.632 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-19T11:21:40.632 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-19T11:21:40.632 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T11:21:40.633 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af2 12/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af2 1/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-19T11:21:40.779 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-19T11:21:40.780 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-19T11:21:40.780 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-19T11:21:40.780 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-19T11:21:40.780 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T11:21:41.032 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T11:21:41.033 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T11:21:41.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T11:21:41.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T11:21:41.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T11:21:41.033 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T11:21:41.033 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:41.033 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:41.183 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-19T11:21:41.183 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:41.183 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:41.183 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:41.183 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T11:21:41.183 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:41.184 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:41.474 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-rook 2024-09-19T11:21:41.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-rook 2024-09-19T11:21:42.092 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:24 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:42.217 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:23 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:42.231 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:42.232 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:42.232 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:21:42.232 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 49 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T11:21:42.233 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout:Install 13 Packages 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:42.234 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 2.2 M 2024-09-19T11:21:42.235 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 26 M 2024-09-19T11:21:42.235 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:42.356 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 49 k 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T11:21:42.357 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout:Install 13 Packages 2024-09-19T11:21:42.358 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:42.359 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 2.2 M 2024-09-19T11:21:42.359 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 26 M 2024-09-19T11:21:42.359 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:42.489 INFO:teuthology.orchestra.run.smithi028.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 229 kB/s | 19 kB 00:00 2024-09-19T11:21:42.514 INFO:teuthology.orchestra.run.smithi028.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 235 kB/s | 26 kB 00:00 2024-09-19T11:21:42.572 INFO:teuthology.orchestra.run.smithi028.stdout:(3/13): ceph-mgr-rook-19.3.0-5012.g08af21e0.el9 294 kB/s | 49 kB 00:00 2024-09-19T11:21:42.598 INFO:teuthology.orchestra.run.smithi028.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-19T11:21:42.606 INFO:teuthology.orchestra.run.smithi022.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 207 kB/s | 26 kB 00:00 2024-09-19T11:21:42.623 INFO:teuthology.orchestra.run.smithi028.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-19T11:21:42.631 INFO:teuthology.orchestra.run.smithi022.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 128 kB/s | 19 kB 00:00 2024-09-19T11:21:42.657 INFO:teuthology.orchestra.run.smithi022.stdout:(3/13): ceph-mgr-rook-19.3.0-5012.g08af21e0.el9 282 kB/s | 49 kB 00:00 2024-09-19T11:21:42.665 INFO:teuthology.orchestra.run.smithi028.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 770 kB/s | 32 kB 00:00 2024-09-19T11:21:42.690 INFO:teuthology.orchestra.run.smithi028.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 583 kB/s | 54 kB 00:00 2024-09-19T11:21:42.715 INFO:teuthology.orchestra.run.smithi028.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 281 kB/s | 14 kB 00:00 2024-09-19T11:21:42.766 INFO:teuthology.orchestra.run.smithi028.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 2.9 MB/s | 220 kB 00:00 2024-09-19T11:21:42.774 INFO:teuthology.orchestra.run.smithi022.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-19T11:21:42.791 INFO:teuthology.orchestra.run.smithi028.stdout:(10/13): python3-kubernetes-26.1.0-2.el9.noarch 15 MB/s | 1.0 MB 00:00 2024-09-19T11:21:42.799 INFO:teuthology.orchestra.run.smithi022.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-09-19T11:21:42.816 INFO:teuthology.orchestra.run.smithi028.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-19T11:21:42.832 INFO:teuthology.orchestra.run.smithi022.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-09-19T11:21:42.841 INFO:teuthology.orchestra.run.smithi028.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-19T11:21:42.866 INFO:teuthology.orchestra.run.smithi022.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-19T11:21:42.867 INFO:teuthology.orchestra.run.smithi028.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-09-19T11:21:42.869 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:42.869 INFO:teuthology.orchestra.run.smithi028.stdout:Total 3.5 MB/s | 2.2 MB 00:00 2024-09-19T11:21:42.892 INFO:teuthology.orchestra.run.smithi022.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-09-19T11:21:42.917 INFO:teuthology.orchestra.run.smithi022.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 172 kB/s | 14 kB 00:00 2024-09-19T11:21:42.935 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:42.950 INFO:teuthology.orchestra.run.smithi022.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-19T11:21:42.951 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:42.952 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:42.976 INFO:teuthology.orchestra.run.smithi022.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-19T11:21:43.001 INFO:teuthology.orchestra.run.smithi022.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-19T11:21:43.043 INFO:teuthology.orchestra.run.smithi022.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-19T11:21:43.045 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:43.046 INFO:teuthology.orchestra.run.smithi022.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-19T11:21:43.049 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:43.050 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:43.114 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:43.130 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:43.130 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:43.229 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:43.229 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:43.569 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:43.758 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:43.758 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-19T11:21:43.820 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-19T11:21:43.876 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-19T11:21:43.911 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-19T11:21:43.931 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-19T11:21:43.953 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-19T11:21:43.996 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-19T11:21:44.060 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-19T11:21:44.060 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-19T11:21:44.096 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-19T11:21:44.146 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-19T11:21:44.178 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-19T11:21:44.238 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-19T11:21:44.254 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-19T11:21:44.373 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-19T11:21:44.432 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-19T11:21:45.004 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-19T11:21:45.051 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-19T11:21:45.084 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-19T11:21:45.134 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-19T11:21:45.162 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 13/13 2024-09-19T11:21:45.191 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-19T11:21:45.244 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-19T11:21:45.278 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-19T11:21:45.329 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-19T11:21:45.364 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 13/13 2024-09-19T11:21:45.930 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 13/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 1/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-19T11:21:45.931 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-19T11:21:46.161 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 13/13 2024-09-19T11:21:46.161 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 1/13 2024-09-19T11:21:46.161 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-19T11:21:46.161 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-19T11:21:46.161 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-19T11:21:46.161 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-19T11:21:46.162 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-19T11:21:46.162 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-19T11:21:46.162 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-19T11:21:46.162 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-19T11:21:46.162 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-19T11:21:46.162 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-19T11:21:46.162 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T11:21:46.271 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T11:21:46.272 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T11:21:46.272 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T11:21:46.272 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T11:21:46.272 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T11:21:46.272 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T11:21:46.272 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:46.272 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:46.518 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-mgr-cephadm 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T11:21:46.523 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T11:21:46.524 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T11:21:46.524 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T11:21:46.524 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T11:21:46.524 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T11:21:46.524 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T11:21:46.524 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:46.524 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:46.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-mgr-cephadm 2024-09-19T11:21:47.121 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:29 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:47.261 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 154 k 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T11:21:47.262 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout:Install 6 Packages 2024-09-19T11:21:47.263 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:47.264 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 7.0 M 2024-09-19T11:21:47.264 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 33 M 2024-09-19T11:21:47.264 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:47.411 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:28 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:47.467 INFO:teuthology.orchestra.run.smithi028.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 2.0 MB/s | 248 kB 00:00 2024-09-19T11:21:47.526 INFO:teuthology.orchestra.run.smithi028.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-19T11:21:47.549 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:47.550 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:47.550 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:47.550 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 154 k 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout:Install 6 Packages 2024-09-19T11:21:47.551 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:47.552 INFO:teuthology.orchestra.run.smithi028.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-19T11:21:47.552 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 7.0 M 2024-09-19T11:21:47.552 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 33 M 2024-09-19T11:21:47.552 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:47.576 INFO:teuthology.orchestra.run.smithi028.stdout:(4/6): ceph-mgr-cephadm-19.3.0-5012.g08af21e0.e 665 kB/s | 154 kB 00:00 2024-09-19T11:21:47.601 INFO:teuthology.orchestra.run.smithi028.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-19T11:21:47.693 INFO:teuthology.orchestra.run.smithi028.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-19T11:21:47.695 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:47.695 INFO:teuthology.orchestra.run.smithi028.stdout:Total 16 MB/s | 7.0 MB 00:00 2024-09-19T11:21:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:47.780 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:47.780 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:47.886 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:47.886 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:47.954 INFO:teuthology.orchestra.run.smithi022.stdout:(1/6): ceph-mgr-cephadm-19.3.0-5012.g08af21e0.e 714 kB/s | 154 kB 00:00 2024-09-19T11:21:48.004 INFO:teuthology.orchestra.run.smithi022.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-19T11:21:48.021 INFO:teuthology.orchestra.run.smithi022.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-19T11:21:48.046 INFO:teuthology.orchestra.run.smithi022.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-19T11:21:48.235 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:48.371 INFO:teuthology.orchestra.run.smithi022.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 392 kB/s | 248 kB 00:00 2024-09-19T11:21:48.401 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-19T11:21:48.454 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-19T11:21:49.011 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-19T11:21:49.101 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-19T11:21:49.177 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-19T11:21:49.207 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 6/6 2024-09-19T11:21:50.000 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 6/6 2024-09-19T11:21:50.001 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/6 2024-09-19T11:21:50.001 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-19T11:21:50.001 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-19T11:21:50.001 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-19T11:21:50.001 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-19T11:21:50.038 INFO:teuthology.orchestra.run.smithi022.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.6 MB/s | 6.0 MB 00:02 2024-09-19T11:21:50.040 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:50.041 INFO:teuthology.orchestra.run.smithi022.stdout:Total 2.8 MB/s | 7.0 MB 00:02 2024-09-19T11:21:50.118 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:50.130 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:50.130 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:50.237 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:50.237 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:50.297 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T11:21:50.298 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T11:21:50.299 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:50.299 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:50.552 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-fuse 2024-09-19T11:21:50.671 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:50.852 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-19T11:21:50.900 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-19T11:21:51.157 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:33 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:51.296 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 900 k 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T11:21:51.297 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:51.298 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:51.298 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:51.298 INFO:teuthology.orchestra.run.smithi028.stdout:Install 2 Packages 2024-09-19T11:21:51.298 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:51.298 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 979 k 2024-09-19T11:21:51.298 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 2.9 M 2024-09-19T11:21:51.298 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:51.459 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-19T11:21:51.503 INFO:teuthology.orchestra.run.smithi028.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 517 kB/s | 80 kB 00:00 2024-09-19T11:21:51.554 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-19T11:21:51.630 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-19T11:21:51.653 INFO:teuthology.orchestra.run.smithi028.stdout:(2/2): ceph-fuse-19.3.0-5012.g08af21e0.el9.x86_ 2.9 MB/s | 900 kB 00:00 2024-09-19T11:21:51.653 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:51.654 INFO:teuthology.orchestra.run.smithi028.stdout:Total 2.7 MB/s | 979 kB 00:00 2024-09-19T11:21:51.661 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 6/6 2024-09-19T11:21:51.665 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:51.688 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:51.688 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:51.774 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:51.774 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:51.968 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:52.123 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-19T11:21:52.196 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 2/2 2024-09-19T11:21:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 6/6 2024-09-19T11:21:52.361 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 1/6 2024-09-19T11:21:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-19T11:21:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-19T11:21:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-19T11:21:52.362 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T11:21:52.654 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:52.655 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:52.885 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 2/2 2024-09-19T11:21:52.885 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/2 2024-09-19T11:21:53.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-fuse 2024-09-19T11:21:53.154 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-19T11:21:53.154 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:53.154 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:53.154 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-19T11:21:53.154 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:53.154 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:53.440 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-volume 2024-09-19T11:21:53.636 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:34 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:53.774 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 900 k 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:53.775 INFO:teuthology.orchestra.run.smithi022.stdout:Install 2 Packages 2024-09-19T11:21:53.776 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:53.776 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 979 k 2024-09-19T11:21:53.776 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 2.9 M 2024-09-19T11:21:53.776 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:54.051 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:36 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:54.192 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:54.192 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 285 k 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 285 k 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 1.3 M 2024-09-19T11:21:54.193 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:54.215 INFO:teuthology.orchestra.run.smithi022.stdout:(1/2): ceph-fuse-19.3.0-5012.g08af21e0.el9.x86_ 3.1 MB/s | 900 kB 00:00 2024-09-19T11:21:54.423 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-volume-19.3.0-5012.g08af21e0.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-19T11:21:54.424 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:54.424 INFO:teuthology.orchestra.run.smithi028.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-19T11:21:54.424 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:54.434 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:54.434 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:54.464 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:54.464 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:54.557 INFO:teuthology.orchestra.run.smithi022.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 127 kB/s | 80 kB 00:00 2024-09-19T11:21:54.557 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:54.558 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.2 MB/s | 979 kB 00:00 2024-09-19T11:21:54.572 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:54.597 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:54.597 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:54.689 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:54.689 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:54.699 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:54.783 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:54.801 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:54.801 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:54.801 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T11:21:54.801 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:54.880 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:55.036 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-19T11:21:55.110 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 2/2 2024-09-19T11:21:55.769 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 2/2 2024-09-19T11:21:55.769 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/2 2024-09-19T11:21:55.811 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:55.811 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:55.811 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:55.811 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:55.811 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:55.811 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:56.087 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-19T11:21:56.087 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:56.087 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:56.087 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-19T11:21:56.087 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:56.087 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:56.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librados-devel 2024-09-19T11:21:56.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-volume 2024-09-19T11:21:56.718 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:38 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:56.857 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:56.858 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:56.858 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-09-19T11:21:56.858 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:56.858 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:56.858 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 127 k 2024-09-19T11:21:56.858 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:56.859 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:56.859 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:56.859 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:21:56.859 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:56.859 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 127 k 2024-09-19T11:21:56.859 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 456 k 2024-09-19T11:21:56.859 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:21:57.026 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:38 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:57.060 INFO:teuthology.orchestra.run.smithi028.stdout:librados-devel-19.3.0-5012.g08af21e0.el9.x86_64 628 kB/s | 127 kB 00:00 2024-09-19T11:21:57.061 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:57.061 INFO:teuthology.orchestra.run.smithi028.stdout:Total 623 kB/s | 127 kB 00:00 2024-09-19T11:21:57.061 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:21:57.067 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:21:57.067 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:21:57.134 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:21:57.134 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:21:57.167 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:57.167 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:57.167 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-19T11:21:57.167 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:19.3.0-5012.g08af21e0.el9 ceph-noarch 285 k 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 285 k 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 1.3 M 2024-09-19T11:21:57.168 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:57.290 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:21:57.398 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:21:57.403 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-volume-19.3.0-5012.g08af21e0.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-19T11:21:57.404 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:21:57.404 INFO:teuthology.orchestra.run.smithi022.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-19T11:21:57.404 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:21:57.413 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:21:57.414 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:21:57.445 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:21:57.445 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:21:57.676 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:21:57.747 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:57.765 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:21:57.765 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T11:21:57.765 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:57.842 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:21:58.079 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:21:58.079 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:58.079 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:21:58.079 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:21:58.079 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:58.079 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:58.303 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install libcephfs2 2024-09-19T11:21:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 1/1 2024-09-19T11:21:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:58.819 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:21:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:19.3.0-5012.g08af21e0.el9.noarch 2024-09-19T11:21:58.819 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:58.819 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:21:58.912 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:40 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:59.006 INFO:teuthology.orchestra.run.smithi028.stdout:Package libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:21:59.053 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:59.054 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:21:59.054 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:21:59.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librados-devel 2024-09-19T11:21:59.122 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install libcephfs-devel 2024-09-19T11:21:59.712 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:40 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:21:59.730 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:41 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 127 k 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:21:59.854 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:21:59.855 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:21:59.855 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:21:59.855 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 127 k 2024-09-19T11:21:59.855 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 456 k 2024-09-19T11:21:59.855 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:21:59.870 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:21:59.870 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 32 k 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:21:59.871 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:21:59.872 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 32 k 2024-09-19T11:21:59.872 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 142 k 2024-09-19T11:21:59.872 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:22:00.012 INFO:teuthology.orchestra.run.smithi028.stdout:libcephfs-devel-19.3.0-5012.g08af21e0.el9.x86_6 226 kB/s | 32 kB 00:00 2024-09-19T11:22:00.013 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:00.013 INFO:teuthology.orchestra.run.smithi028.stdout:Total 224 kB/s | 32 kB 00:00 2024-09-19T11:22:00.013 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:22:00.017 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:22:00.017 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:22:00.039 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:22:00.040 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:22:00.050 INFO:teuthology.orchestra.run.smithi022.stdout:librados-devel-19.3.0-5012.g08af21e0.el9.x86_64 639 kB/s | 127 kB 00:00 2024-09-19T11:22:00.050 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:00.051 INFO:teuthology.orchestra.run.smithi022.stdout:Total 634 kB/s | 127 kB 00:00 2024-09-19T11:22:00.051 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:22:00.057 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:22:00.057 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:22:00.124 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:22:00.125 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:22:00.125 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:22:00.246 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:00.286 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:22:00.393 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:00.601 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:00.837 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:00.837 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:00.837 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:22:00.837 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:00.837 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:00.837 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:00.849 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:01.012 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librados2 2024-09-19T11:22:01.128 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:01.128 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:01.128 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:22:01.129 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:01.129 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:01.129 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:01.405 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs2 2024-09-19T11:22:01.618 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:43 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:01.709 INFO:teuthology.orchestra.run.smithi028.stdout:Package librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:01.757 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:01.758 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:22:01.758 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:01.829 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install librbd1 2024-09-19T11:22:02.021 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:43 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:02.115 INFO:teuthology.orchestra.run.smithi022.stdout:Package libcephfs2-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:02.163 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:02.163 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:22:02.163 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:02.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install libcephfs-devel 2024-09-19T11:22:02.450 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:44 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:02.545 INFO:teuthology.orchestra.run.smithi028.stdout:Package librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:02.591 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:02.592 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:22:02.592 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:02.666 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rados 2024-09-19T11:22:02.847 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:43 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:02.987 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 32 k 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 32 k 2024-09-19T11:22:02.988 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 142 k 2024-09-19T11:22:02.989 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:22:03.146 INFO:teuthology.orchestra.run.smithi022.stdout:libcephfs-devel-19.3.0-5012.g08af21e0.el9.x86_6 203 kB/s | 32 kB 00:00 2024-09-19T11:22:03.146 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:03.146 INFO:teuthology.orchestra.run.smithi022.stdout:Total 201 kB/s | 32 kB 00:00 2024-09-19T11:22:03.146 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:22:03.150 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:22:03.150 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:22:03.170 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:22:03.170 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:22:03.263 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:22:03.282 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:45 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:03.376 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:03.389 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:03.421 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:03.422 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:22:03.423 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:03.499 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rgw 2024-09-19T11:22:03.782 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:04.093 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:22:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:04.093 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:04.093 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:04.102 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:46 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:04.198 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:04.245 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:04.246 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:22:04.246 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:04.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librados2 2024-09-19T11:22:04.315 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-cephfs 2024-09-19T11:22:04.894 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:45 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:04.924 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:46 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:04.987 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:05.017 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:05.033 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:05.034 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:22:05.034 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:05.062 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:05.063 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:22:05.063 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:05.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install librbd1 2024-09-19T11:22:05.138 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install python3-rbd 2024-09-19T11:22:05.712 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:46 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:05.744 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:47 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:05.805 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:05.837 INFO:teuthology.orchestra.run.smithi028.stdout:Package python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:05.852 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:05.853 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:22:05.853 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:05.883 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:05.884 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-19T11:22:05.884 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:05.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rados 2024-09-19T11:22:05.964 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-fuse 2024-09-19T11:22:06.529 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:47 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:06.570 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:48 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:06.623 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rados-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:06.670 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:06.671 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:22:06.671 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:06.712 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 85 k 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:22:06.713 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:06.714 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 85 k 2024-09-19T11:22:06.714 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 231 k 2024-09-19T11:22:06.714 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:22:06.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rgw 2024-09-19T11:22:06.916 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-fuse-19.3.0-5012.g08af21e0.el9.x86_64.rpm 422 kB/s | 85 kB 00:00 2024-09-19T11:22:06.916 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:06.917 INFO:teuthology.orchestra.run.smithi028.stdout:Total 420 kB/s | 85 kB 00:00 2024-09-19T11:22:06.917 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:22:06.924 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:22:06.925 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:22:06.991 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:22:06.992 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:22:07.143 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:22:07.233 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:07.347 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:48 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:07.441 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rgw-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:07.488 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:07.489 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:22:07.489 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:07.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-cephfs 2024-09-19T11:22:07.686 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:07.944 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:07.944 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:07.944 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:22:07.944 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:07.944 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:07.944 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:08.166 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:49 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:08.245 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-mirror 2024-09-19T11:22:08.259 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-cephfs-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:08.305 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:08.306 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:22:08.306 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:08.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install python3-rbd 2024-09-19T11:22:08.858 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:50 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:08.980 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:49 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:09.001 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:09.001 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:09.001 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repo Size 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 3.2 M 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:22:09.002 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:09.003 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 3.2 M 2024-09-19T11:22:09.003 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 14 M 2024-09-19T11:22:09.003 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:22:09.074 INFO:teuthology.orchestra.run.smithi022.stdout:Package python3-rbd-2:19.3.0-5012.g08af21e0.el9.x86_64 is already installed. 2024-09-19T11:22:09.120 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:09.121 INFO:teuthology.orchestra.run.smithi022.stdout:Nothing to do. 2024-09-19T11:22:09.121 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:09.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-fuse 2024-09-19T11:22:09.352 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-mirror-19.3.0-5012.g08af21e0.el9.x86_64.rpm 9.2 MB/s | 3.2 MB 00:00 2024-09-19T11:22:09.352 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:09.353 INFO:teuthology.orchestra.run.smithi028.stdout:Total 9.1 MB/s | 3.2 MB 00:00 2024-09-19T11:22:09.353 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:22:09.363 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:22:09.363 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:22:09.450 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:22:09.450 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:22:09.793 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:50 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:09.921 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:22:09.931 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:09.931 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:09.931 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 85 k 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:22:09.932 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:09.933 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 85 k 2024-09-19T11:22:09.933 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 231 k 2024-09-19T11:22:09.933 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:22:09.969 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:09.996 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:09.996 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:22:09.996 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T11:22:09.997 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T11:22:09.997 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T11:22:09.997 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:10.124 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-fuse-19.3.0-5012.g08af21e0.el9.x86_64.rpm 446 kB/s | 85 kB 00:00 2024-09-19T11:22:10.125 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:10.125 INFO:teuthology.orchestra.run.smithi022.stdout:Total 442 kB/s | 85 kB 00:00 2024-09-19T11:22:10.125 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:22:10.132 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:22:10.132 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:22:10.199 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:22:10.199 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:22:10.353 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:22:10.457 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:10.849 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:11.106 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:11.106 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:11.106 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:22:11.107 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:11.107 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:11.107 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:11.117 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:22:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:11.117 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:11.117 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:11.355 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install rbd-nbd 2024-09-19T11:22:11.370 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-mirror 2024-09-19T11:22:11.969 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:01:53 ago on Thu 19 Sep 2024 11:20:18 AM UTC. 2024-09-19T11:22:11.996 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:52 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:12.107 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-19T11:22:12.108 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:12.108 INFO:teuthology.orchestra.run.smithi028.stdout: Package Architecture Version Repository Size 2024-09-19T11:22:12.108 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 171 k 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout:================================================================================ 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout:Install 1 Package 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:12.109 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 171 k 2024-09-19T11:22:12.110 INFO:teuthology.orchestra.run.smithi028.stdout:Installed size: 490 k 2024-09-19T11:22:12.110 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-19T11:22:12.137 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:12.138 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:12.138 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repo Size 2024-09-19T11:22:12.138 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:12.138 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:22:12.138 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 3.2 M 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 3.2 M 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 14 M 2024-09-19T11:22:12.139 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:22:12.312 INFO:teuthology.orchestra.run.smithi028.stdout:rbd-nbd-19.3.0-5012.g08af21e0.el9.x86_64.rpm 838 kB/s | 171 kB 00:00 2024-09-19T11:22:12.313 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:12.313 INFO:teuthology.orchestra.run.smithi028.stdout:Total 833 kB/s | 171 kB 00:00 2024-09-19T11:22:12.313 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-19T11:22:12.322 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-19T11:22:12.322 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-19T11:22:12.389 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-19T11:22:12.390 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-19T11:22:12.485 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-mirror-19.3.0-5012.g08af21e0.el9.x86_64.rpm 9.2 MB/s | 3.2 MB 00:00 2024-09-19T11:22:12.486 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:12.487 INFO:teuthology.orchestra.run.smithi022.stdout:Total 9.2 MB/s | 3.2 MB 00:00 2024-09-19T11:22:12.487 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:22:12.497 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:22:12.497 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:22:12.551 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-19T11:22:12.583 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:22:12.584 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:22:12.655 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:13.059 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:22:13.073 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:13.120 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:13.147 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:13.147 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T11:22:13.147 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T11:22:13.147 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T11:22:13.147 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T11:22:13.147 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:13.339 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:13.339 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:13.339 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-19T11:22:13.339 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:13.339 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:13.339 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-19T11:22:13.576 DEBUG:teuthology.parallel:result is None 2024-09-19T11:22:14.447 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:14.447 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:14.447 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:22:14.447 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:14.447 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:14.447 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:14.683 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install rbd-nbd 2024-09-19T11:22:15.298 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:01:56 ago on Thu 19 Sep 2024 11:20:19 AM UTC. 2024-09-19T11:22:15.436 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-19T11:22:15.436 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:15.436 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:19.3.0-5012.g08af21e0.el9 ceph 171 k 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-09-19T11:22:15.437 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:15.438 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 171 k 2024-09-19T11:22:15.438 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 490 k 2024-09-19T11:22:15.438 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-19T11:22:15.640 INFO:teuthology.orchestra.run.smithi022.stdout:rbd-nbd-19.3.0-5012.g08af21e0.el9.x86_64.rpm 846 kB/s | 171 kB 00:00 2024-09-19T11:22:15.640 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-19T11:22:15.641 INFO:teuthology.orchestra.run.smithi022.stdout:Total 840 kB/s | 171 kB 00:00 2024-09-19T11:22:15.641 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-19T11:22:15.649 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-19T11:22:15.649 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-19T11:22:15.719 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-19T11:22:15.719 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-19T11:22:15.879 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-19T11:22:15.990 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:16.415 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:16.682 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 1/1 2024-09-19T11:22:16.683 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:16.683 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-19T11:22:16.683 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:19.3.0-5012.g08af21e0.el9.x86_64 2024-09-19T11:22:16.683 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:16.683 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-19T11:22:16.927 DEBUG:teuthology.parallel:result is None 2024-09-19T11:22:16.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08af21e0d16078b88ddf808fe9ba2ab84dbe6754 2024-09-19T11:22:17.086 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T11:22:17.108 INFO:teuthology.orchestra.run.smithi022.stdout:19.3.0-5012.g08af21e0.el9 2024-09-19T11:22:17.109 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5012.g08af21e0.el9 2024-09-19T11:22:17.109 INFO:teuthology.task.install:The correct ceph version 19.3.0-5012.g08af21e0 is installed. 2024-09-19T11:22:17.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=08af21e0d16078b88ddf808fe9ba2ab84dbe6754 2024-09-19T11:22:17.264 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T11:22:17.290 INFO:teuthology.orchestra.run.smithi028.stdout:19.3.0-5012.g08af21e0.el9 2024-09-19T11:22:17.290 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5012.g08af21e0.el9 2024-09-19T11:22:17.290 INFO:teuthology.task.install:The correct ceph version 19.3.0-5012.g08af21e0 is installed. 2024-09-19T11:22:17.292 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T11:22:17.292 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:17.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T11:22:17.321 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:17.321 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T11:22:17.361 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T11:22:17.361 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:17.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T11:22:17.389 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T11:22:17.457 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:17.457 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T11:22:17.487 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T11:22:17.556 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T11:22:17.557 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:17.557 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T11:22:17.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T11:22:17.651 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:17.651 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T11:22:17.681 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T11:22:17.752 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T11:22:17.752 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:17.753 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T11:22:17.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T11:22:17.845 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:17.846 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T11:22:17.876 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T11:22:17.946 INFO:teuthology.run_tasks:Running task ceph... 2024-09-19T11:22:18.023 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-19T11:22:18.023 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2024-09-19T11:22:18.026 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 777 /var/log/ceph 2024-09-19T11:22:18.055 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-19T11:22:18.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T11:22:18.094 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T11:22:18.123 INFO:tasks.ceph:Creating extra log directories... 2024-09-19T11:22:18.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T11:22:18.161 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T11:22:18.199 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-19T11:22:18.199 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-19T11:22:18.199 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434', 'branch': 'wip-yuval-test', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}}', 'email': None, 'first_in_suite': False, 'job_id': '7912434', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuval-test'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754'}}, 'rgw': {'ec-data-pool': True, 'frontend': 'beast'}, 's3tests': {'slow_backend': True}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuval-test', 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754'}}, 'owner': 'scheduled_yuvalif@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 4251, 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754', 'sleep_before_teardown': 0, 'subset': '1/3', 'suite': 'rgw', 'suite_branch': 'wip-yuval-test', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_08af21e0d16078b88ddf808fe9ba2ab84dbe6754/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754', 'targets': {'smithi022.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBImOkcQMyTTsxZiTVVPdRrkNVGDdXuiUSCHaj35l0O1g/LEZt8LQeLO0yDuqCM2XEh54Yi6Np2VyPaIhVs1bRcc=', 'smithi028.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBMGD8ZDswDSqOs0qmNxS6fFMcpqZhrjRjpde0qAFVfctzs4Q03rGA9Ia2s82d6gFMC/XQRgqx1XCIvBG27Icg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '08af21e0d16078b88ddf808fe9ba2ab84dbe6754', 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-19_08:49:34', 'tube': 'smithi', 'user': 'yuvalif', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-19T11:22:18.200 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T11:22:18.227 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T11:22:18.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T11:22:18.283 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T11:22:18.316 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:18.316 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T11:22:18.365 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T11:22:18.365 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 11:22:13.401684769 +0000 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 11:19:39.763460152 +0000 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 11:19:39.763460152 +0000 2024-09-19T11:22:18.421 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 11:19:39.763460152 +0000 2024-09-19T11:22:18.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T11:22:18.484 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T11:22:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T11:22:18.485 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.00010155 s, 5.0 MB/s 2024-09-19T11:22:18.486 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T11:22:18.543 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-09-19T11:22:18.599 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T11:22:18.599 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:18.599 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-19T11:22:18.599 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:18.600 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:18.600 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 11:22:13.401684769 +0000 2024-09-19T11:22:18.600 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 11:19:40.077455761 +0000 2024-09-19T11:22:18.600 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 11:19:40.077455761 +0000 2024-09-19T11:22:18.600 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 11:19:40.077455761 +0000 2024-09-19T11:22:18.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T11:22:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T11:22:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T11:22:18.663 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000101291 s, 5.1 MB/s 2024-09-19T11:22:18.664 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T11:22:18.721 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 11:22:13.402684755 +0000 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 11:19:40.408451132 +0000 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 11:19:40.408451132 +0000 2024-09-19T11:22:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 11:19:40.408451132 +0000 2024-09-19T11:22:18.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T11:22:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T11:22:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T11:22:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 9.8085e-05 s, 5.2 MB/s 2024-09-19T11:22:18.843 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T11:22:18.901 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-19 11:22:13.402684755 +0000 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-19 11:19:40.729446642 +0000 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-19 11:19:40.729446642 +0000 2024-09-19T11:22:18.957 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-19 11:19:40.729446642 +0000 2024-09-19T11:22:18.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T11:22:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-19T11:22:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-19T11:22:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000107168 s, 4.8 MB/s 2024-09-19T11:22:19.022 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T11:22:19.079 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-09-19T11:22:19.079 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:19.079 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T11:22:19.099 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T11:22:19.099 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_1 2024-09-19T11:22:19.158 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T11:22:19.158 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:19.159 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-19T11:22:19.159 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:19.159 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:19.159 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 11:22:10.265337104 +0000 2024-09-19T11:22:19.159 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 11:19:40.306982856 +0000 2024-09-19T11:22:19.159 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 11:19:40.306982856 +0000 2024-09-19T11:22:19.159 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 11:19:40.306982856 +0000 2024-09-19T11:22:19.160 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T11:22:19.227 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T11:22:19.227 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T11:22:19.227 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.00010219 s, 5.0 MB/s 2024-09-19T11:22:19.228 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T11:22:19.291 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_2 2024-09-19T11:22:19.350 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T11:22:19.350 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:19.350 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-19T11:22:19.350 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:19.350 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:19.350 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 11:22:10.265337104 +0000 2024-09-19T11:22:19.351 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 11:19:40.653973002 +0000 2024-09-19T11:22:19.351 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 11:19:40.653973002 +0000 2024-09-19T11:22:19.351 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 11:19:40.653973002 +0000 2024-09-19T11:22:19.351 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T11:22:19.418 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T11:22:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T11:22:19.419 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000119118 s, 4.3 MB/s 2024-09-19T11:22:19.420 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T11:22:19.479 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_3 2024-09-19T11:22:19.538 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T11:22:19.538 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:19.539 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-19T11:22:19.539 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:19.539 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:19.539 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 11:22:10.265337104 +0000 2024-09-19T11:22:19.539 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 11:19:40.971963970 +0000 2024-09-19T11:22:19.539 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 11:19:40.971963970 +0000 2024-09-19T11:22:19.539 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 11:19:40.971963970 +0000 2024-09-19T11:22:19.539 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T11:22:19.606 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T11:22:19.606 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T11:22:19.606 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 9.7036e-05 s, 5.3 MB/s 2024-09-19T11:22:19.607 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T11:22:19.666 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_4 2024-09-19T11:22:19.724 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-19 11:22:10.266337076 +0000 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-19 11:19:41.291954883 +0000 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-19 11:19:41.291954883 +0000 2024-09-19T11:22:19.725 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-19 11:19:41.291954883 +0000 2024-09-19T11:22:19.725 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T11:22:19.793 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-19T11:22:19.793 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-19T11:22:19.793 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000106508 s, 4.8 MB/s 2024-09-19T11:22:19.794 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T11:22:19.854 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-19T11:22:19.854 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-09-19T11:22:19.854 INFO:tasks.ceph:Generating config... 2024-09-19T11:22:19.855 INFO:tasks.ceph:[client] debug ms = 1 2024-09-19T11:22:19.855 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-09-19T11:22:19.855 INFO:tasks.ceph:[client] debug rgw = 20 2024-09-19T11:22:19.855 INFO:tasks.ceph:[client] setgroup = ceph 2024-09-19T11:22:19.855 INFO:tasks.ceph:[client] setuser = ceph 2024-09-19T11:22:19.855 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] debug objclass = 20 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-19T11:22:19.856 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-19T11:22:19.857 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-19T11:22:19.857 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-19T11:22:19.857 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-19T11:22:19.857 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-19T11:22:19.857 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-19T11:22:19.857 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-19T11:22:19.857 INFO:tasks.ceph:Setting up mon.a... 2024-09-19T11:22:19.857 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-19T11:22:19.915 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.keyring 2024-09-19T11:22:19.918 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-19T11:22:19.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T11:22:19.993 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.22')] 2024-09-19T11:22:19.993 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.22'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-09-19T11:22:19.994 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:19.994 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T11:22:20.050 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.22 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:22:20.132 INFO:teuthology.orchestra.run.smithi022.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:22:20.132 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:20.132 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: generated fsid 200d5f3b-d410-4b92-82cf-1cbe85431f50 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 0 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:fsid 200d5f3b-d410-4b92-82cf-1cbe85431f50 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:last_changed 2024-09-19T11:22:20.132195+0000 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-09-19T11:22:20.132195+0000 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:min_mon_release 17 (quincy) 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:election_strategy: 1 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-19T11:22:20.133 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-19T11:22:20.134 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T11:22:20.190 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 200d5f3b-d410-4b92-82cf-1cbe85431f50... 2024-09-19T11:22:20.191 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T11:22:20.232 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-09-19T11:22:20.275 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.276 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T11:22:20.277 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = 172.21.15.22 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 200d5f3b-d410-4b92-82cf-1cbe85431f50 2024-09-19T11:22:20.278 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-09-19T11:22:20.279 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-09-19T11:22:20.280 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: debug objclass = 20 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-09-19T11:22:20.281 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-19T11:22:20.282 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T11:22:20.283 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-09-19T11:22:20.284 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: rgw gc obj min wait = 15 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: debug rgw = 20 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: setgroup = ceph 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout: setuser = ceph 2024-09-19T11:22:20.285 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout:[global] 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: chdir = "" 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: auth supported = cephx 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: filestore xattr use omap = true 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: mon clock drift allowed = 1.000 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: osd crush chooseleaf type = 0 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: auth debug = true 2024-09-19T11:22:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on old message = true 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on bug = true 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: mon pg warn max object skew = 0 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default size = 2 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow primary affinity = true 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow pg remap = true 2024-09-19T11:22:20.287 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on legacy crush tunables = false 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on crush straw calc version zero = false 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on no sortbitwise = false 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on osd down out interval zero = false 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on too few osds = false 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon_allow_pool_size_one = true 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: osd default data pool replay window = 5 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: mon allow pool delete = true 2024-09-19T11:22:20.288 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: mon cluster log file level = debug 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: debug asserts on shutdown = true 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: mon health detail to clog = false 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: mon host = 172.21.15.22 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: fsid = 200d5f3b-d410-4b92-82cf-1cbe85431f50 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout:[osd] 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: osd journal size = 100 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub load threshold = 5.0 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub max interval = 600 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: osd mclock profile = high_recovery_ops 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: osd recover clone overlap = true 2024-09-19T11:22:20.289 INFO:teuthology.orchestra.run.smithi028.stdout: osd recovery max chunk = 1048576 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug shutdown = true 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug op order = true 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug verify stray on activate = true 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug trim objects = true 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd open classes on start = true 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug pg log writeout = true 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd deep scrub update digest min age = 30 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: osd map max advance = 10 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.290 INFO:teuthology.orchestra.run.smithi028.stdout: journal zero on create = true 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: filestore ondisk finisher threads = 3 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: filestore apply finisher threads = 3 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: bdev debug aio = true 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug misdirected ops = true 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: debug objclass = 20 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: bdev async discard = True 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: bdev enable discard = True 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore allocator = bitmap 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore block size = 96636764160 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore fsck on mount = True 2024-09-19T11:22:20.291 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluefs = 1/20 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluestore = 1/20 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: debug osd = 20 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: debug rocksdb = 4/10 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd full ratio = 0.9 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd nearfull ratio = 0.8 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: osd failsafe full ratio = 0.95 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: osd objectstore = bluestore 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr] 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: debug mgr = 20 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-09-19T11:22:20.292 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mgr/telemetry/nag = false 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout:[mon] 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: debug paxos = 20 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mon data avail warn = 5 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mon mgr mkfs grace = 240 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd reporter subtree level = osd 2024-09-19T11:22:20.293 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd prime pg temp = true 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: # 1m isn't quite enough 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: mon_down_mkfs_grace = 2m 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.294 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_filestore_osds = false 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout:[client] 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: rgw cache enabled = true 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable ops log = true 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable usage log = true 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: rgw gc obj min wait = 15 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: debug rgw = 20 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: setgroup = ceph 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout: setuser = ceph 2024-09-19T11:22:20.295 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.a] 2024-09-19T11:22:20.298 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-19T11:22:20.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-19T11:22:20.393 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-19T11:22:20.393 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:20.393 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-19T11:22:20.409 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:20.409 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-19T11:22:20.464 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T11:22:20.465 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:20.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T11:22:20.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T11:22:20.540 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:20.540 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:20.595 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:22:20.596 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:20.596 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T11:22:20.596 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T11:22:20.638 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:20.638 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:20.696 INFO:tasks.ceph:Setting up mon nodes... 2024-09-19T11:22:20.696 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-19T11:22:20.696 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T11:22:20.800 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T11:22:20.803 INFO:tasks.ceph:Setting up mds nodes... 2024-09-19T11:22:20.803 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-19T11:22:20.803 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T11:22:20.848 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-19T11:22:20.861 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-19T11:22:20.862 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-09-19T11:22:20.862 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-19T11:22:20.887 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-09-19T11:22:20.887 INFO:tasks.ceph:role: osd.0 2024-09-19T11:22:20.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T11:22:20.888 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T11:22:20.954 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T11:22:20.986 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-09-19T11:22:20.988 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-09-19T11:22:20.989 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-19T11:22:21.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-19T11:22:21.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:21.233+0000 7f4f64a5e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-19T11:22:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:21.233+0000 7f4f64a5e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-19T11:22:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:21.233+0000 7f4f64a5e640 -1 bdev(0x5558fa09f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-19T11:22:21.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:21.233+0000 7f4f64a5e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-19T11:22:24.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-19T11:22:24.447 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-19T11:22:24.478 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-19T11:22:24.478 INFO:tasks.ceph:role: osd.1 2024-09-19T11:22:24.478 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:22:24.478 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T11:22:24.548 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T11:22:24.548 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T11:22:24.548 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T11:22:24.548 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T11:22:24.548 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T11:22:24.549 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-09-19T11:22:24.549 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T11:22:24.549 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T11:22:24.549 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T11:22:24.549 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T11:22:24.581 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-09-19T11:22:24.584 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-09-19T11:22:24.584 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-19T11:22:24.656 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-19T11:22:24.727 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-19T11:22:24.794 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-19T11:22:24.794 INFO:tasks.ceph:role: osd.2 2024-09-19T11:22:24.794 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:22:24.794 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T11:22:24.859 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T11:22:24.860 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T11:22:24.860 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T11:22:24.890 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-09-19T11:22:24.893 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-09-19T11:22:24.894 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-09-19T11:22:24.963 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-19T11:22:25.034 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-19T11:22:25.100 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-19T11:22:25.100 INFO:tasks.ceph:role: osd.3 2024-09-19T11:22:25.100 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:22:25.100 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T11:22:25.166 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T11:22:25.199 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-09-19T11:22:25.200 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-09-19T11:22:25.200 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-09-19T11:22:25.274 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-19T11:22:25.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:25.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:25.456+0000 7f8ac0a11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-19T11:22:25.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:25.456+0000 7f8ac0a11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-19T11:22:25.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:25.456+0000 7f8ac0a11640 -1 bdev(0x55a75bbeb800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-19T11:22:25.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:25.456+0000 7f8ac0a11640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-19T11:22:28.623 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-19T11:22:28.655 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:28.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:28.744+0000 7fc547a16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-19T11:22:28.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:28.744+0000 7fc547a16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-19T11:22:28.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:28.744+0000 7fc547a16640 -1 bdev(0x55f5d594d800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-19T11:22:28.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:28.744+0000 7fc547a16640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-19T11:22:31.657 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-19T11:22:31.688 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:31.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:31.778+0000 7f6087302640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-19T11:22:31.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:31.778+0000 7f6087302640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-19T11:22:31.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:31.778+0000 7f6087302640 -1 bdev(0x562c44f31800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-19T11:22:31.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:31.778+0000 7f6087302640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-19T11:22:34.932 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T11:22:34.962 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-19T11:22:34.962 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:34.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-19T11:22:34.995 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:34.995 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-19T11:22:35.032 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:35.032 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-19T11:22:35.100 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:35.100 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-19T11:22:35.167 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:35.168 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-19T11:22:35.236 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:35.236 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-19T11:22:35.293 INFO:tasks.ceph:Adding keys to all mons... 2024-09-19T11:22:35.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.0] 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBtCexmCsv5DRAATCvzU7wtb5XjxdqxqIgTEw== 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.x] 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBsCexm71KwLxAACaTyaxRWYkdQXgyWyOOE8w== 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.1] 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBxCexmwnNPGxAAgtg33S09PpxPflD0KtUk6Q== 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.2] 2024-09-19T11:22:35.322 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQB0CexmBtB3LBAAHJEJzRSfk73Z1X3Pg+tZ4Q== 2024-09-19T11:22:35.323 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.3] 2024-09-19T11:22:35.323 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQB3CexmqXF1LhAAmg4j/a7HLrrfMPYHqymDNA== 2024-09-19T11:22:35.323 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-19T11:22:35.323 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBsCexmdL2IMhAAgW7yJ2AEbmiuW3ky9EgOQA== 2024-09-19T11:22:35.324 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T11:22:35.423 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T11:22:35.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T11:22:35.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T11:22:35.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T11:22:35.822 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T11:22:35.878 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-19T11:22:35.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-19T11:22:35.904 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T11:22:36.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-19T11:22:36.027 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T11:22:36.083 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-19T11:22:36.083 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-19T11:22:36.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-19T11:22:36.125 INFO:tasks.ceph.mon.a:Started 2024-09-19T11:22:36.126 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-19T11:22:36.126 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-19T11:22:36.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-19T11:22:36.129 INFO:tasks.ceph.mgr.x:Started 2024-09-19T11:22:36.130 DEBUG:tasks.ceph:set 0 configs 2024-09-19T11:22:36.130 INFO:tasks.ceph:Setting crush tunables to default 2024-09-19T11:22:36.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd crush tunables default 2024-09-19T11:22:36.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.229+0000 7fb064dbc640 1 Processor -- start 2024-09-19T11:22:36.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.229+0000 7fb064dbc640 1 -- start start 2024-09-19T11:22:36.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb064dbc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb06012f520 0x7fb06012f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb064dbc640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb06006fd20 con 0x7fb06012fe40 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb064dbc640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb06006fef0 con 0x7fb06012f520 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb05e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb06012f520 0x7fb06012f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb05dd9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb06012fe40 0x7fb06012c020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39542/0 (socket says 172.21.15.22:39542) 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb05e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb06012f520 0x7fb06012f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55572/0 (socket says 172.21.15.22:55572) 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb05dd9b640 1 -- 172.21.15.22:0/4120492189 learned_addr learned my addr 172.21.15.22:0/4120492189 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb05e59c640 1 -- 172.21.15.22:0/4120492189 >> v1:172.21.15.22:6789/0 conn(0x7fb06012fe40 legacy=0x7fb06012c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.230+0000 7fb05e59c640 1 -- 172.21.15.22:0/4120492189 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb060070490 con 0x7fb06012f520 2024-09-19T11:22:36.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb05e59c640 1 --2- 172.21.15.22:0/4120492189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb06012f900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb050009720 tx=0x7fb05002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7bcca8bf84b848 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:36.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb05d59a640 1 -- 172.21.15.22:0/4120492189 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb050037070 con 0x7fb06012f520 2024-09-19T11:22:36.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb05d59a640 1 -- 172.21.15.22:0/4120492189 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05002ad90 con 0x7fb06012f520 2024-09-19T11:22:36.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb05d59a640 1 -- 172.21.15.22:0/4120492189 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb050032590 con 0x7fb06012f520 2024-09-19T11:22:36.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb064dbc640 1 -- 172.21.15.22:0/4120492189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 msgr2=0x7fb06012f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb064dbc640 1 --2- 172.21.15.22:0/4120492189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb06012f900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb050009720 tx=0x7fb05002a110 comp rx=0 tx=0).stop 2024-09-19T11:22:36.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb064dbc640 1 -- 172.21.15.22:0/4120492189 shutdown_connections 2024-09-19T11:22:36.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb064dbc640 1 --2- 172.21.15.22:0/4120492189 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb06012f900 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb064dbc640 1 -- 172.21.15.22:0/4120492189 >> 172.21.15.22:0/4120492189 conn(0x7fb0600a2080 msgr2=0x7fb0600a2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:36.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb064dbc640 1 -- 172.21.15.22:0/4120492189 shutdown_connections 2024-09-19T11:22:36.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.232+0000 7fb064dbc640 1 -- 172.21.15.22:0/4120492189 wait complete. 2024-09-19T11:22:36.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.233+0000 7fb064dbc640 1 Processor -- start 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.233+0000 7fb064dbc640 1 -- start start 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb064dbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb0600903f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb064dbc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb060130460 con 0x7fb06012f520 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb05e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb0600903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb05e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb0600903f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55580/0 (socket says 172.21.15.22:55580) 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb05e59c640 1 -- 172.21.15.22:0/1679816458 learned_addr learned my addr 172.21.15.22:0/1679816458 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb05e59c640 1 -- 172.21.15.22:0/1679816458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06012c370 con 0x7fb06012f520 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb05e59c640 1 --2- 172.21.15.22:0/1679816458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb0600903f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a150 tx=0x7fb0500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb056ffd640 1 -- 172.21.15.22:0/1679816458 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb050041020 con 0x7fb06012f520 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb056ffd640 1 -- 172.21.15.22:0/1679816458 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb05003d550 con 0x7fb06012f520 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0600740f0 con 0x7fb06012f520 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.234+0000 7fb056ffd640 1 -- 172.21.15.22:0/1679816458 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb050037040 con 0x7fb06012f520 2024-09-19T11:22:36.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.235+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0600933b0 con 0x7fb06012f520 2024-09-19T11:22:36.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.235+0000 7fb056ffd640 1 -- 172.21.15.22:0/1679816458 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb050044020 con 0x7fb06012f520 2024-09-19T11:22:36.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.235+0000 7fb056ffd640 1 -- 172.21.15.22:0/1679816458 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb050046ad0 con 0x7fb06012f520 2024-09-19T11:22:36.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.235+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb06012f900 con 0x7fb06012f520 2024-09-19T11:22:36.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.238+0000 7fb056ffd640 1 -- 172.21.15.22:0/1679816458 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb05004b030 con 0x7fb06012f520 2024-09-19T11:22:36.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.302+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb060075000 con 0x7fb06012f520 2024-09-19T11:22:36.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.303+0000 7fb056ffd640 1 -- 172.21.15.22:0/1679816458 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fb05003dda0 con 0x7fb06012f520 2024-09-19T11:22:36.305 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-09-19T11:22:36.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.304+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 msgr2=0x7fb0600903f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.304+0000 7fb064dbc640 1 --2- 172.21.15.22:0/1679816458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb0600903f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a150 tx=0x7fb0500046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.304+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 shutdown_connections 2024-09-19T11:22:36.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.304+0000 7fb064dbc640 1 --2- 172.21.15.22:0/1679816458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb06012f520 0x7fb0600903f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.304+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 >> 172.21.15.22:0/1679816458 conn(0x7fb0600a2080 msgr2=0x7fb060072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:36.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.304+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 shutdown_connections 2024-09-19T11:22:36.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:36.304+0000 7fb064dbc640 1 -- 172.21.15.22:0/1679816458 wait complete. 2024-09-19T11:22:36.316 INFO:tasks.ceph:check_enable_crimson: False 2024-09-19T11:22:36.316 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-19T11:22:36.316 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-19T11:22:36.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-19T11:22:36.341 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:36.341 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-19T11:22:36.369 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:36.369 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-19T11:22:36.437 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-19T11:22:36.438 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-19T11:22:36.505 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd new f0cbb56a-4180-4b0e-948a-cb474e8486a2 0 2024-09-19T11:22:36.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.645+0000 7f6e7cb82640 1 Processor -- start 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.645+0000 7f6e7cb82640 1 -- start start 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e7cb82640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6e7811bad0 0x7f6e7811dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e7cb82640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e7811e490 con 0x7f6e78116b50 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e7cb82640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7811f670 con 0x7f6e7811bad0 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e6ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6e7811bad0 0x7f6e7811dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e6ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6e7811bad0 0x7f6e7811dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34802/0 (socket says 172.21.15.28:34802) 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e6ffff640 1 -- 172.21.15.28:0/2794407554 learned_addr learned my addr 172.21.15.28:0/2794407554 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e6ffff640 1 -- 172.21.15.28:0/2794407554 >> v1:172.21.15.22:6789/0 conn(0x7f6e78116b50 legacy=0x7f6e78116f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.646+0000 7f6e6ffff640 1 -- 172.21.15.28:0/2794407554 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7811f7f0 con 0x7f6e7811bad0 2024-09-19T11:22:36.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.647+0000 7f6e6ffff640 1 --2- 172.21.15.28:0/2794407554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e7811bad0 0x7f6e7811dec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e600097f0 tx=0x7f6e6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90008ca5e84e494 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:36.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.647+0000 7f6e75d9b640 1 -- 172.21.15.28:0/2794407554 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e60037070 con 0x7f6e7811bad0 2024-09-19T11:22:36.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.647+0000 7f6e75d9b640 1 -- 172.21.15.28:0/2794407554 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e60032440 con 0x7f6e7811bad0 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.647+0000 7f6e75d9b640 1 -- 172.21.15.28:0/2794407554 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e6002ae60 con 0x7f6e7811bad0 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.648+0000 7f6e7cb82640 1 -- 172.21.15.28:0/2794407554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e7811bad0 msgr2=0x7f6e7811dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.648+0000 7f6e7cb82640 1 --2- 172.21.15.28:0/2794407554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e7811bad0 0x7f6e7811dec0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e600097f0 tx=0x7f6e6002a1e0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.648+0000 7f6e7cb82640 1 -- 172.21.15.28:0/2794407554 shutdown_connections 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.648+0000 7f6e7cb82640 1 --2- 172.21.15.28:0/2794407554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e7811bad0 0x7f6e7811dec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.648+0000 7f6e7cb82640 1 -- 172.21.15.28:0/2794407554 >> 172.21.15.28:0/2794407554 conn(0x7f6e7806f730 msgr2=0x7f6e7806fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.648+0000 7f6e7cb82640 1 -- 172.21.15.28:0/2794407554 shutdown_connections 2024-09-19T11:22:36.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.648+0000 7f6e7cb82640 1 -- 172.21.15.28:0/2794407554 wait complete. 2024-09-19T11:22:36.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.649+0000 7f6e7cb82640 1 Processor -- start 2024-09-19T11:22:36.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.649+0000 7f6e7cb82640 1 -- start start 2024-09-19T11:22:36.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.649+0000 7f6e7cb82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e78116b50 0x7f6e781a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:36.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.649+0000 7f6e7cb82640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7811eb40 con 0x7f6e78116b50 2024-09-19T11:22:36.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.649+0000 7f6e7659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e78116b50 0x7f6e781a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:36.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.649+0000 7f6e7659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e78116b50 0x7f6e781a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34816/0 (socket says 172.21.15.28:34816) 2024-09-19T11:22:36.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.649+0000 7f6e7659c640 1 -- 172.21.15.28:0/1826058773 learned_addr learned my addr 172.21.15.28:0/1826058773 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e7659c640 1 -- 172.21.15.28:0/1826058773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e781a7ee0 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e7659c640 1 --2- 172.21.15.28:0/1826058773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e78116b50 0x7f6e781a7910 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c480 tx=0x7f6e6800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e6effd640 1 -- 172.21.15.28:0/1826058773 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e68017070 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e6effd640 1 -- 172.21.15.28:0/1826058773 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e68005200 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e6effd640 1 -- 172.21.15.28:0/1826058773 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e680035f0 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e781a81d0 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e781ad3a0 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e6effd640 1 -- 172.21.15.28:0/1826058773 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e6801a070 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.650+0000 7f6e6effd640 1 -- 172.21.15.28:0/1826058773 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f6e6801c440 con 0x7f6e78116b50 2024-09-19T11:22:36.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.651+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e781a8510 con 0x7f6e78116b50 2024-09-19T11:22:36.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.653+0000 7f6e6effd640 1 -- 172.21.15.28:0/1826058773 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6e68021020 con 0x7f6e78116b50 2024-09-19T11:22:36.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.726+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0cbb56a-4180-4b0e-948a-cb474e8486a2", "id": 0} v 0) -- 0x7f6e7811f120 con 0x7f6e78116b50 2024-09-19T11:22:36.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.727+0000 7f6e6effd640 1 -- 172.21.15.28:0/1826058773 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0cbb56a-4180-4b0e-948a-cb474e8486a2", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f6e68011590 con 0x7f6e78116b50 2024-09-19T11:22:36.728 INFO:teuthology.orchestra.run.smithi028.stdout:0 2024-09-19T11:22:36.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.728+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e78116b50 msgr2=0x7f6e781a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.728+0000 7f6e7cb82640 1 --2- 172.21.15.28:0/1826058773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e78116b50 0x7f6e781a7910 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c480 tx=0x7f6e6800c940 comp rx=0 tx=0).stop 2024-09-19T11:22:36.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.728+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 shutdown_connections 2024-09-19T11:22:36.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.728+0000 7f6e7cb82640 1 --2- 172.21.15.28:0/1826058773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6e78116b50 0x7f6e781a7910 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.728+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 >> 172.21.15.28:0/1826058773 conn(0x7f6e7806f730 msgr2=0x7f6e7811beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:36.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.728+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 shutdown_connections 2024-09-19T11:22:36.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.728+0000 7f6e7cb82640 1 -- 172.21.15.28:0/1826058773 wait complete. 2024-09-19T11:22:36.740 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd new f8538703-1f8e-48d3-8d2e-d423b3fb007a 1 2024-09-19T11:22:36.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.839+0000 7f9abf0ed640 1 Processor -- start 2024-09-19T11:22:36.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.839+0000 7f9abf0ed640 1 -- start start 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.839+0000 7f9abf0ed640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9ab8116b50 0x7f9ab8116f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.839+0000 7f9abf0ed640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab8117650 con 0x7f9ab8112dc0 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.839+0000 7f9abf0ed640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab8113960 con 0x7f9ab8116b50 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.840+0000 7f9abce89640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9ab8116b50 0x7f9ab8116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.840+0000 7f9ab7fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9ab8112dc0 0x7f9ab8113250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:47648/0 (socket says 172.21.15.28:47648) 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.840+0000 7f9abce89640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9ab8116b50 0x7f9ab8116f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34830/0 (socket says 172.21.15.28:34830) 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.840+0000 7f9ab7fff640 1 -- 172.21.15.28:0/215968052 learned_addr learned my addr 172.21.15.28:0/215968052 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.840+0000 7f9abce89640 1 -- 172.21.15.28:0/215968052 >> v1:172.21.15.22:6789/0 conn(0x7f9ab8112dc0 legacy=0x7f9ab8113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.840+0000 7f9abce89640 1 -- 172.21.15.28:0/215968052 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab8113b10 con 0x7f9ab8116b50 2024-09-19T11:22:36.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.840+0000 7f9ab77fe640 1 -- 172.21.15.28:0/215968052 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177190451 0 0) 0x7f9ab8117650 con 0x7f9ab8112dc0 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9abce89640 1 --2- 172.21.15.28:0/215968052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8116b50 0x7f9ab8116f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009720 tx=0x7f9aac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194547d941cb6cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9ab77fe640 1 -- 172.21.15.28:0/215968052 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac037070 con 0x7f9ab8116b50 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9ab77fe640 1 -- 172.21.15.28:0/215968052 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aac02ad90 con 0x7f9ab8116b50 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9ab77fe640 1 -- 172.21.15.28:0/215968052 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac032590 con 0x7f9ab8116b50 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9abf0ed640 1 -- 172.21.15.28:0/215968052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8116b50 msgr2=0x7f9ab8116f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9abf0ed640 1 --2- 172.21.15.28:0/215968052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8116b50 0x7f9ab8116f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009720 tx=0x7f9aac02a110 comp rx=0 tx=0).stop 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9abf0ed640 1 -- 172.21.15.28:0/215968052 shutdown_connections 2024-09-19T11:22:36.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9abf0ed640 1 --2- 172.21.15.28:0/215968052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8116b50 0x7f9ab8116f30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.841+0000 7f9abf0ed640 1 -- 172.21.15.28:0/215968052 >> 172.21.15.28:0/215968052 conn(0x7f9ab806f730 msgr2=0x7f9ab806fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.842+0000 7f9abf0ed640 1 -- 172.21.15.28:0/215968052 shutdown_connections 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.842+0000 7f9abf0ed640 1 -- 172.21.15.28:0/215968052 wait complete. 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.842+0000 7f9abf0ed640 1 Processor -- start 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.842+0000 7f9abf0ed640 1 -- start start 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.843+0000 7f9abf0ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8112dc0 0x7f9ab81a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.843+0000 7f9abf0ed640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab811efa0 con 0x7f9ab8112dc0 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.843+0000 7f9abce89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8112dc0 0x7f9ab81a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.843+0000 7f9abce89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8112dc0 0x7f9ab81a7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34840/0 (socket says 172.21.15.28:34840) 2024-09-19T11:22:36.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.843+0000 7f9abce89640 1 -- 172.21.15.28:0/3682603304 learned_addr learned my addr 172.21.15.28:0/3682603304 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:22:36.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.843+0000 7f9abce89640 1 -- 172.21.15.28:0/3682603304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab81a7cd0 con 0x7f9ab8112dc0 2024-09-19T11:22:36.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.843+0000 7f9abce89640 1 --2- 172.21.15.28:0/3682603304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8112dc0 0x7f9ab81a7700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009720 tx=0x7f9aac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:36.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.844+0000 7f9ab57fa640 1 -- 172.21.15.28:0/3682603304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac037040 con 0x7f9ab8112dc0 2024-09-19T11:22:36.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.844+0000 7f9ab57fa640 1 -- 172.21.15.28:0/3682603304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aac004160 con 0x7f9ab8112dc0 2024-09-19T11:22:36.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.844+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab81a7f60 con 0x7f9ab8112dc0 2024-09-19T11:22:36.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.844+0000 7f9ab57fa640 1 -- 172.21.15.28:0/3682603304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac004440 con 0x7f9ab8112dc0 2024-09-19T11:22:36.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.844+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab81acf30 con 0x7f9ab8112dc0 2024-09-19T11:22:36.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.844+0000 7f9ab57fa640 1 -- 172.21.15.28:0/3682603304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9aac0039a0 con 0x7f9ab8112dc0 2024-09-19T11:22:36.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.844+0000 7f9ab57fa640 1 -- 172.21.15.28:0/3682603304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f9aac03dc30 con 0x7f9ab8112dc0 2024-09-19T11:22:36.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.845+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab8116b50 con 0x7f9ab8112dc0 2024-09-19T11:22:36.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.847+0000 7f9ab57fa640 1 -- 172.21.15.28:0/3682603304 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9aac02ace0 con 0x7f9ab8112dc0 2024-09-19T11:22:36.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.920+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f8538703-1f8e-48d3-8d2e-d423b3fb007a", "id": 1} v 0) -- 0x7f9ab8116d20 con 0x7f9ab8112dc0 2024-09-19T11:22:36.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.921+0000 7f9ab57fa640 1 -- 172.21.15.28:0/3682603304 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8538703-1f8e-48d3-8d2e-d423b3fb007a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f9aac03d470 con 0x7f9ab8112dc0 2024-09-19T11:22:36.922 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-19T11:22:36.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.922+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8112dc0 msgr2=0x7f9ab81a7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:36.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.922+0000 7f9abf0ed640 1 --2- 172.21.15.28:0/3682603304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8112dc0 0x7f9ab81a7700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009720 tx=0x7f9aac0046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.922+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 shutdown_connections 2024-09-19T11:22:36.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.922+0000 7f9abf0ed640 1 --2- 172.21.15.28:0/3682603304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ab8112dc0 0x7f9ab81a7700 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:36.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.922+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 >> 172.21.15.28:0/3682603304 conn(0x7f9ab806f730 msgr2=0x7f9ab8111db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:36.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.922+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 shutdown_connections 2024-09-19T11:22:36.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:36.922+0000 7f9abf0ed640 1 -- 172.21.15.28:0/3682603304 wait complete. 2024-09-19T11:22:36.933 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd new e738a240-46c1-4d91-98ec-1e2d64c3f437 2 2024-09-19T11:22:37.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.032+0000 7fa95579b640 1 Processor -- start 2024-09-19T11:22:37.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.032+0000 7fa95579b640 1 -- start start 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.032+0000 7fa95579b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa950116b50 0x7fa950116f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.032+0000 7fa95579b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa950117650 con 0x7fa950112dc0 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.032+0000 7fa95579b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa950113960 con 0x7fa950116b50 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.033+0000 7fa94effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa950116b50 0x7fa950116f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.033+0000 7fa94e7fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa950112dc0 0x7fa950113250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:47654/0 (socket says 172.21.15.28:47654) 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.033+0000 7fa94e7fc640 1 -- 172.21.15.28:0/2010153614 learned_addr learned my addr 172.21.15.28:0/2010153614 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.033+0000 7fa94dffb640 1 -- 172.21.15.28:0/2010153614 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379099024 0 0) 0x7fa950117650 con 0x7fa950112dc0 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.033+0000 7fa94effd640 1 -- 172.21.15.28:0/2010153614 >> v1:172.21.15.22:6789/0 conn(0x7fa950112dc0 legacy=0x7fa950113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.033+0000 7fa94effd640 1 -- 172.21.15.28:0/2010153614 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa950113b10 con 0x7fa950116b50 2024-09-19T11:22:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.034+0000 7fa94effd640 1 --2- 172.21.15.28:0/2010153614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950116b50 0x7fa950116f30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa940009720 tx=0x7fa94002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de98460e45996f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.034+0000 7fa94dffb640 1 -- 172.21.15.28:0/2010153614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa940037070 con 0x7fa950116b50 2024-09-19T11:22:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.034+0000 7fa94dffb640 1 -- 172.21.15.28:0/2010153614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94002ad90 con 0x7fa950116b50 2024-09-19T11:22:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.034+0000 7fa94dffb640 1 -- 172.21.15.28:0/2010153614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa940032590 con 0x7fa950116b50 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 -- 172.21.15.28:0/2010153614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950116b50 msgr2=0x7fa950116f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 --2- 172.21.15.28:0/2010153614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950116b50 0x7fa950116f30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa940009720 tx=0x7fa94002a110 comp rx=0 tx=0).stop 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 -- 172.21.15.28:0/2010153614 shutdown_connections 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 --2- 172.21.15.28:0/2010153614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950116b50 0x7fa950116f30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 -- 172.21.15.28:0/2010153614 >> 172.21.15.28:0/2010153614 conn(0x7fa95006f730 msgr2=0x7fa95006fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 -- 172.21.15.28:0/2010153614 shutdown_connections 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 -- 172.21.15.28:0/2010153614 wait complete. 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.035+0000 7fa95579b640 1 Processor -- start 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa95579b640 1 -- start start 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa95579b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950112dc0 0x7fa9501a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa95579b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa95011efa0 con 0x7fa950112dc0 2024-09-19T11:22:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa94effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950112dc0 0x7fa9501a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa94effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950112dc0 0x7fa9501a7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34860/0 (socket says 172.21.15.28:34860) 2024-09-19T11:22:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa94effd640 1 -- 172.21.15.28:0/3509736739 learned_addr learned my addr 172.21.15.28:0/3509736739 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:22:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa94effd640 1 -- 172.21.15.28:0/3509736739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9501a7cd0 con 0x7fa950112dc0 2024-09-19T11:22:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.036+0000 7fa94effd640 1 --2- 172.21.15.28:0/3509736739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950112dc0 0x7fa9501a7700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa940009720 tx=0x7fa9400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.037+0000 7fa9377fe640 1 -- 172.21.15.28:0/3509736739 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa940037040 con 0x7fa950112dc0 2024-09-19T11:22:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.037+0000 7fa9377fe640 1 -- 172.21.15.28:0/3509736739 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940004160 con 0x7fa950112dc0 2024-09-19T11:22:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.037+0000 7fa9377fe640 1 -- 172.21.15.28:0/3509736739 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa940004440 con 0x7fa950112dc0 2024-09-19T11:22:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.037+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9501a7f60 con 0x7fa950112dc0 2024-09-19T11:22:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.037+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9501acf30 con 0x7fa950112dc0 2024-09-19T11:22:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.037+0000 7fa9377fe640 1 -- 172.21.15.28:0/3509736739 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa9400039a0 con 0x7fa950112dc0 2024-09-19T11:22:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.037+0000 7fa9377fe640 1 -- 172.21.15.28:0/3509736739 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fa94002ace0 con 0x7fa950112dc0 2024-09-19T11:22:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.038+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa950116b50 con 0x7fa950112dc0 2024-09-19T11:22:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.040+0000 7fa9377fe640 1 -- 172.21.15.28:0/3509736739 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa940003c90 con 0x7fa950112dc0 2024-09-19T11:22:37.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.112+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e738a240-46c1-4d91-98ec-1e2d64c3f437", "id": 2} v 0) -- 0x7fa950116d20 con 0x7fa950112dc0 2024-09-19T11:22:37.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.113+0000 7fa9377fe640 1 -- 172.21.15.28:0/3509736739 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e738a240-46c1-4d91-98ec-1e2d64c3f437", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fa94003dbf0 con 0x7fa950112dc0 2024-09-19T11:22:37.115 INFO:teuthology.orchestra.run.smithi028.stdout:2 2024-09-19T11:22:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.114+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950112dc0 msgr2=0x7fa9501a7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.114+0000 7fa95579b640 1 --2- 172.21.15.28:0/3509736739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950112dc0 0x7fa9501a7700 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa940009720 tx=0x7fa9400046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.114+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 shutdown_connections 2024-09-19T11:22:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.114+0000 7fa95579b640 1 --2- 172.21.15.28:0/3509736739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa950112dc0 0x7fa9501a7700 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.114+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 >> 172.21.15.28:0/3509736739 conn(0x7fa95006f730 msgr2=0x7fa950111db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.114+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 shutdown_connections 2024-09-19T11:22:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.114+0000 7fa95579b640 1 -- 172.21.15.28:0/3509736739 wait complete. 2024-09-19T11:22:37.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd new a884f1f1-c119-4e37-b10b-9038bb154d55 3 2024-09-19T11:22:37.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.225+0000 7f7191472640 1 Processor -- start 2024-09-19T11:22:37.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.225+0000 7f7191472640 1 -- start start 2024-09-19T11:22:37.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.225+0000 7f7191472640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f718c11bad0 0x7f718c11dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:37.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.225+0000 7f7191472640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f718c11e490 con 0x7f718c116b50 2024-09-19T11:22:37.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.225+0000 7f7191472640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f718c11f670 con 0x7f718c11bad0 2024-09-19T11:22:37.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.226+0000 7f7183fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f718c11bad0 0x7f718c11dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:37.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.226+0000 7f7183fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f718c11bad0 0x7f718c11dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34876/0 (socket says 172.21.15.28:34876) 2024-09-19T11:22:37.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.226+0000 7f7183fff640 1 -- 172.21.15.28:0/1517643499 learned_addr learned my addr 172.21.15.28:0/1517643499 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:22:37.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.226+0000 7f718affd640 1 --1- 172.21.15.28:0/1517643499 >> v1:172.21.15.22:6789/0 conn(0x7f718c116b50 0x7f718c116f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:47658/0 (socket says 172.21.15.28:47658) 2024-09-19T11:22:37.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.226+0000 7f7183fff640 1 -- 172.21.15.28:0/1517643499 >> v1:172.21.15.22:6789/0 conn(0x7f718c116b50 legacy=0x7f718c116f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.226+0000 7f7183fff640 1 -- 172.21.15.28:0/1517643499 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c11f7f0 con 0x7f718c11bad0 2024-09-19T11:22:37.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.227+0000 7f7183fff640 1 --2- 172.21.15.28:0/1517643499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c11bad0 0x7f718c11dec0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f71740097f0 tx=0x7f717402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6680b2d3fcb7deab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:37.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.227+0000 7f718a7fc640 1 -- 172.21.15.28:0/1517643499 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7174037070 con 0x7f718c11bad0 2024-09-19T11:22:37.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.227+0000 7f718a7fc640 1 -- 172.21.15.28:0/1517643499 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7174032440 con 0x7f718c11bad0 2024-09-19T11:22:37.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.227+0000 7f718a7fc640 1 -- 172.21.15.28:0/1517643499 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f717402ae60 con 0x7f718c11bad0 2024-09-19T11:22:37.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 -- 172.21.15.28:0/1517643499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c11bad0 msgr2=0x7f718c11dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 --2- 172.21.15.28:0/1517643499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c11bad0 0x7f718c11dec0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f71740097f0 tx=0x7f717402a1e0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 -- 172.21.15.28:0/1517643499 shutdown_connections 2024-09-19T11:22:37.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 --2- 172.21.15.28:0/1517643499 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c11bad0 0x7f718c11dec0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 -- 172.21.15.28:0/1517643499 >> 172.21.15.28:0/1517643499 conn(0x7f718c06f730 msgr2=0x7f718c06fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:37.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 -- 172.21.15.28:0/1517643499 shutdown_connections 2024-09-19T11:22:37.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 -- 172.21.15.28:0/1517643499 wait complete. 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 Processor -- start 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.228+0000 7f7191472640 1 -- start start 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f7191472640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c116b50 0x7f718c1a7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f7191472640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f718c11eb40 con 0x7f718c116b50 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f718affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c116b50 0x7f718c1a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f718affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c116b50 0x7f718c1a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34888/0 (socket says 172.21.15.28:34888) 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f718affd640 1 -- 172.21.15.28:0/3184802796 learned_addr learned my addr 172.21.15.28:0/3184802796 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f718affd640 1 -- 172.21.15.28:0/3184802796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c1a7ee0 con 0x7f718c116b50 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f718affd640 1 --2- 172.21.15.28:0/3184802796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c116b50 0x7f718c1a7910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c450 tx=0x7f717c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:37.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f71837fe640 1 -- 172.21.15.28:0/3184802796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f717c007410 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f71837fe640 1 -- 172.21.15.28:0/3184802796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717c0075b0 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c1a81d0 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.229+0000 7f71837fe640 1 -- 172.21.15.28:0/3184802796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f717c004950 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.230+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1ad3a0 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.230+0000 7f71837fe640 1 -- 172.21.15.28:0/3184802796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f717c004bc0 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.230+0000 7f71837fe640 1 -- 172.21.15.28:0/3184802796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f717c01b880 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.230+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c1a8510 con 0x7f718c116b50 2024-09-19T11:22:37.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.233+0000 7f71837fe640 1 -- 172.21.15.28:0/3184802796 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f717c01bba0 con 0x7f718c116b50 2024-09-19T11:22:37.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.305+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a884f1f1-c119-4e37-b10b-9038bb154d55", "id": 3} v 0) -- 0x7f718c11f120 con 0x7f718c116b50 2024-09-19T11:22:37.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.306+0000 7f71837fe640 1 -- 172.21.15.28:0/3184802796 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a884f1f1-c119-4e37-b10b-9038bb154d55", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f717c01b3f0 con 0x7f718c116b50 2024-09-19T11:22:37.308 INFO:teuthology.orchestra.run.smithi028.stdout:3 2024-09-19T11:22:37.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.307+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c116b50 msgr2=0x7f718c1a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.307+0000 7f7191472640 1 --2- 172.21.15.28:0/3184802796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c116b50 0x7f718c1a7910 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c450 tx=0x7f717c00c910 comp rx=0 tx=0).stop 2024-09-19T11:22:37.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.307+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 shutdown_connections 2024-09-19T11:22:37.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.307+0000 7f7191472640 1 --2- 172.21.15.28:0/3184802796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f718c116b50 0x7f718c1a7910 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.307+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 >> 172.21.15.28:0/3184802796 conn(0x7f718c06f730 msgr2=0x7f718c11beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:37.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.307+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 shutdown_connections 2024-09-19T11:22:37.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:22:37.307+0000 7f7191472640 1 -- 172.21.15.28:0/3184802796 wait complete. 2024-09-19T11:22:37.319 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-19T11:22:37.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-19T11:22:37.322 INFO:tasks.ceph.osd.0:Started 2024-09-19T11:22:37.322 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-19T11:22:37.322 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-19T11:22:37.324 INFO:tasks.ceph.osd.1:Started 2024-09-19T11:22:37.324 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-19T11:22:37.324 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-19T11:22:37.326 INFO:tasks.ceph.osd.2:Started 2024-09-19T11:22:37.326 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-19T11:22:37.326 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-19T11:22:37.328 INFO:tasks.ceph.osd.3:Started 2024-09-19T11:22:37.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T11:22:37.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.436+0000 7f5c6309b640 1 Processor -- start 2024-09-19T11:22:37.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.436+0000 7f5c6309b640 1 -- start start 2024-09-19T11:22:37.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c6309b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5c5c09bb40 0x7f5c5c097190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:37.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c6309b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c5c097de0 con 0x7f5c5c09c060 2024-09-19T11:22:37.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c6309b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5c099cb0 con 0x7f5c5c09bb40 2024-09-19T11:22:37.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c60e37640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f5c5c09bb40 0x7f5c5c097190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:37.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c5bfff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5c5c09c060 0x7f5c5c0976d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39572/0 (socket says 172.21.15.22:39572) 2024-09-19T11:22:37.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c5bfff640 1 -- 172.21.15.22:0/3710580041 learned_addr learned my addr 172.21.15.22:0/3710580041 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:37.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c60e37640 1 -- 172.21.15.22:0/3710580041 >> v1:172.21.15.22:6789/0 conn(0x7f5c5c09c060 legacy=0x7f5c5c0976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.437+0000 7f5c60e37640 1 -- 172.21.15.22:0/3710580041 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c097fb0 con 0x7f5c5c09bb40 2024-09-19T11:22:37.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.438+0000 7f5c60e37640 1 --2- 172.21.15.22:0/3710580041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c097190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009720 tx=0x7f5c4c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f26522f7d246e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:37.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.438+0000 7f5c5b7fe640 1 -- 172.21.15.22:0/3710580041 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c4c037070 con 0x7f5c5c09bb40 2024-09-19T11:22:37.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.438+0000 7f5c5b7fe640 1 -- 172.21.15.22:0/3710580041 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c02ad90 con 0x7f5c5c09bb40 2024-09-19T11:22:37.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.438+0000 7f5c5b7fe640 1 -- 172.21.15.22:0/3710580041 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c4c032590 con 0x7f5c5c09bb40 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 -- 172.21.15.22:0/3710580041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 msgr2=0x7f5c5c097190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 --2- 172.21.15.22:0/3710580041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c097190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009720 tx=0x7f5c4c02a110 comp rx=0 tx=0).stop 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 -- 172.21.15.22:0/3710580041 shutdown_connections 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 --2- 172.21.15.22:0/3710580041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c097190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 -- 172.21.15.22:0/3710580041 >> 172.21.15.22:0/3710580041 conn(0x7f5c5c09cb40 msgr2=0x7f5c5c09cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 -- 172.21.15.22:0/3710580041 shutdown_connections 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 -- 172.21.15.22:0/3710580041 wait complete. 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.439+0000 7f5c6309b640 1 Processor -- start 2024-09-19T11:22:37.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.440+0000 7f5c6309b640 1 -- start start 2024-09-19T11:22:37.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.440+0000 7f5c6309b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c124ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:37.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.440+0000 7f5c6309b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c135150 con 0x7f5c5c09bb40 2024-09-19T11:22:37.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.440+0000 7f5c60e37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c124ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:37.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.440+0000 7f5c60e37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c124ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55616/0 (socket says 172.21.15.22:55616) 2024-09-19T11:22:37.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.440+0000 7f5c60e37640 1 -- 172.21.15.22:0/1393276752 learned_addr learned my addr 172.21.15.22:0/1393276752 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:37.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.440+0000 7f5c60e37640 1 -- 172.21.15.22:0/1393276752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c095300 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c60e37640 1 --2- 172.21.15.22:0/1393276752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c124ce0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009840 tx=0x7f5c4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c597fa640 1 -- 172.21.15.22:0/1393276752 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c4c041020 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c597fa640 1 -- 172.21.15.22:0/1393276752 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c03d550 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c126690 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c597fa640 1 -- 172.21.15.22:0/1393276752 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c4c037040 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c125220 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c597fa640 1 -- 172.21.15.22:0/1393276752 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c4c044020 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.441+0000 7f5c597fa640 1 -- 172.21.15.22:0/1393276752 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5c4c03b3f0 con 0x7f5c5c09bb40 2024-09-19T11:22:37.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.442+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c097190 con 0x7f5c5c09bb40 2024-09-19T11:22:37.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.444+0000 7f5c597fa640 1 -- 172.21.15.22:0/1393276752 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5c4c04b030 con 0x7f5c5c09bb40 2024-09-19T11:22:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.505+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c5c050ee0 con 0x7f5c5c09bb40 2024-09-19T11:22:37.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.505+0000 7f5c597fa640 1 -- 172.21.15.22:0/1393276752 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7f5c4c02ad40 con 0x7f5c5c09bb40 2024-09-19T11:22:37.507 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:37.507 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":6,"fsid":"200d5f3b-d410-4b92-82cf-1cbe85431f50","created":"2024-09-19T11:22:36.212681+0000","modified":"2024-09-19T11:22:37.307013+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T11:22:37.307013+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f0cbb56a-4180-4b0e-948a-cb474e8486a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f8538703-1f8e-48d3-8d2e-d423b3fb007a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e738a240-46c1-4d91-98ec-1e2d64c3f437","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a884f1f1-c119-4e37-b10b-9038bb154d55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T11:22:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.506+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 msgr2=0x7f5c5c124ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.506+0000 7f5c6309b640 1 --2- 172.21.15.22:0/1393276752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c124ce0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009840 tx=0x7f5c4c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.506+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 shutdown_connections 2024-09-19T11:22:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.506+0000 7f5c6309b640 1 --2- 172.21.15.22:0/1393276752 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c5c09bb40 0x7f5c5c124ce0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.506+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 >> 172.21.15.22:0/1393276752 conn(0x7f5c5c09cb40 msgr2=0x7f5c5c0715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.506+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 shutdown_connections 2024-09-19T11:22:37.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:37.506+0000 7f5c6309b640 1 -- 172.21.15.22:0/1393276752 wait complete. 2024-09-19T11:22:37.517 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-19T11:22:37.518 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-19T11:22:37.920 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T11:22:38.016 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.014+0000 7f555d5d3640 1 Processor -- start 2024-09-19T11:22:38.016 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.014+0000 7f555d5d3640 1 -- start start 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.014+0000 7f555d5d3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f555812d5f0 0x7f555812d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.014+0000 7f555d5d3640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5558072fb0 con 0x7f555812df10 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.014+0000 7f555d5d3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f555806ce00 con 0x7f555812d5f0 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.015+0000 7f5556ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f555812d5f0 0x7f555812d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.015+0000 7f55567fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f555812df10 0x7f555812a0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:39582/0 (socket says 172.21.15.22:39582) 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.015+0000 7f55567fc640 1 -- 172.21.15.22:0/3280829770 learned_addr learned my addr 172.21.15.22:0/3280829770 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.015+0000 7f5556ffd640 1 -- 172.21.15.22:0/3280829770 >> v1:172.21.15.22:6789/0 conn(0x7f555812df10 legacy=0x7f555812a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.015+0000 7f5556ffd640 1 -- 172.21.15.22:0/3280829770 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555806f880 con 0x7f555812d5f0 2024-09-19T11:22:38.017 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.015+0000 7f5555ffb640 1 -- 172.21.15.22:0/3280829770 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727292865 0 0) 0x7f5558072fb0 con 0x7f555812df10 2024-09-19T11:22:38.018 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f5556ffd640 1 --2- 172.21.15.22:0/3280829770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f555812d9d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5548009720 tx=0x7f554802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5d9f183b7d0986 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:38.018 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f5555ffb640 1 -- 172.21.15.22:0/3280829770 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5548037070 con 0x7f555812d5f0 2024-09-19T11:22:38.018 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f5555ffb640 1 -- 172.21.15.22:0/3280829770 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554802ad90 con 0x7f555812d5f0 2024-09-19T11:22:38.018 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f5555ffb640 1 -- 172.21.15.22:0/3280829770 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5548032590 con 0x7f555812d5f0 2024-09-19T11:22:38.019 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f555d5d3640 1 -- 172.21.15.22:0/3280829770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 msgr2=0x7f555812d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:38.019 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f555d5d3640 1 --2- 172.21.15.22:0/3280829770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f555812d9d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5548009720 tx=0x7f554802a110 comp rx=0 tx=0).stop 2024-09-19T11:22:38.019 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f555d5d3640 1 -- 172.21.15.22:0/3280829770 shutdown_connections 2024-09-19T11:22:38.019 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f555d5d3640 1 --2- 172.21.15.22:0/3280829770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f555812d9d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:38.019 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f555d5d3640 1 -- 172.21.15.22:0/3280829770 >> 172.21.15.22:0/3280829770 conn(0x7f555809b950 msgr2=0x7f555809bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:38.019 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f555d5d3640 1 -- 172.21.15.22:0/3280829770 shutdown_connections 2024-09-19T11:22:38.019 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.016+0000 7f555d5d3640 1 -- 172.21.15.22:0/3280829770 wait complete. 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.017+0000 7f555d5d3640 1 Processor -- start 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.017+0000 7f555d5d3640 1 -- start start 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.018+0000 7f555d5d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f55581c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.018+0000 7f555d5d3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5558072ca0 con 0x7f555812d5f0 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.018+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f55581c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.018+0000 7f5556ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f55581c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55638/0 (socket says 172.21.15.22:55638) 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.018+0000 7f5556ffd640 1 -- 172.21.15.22:0/345939843 learned_addr learned my addr 172.21.15.22:0/345939843 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.018+0000 7f5556ffd640 1 -- 172.21.15.22:0/345939843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5558127de0 con 0x7f555812d5f0 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.018+0000 7f5556ffd640 1 --2- 172.21.15.22:0/345939843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f55581c2750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5548035040 tx=0x7f55480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:38.021 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f553f7fe640 1 -- 172.21.15.22:0/345939843 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5548041070 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f553f7fe640 1 -- 172.21.15.22:0/345939843 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554803c460 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f555808f110 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f553f7fe640 1 -- 172.21.15.22:0/345939843 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5548037040 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55581c2e70 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f553f7fe640 1 -- 172.21.15.22:0/345939843 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5548044070 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f553f7fe640 1 -- 172.21.15.22:0/345939843 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f5548046be0 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.019+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555812d9d0 con 0x7f555812d5f0 2024-09-19T11:22:38.022 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.021+0000 7f553f7fe640 1 -- 172.21.15.22:0/345939843 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f55480468e0 con 0x7f555812d5f0 2024-09-19T11:22:38.084 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.082+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f555806cb70 con 0x7f555812d5f0 2024-09-19T11:22:38.084 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.083+0000 7f553f7fe640 1 -- 172.21.15.22:0/345939843 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7f554803c780 con 0x7f555812d5f0 2024-09-19T11:22:38.084 INFO:teuthology.misc.health.smithi022.stdout: 2024-09-19T11:22:38.084 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":6,"fsid":"200d5f3b-d410-4b92-82cf-1cbe85431f50","created":"2024-09-19T11:22:36.212681+0000","modified":"2024-09-19T11:22:37.307013+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T11:22:37.307013+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f0cbb56a-4180-4b0e-948a-cb474e8486a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f8538703-1f8e-48d3-8d2e-d423b3fb007a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e738a240-46c1-4d91-98ec-1e2d64c3f437","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a884f1f1-c119-4e37-b10b-9038bb154d55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T11:22:38.085 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.084+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 msgr2=0x7f55581c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:38.085 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.084+0000 7f555d5d3640 1 --2- 172.21.15.22:0/345939843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f55581c2750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5548035040 tx=0x7f55480046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:38.085 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.084+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 shutdown_connections 2024-09-19T11:22:38.086 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.084+0000 7f555d5d3640 1 --2- 172.21.15.22:0/345939843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f555812d5f0 0x7f55581c2750 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:38.086 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.084+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 >> 172.21.15.22:0/345939843 conn(0x7f555809b950 msgr2=0x7f555809c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:38.086 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.084+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 shutdown_connections 2024-09-19T11:22:38.086 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:38.084+0000 7f555d5d3640 1 -- 172.21.15.22:0/345939843 wait complete. 2024-09-19T11:22:38.094 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-19T11:22:38.280 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-19T11:22:38.279+0000 7f4aede17640 -1 Falling back to public interface 2024-09-19T11:22:38.506 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-19T11:22:38.504+0000 7f973480f640 -1 Falling back to public interface 2024-09-19T11:22:38.531 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-19T11:22:38.530+0000 7f960801e640 -1 Falling back to public interface 2024-09-19T11:22:39.033 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-19T11:22:39.032+0000 7f3caca62640 -1 Falling back to public interface 2024-09-19T11:22:41.783 INFO:tasks.ceph.mgr.x.smithi028.stderr:2024-09-19T11:22:41.781+0000 7f5cb191b640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-19T11:22:41.851 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-19T11:22:41.850+0000 7f4aede17640 -1 osd.2 0 log_to_monitors true 2024-09-19T11:22:42.014 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-19T11:22:42.013+0000 7f973480f640 -1 osd.0 0 log_to_monitors true 2024-09-19T11:22:42.067 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-19T11:22:42.065+0000 7f960801e640 -1 osd.3 0 log_to_monitors true 2024-09-19T11:22:43.048 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-19T11:22:43.047+0000 7f3caca62640 -1 osd.1 0 log_to_monitors true 2024-09-19T11:22:44.498 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T11:22:44.536 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-19T11:22:44.535+0000 7f9730932640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T11:22:44.544 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-19T11:22:44.543+0000 7f9727f3d640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T11:22:44.592 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.590+0000 7f900873d640 1 Processor -- start 2024-09-19T11:22:44.592 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.590+0000 7f900873d640 1 -- start start 2024-09-19T11:22:44.593 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f900873d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f90000712e0 0x7f90001317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:44.593 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f900873d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f900006cc20 con 0x7f9000129370 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f900873d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f9000074250 con 0x7f90000712e0 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f9005cd8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f90000712e0 0x7f90001317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f90064d9640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f9000129370 0x7f9000129750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51758/0 (socket says 172.21.15.22:51758) 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f9005cd8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f90000712e0 0x7f90001317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54410/0 (socket says 172.21.15.22:54410) 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f90064d9640 1 -- 172.21.15.22:0/3554181815 learned_addr learned my addr 172.21.15.22:0/3554181815 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f9005cd8640 1 -- 172.21.15.22:0/3554181815 >> v1:172.21.15.22:6789/0 conn(0x7f9000129370 legacy=0x7f9000129750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f9005cd8640 1 -- 172.21.15.22:0/3554181815 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900006e640 con 0x7f90000712e0 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.591+0000 7f90054d7640 1 -- 172.21.15.22:0/3554181815 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755140842 0 0) 0x7f900006cc20 con 0x7f9000129370 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.592+0000 7f9005cd8640 1 --2- 172.21.15.22:0/3554181815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001317f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009720 tx=0x7f8ff002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f3606be2b17b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.592+0000 7f90054d7640 1 -- 172.21.15.22:0/3554181815 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ff0037070 con 0x7f90000712e0 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.592+0000 7f90054d7640 1 -- 172.21.15.22:0/3554181815 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff002ad90 con 0x7f90000712e0 2024-09-19T11:22:44.594 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.592+0000 7f90054d7640 1 -- 172.21.15.22:0/3554181815 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ff0032590 con 0x7f90000712e0 2024-09-19T11:22:44.596 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.593+0000 7f900873d640 1 -- 172.21.15.22:0/3554181815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 msgr2=0x7f90001317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.593+0000 7f900873d640 1 --2- 172.21.15.22:0/3554181815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001317f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009720 tx=0x7f8ff002a110 comp rx=0 tx=0).stop 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.593+0000 7f900873d640 1 -- 172.21.15.22:0/3554181815 shutdown_connections 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.593+0000 7f900873d640 1 --2- 172.21.15.22:0/3554181815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001317f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.593+0000 7f900873d640 1 -- 172.21.15.22:0/3554181815 >> 172.21.15.22:0/3554181815 conn(0x7f900009b950 msgr2=0x7f900009bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.593+0000 7f900873d640 1 -- 172.21.15.22:0/3554181815 shutdown_connections 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.593+0000 7f900873d640 1 -- 172.21.15.22:0/3554181815 wait complete. 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.594+0000 7f900873d640 1 Processor -- start 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.594+0000 7f900873d640 1 -- start start 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.594+0000 7f900873d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001c0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.594+0000 7f900873d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9000071a50 con 0x7f90000712e0 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.594+0000 7f90064d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:44.597 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.594+0000 7f90064d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001c0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54426/0 (socket says 172.21.15.22:54426) 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.594+0000 7f90064d9640 1 -- 172.21.15.22:0/2996897436 learned_addr learned my addr 172.21.15.22:0/2996897436 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.595+0000 7f90064d9640 1 -- 172.21.15.22:0/2996897436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900008fb10 con 0x7f90000712e0 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.595+0000 7f90064d9640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001c0340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c3e0 tx=0x7f8ffc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.595+0000 7f8ff6ffd640 1 -- 172.21.15.22:0/2996897436 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc017070 con 0x7f90000712e0 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.595+0000 7f8ff6ffd640 1 -- 172.21.15.22:0/2996897436 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ffc005c10 con 0x7f90000712e0 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.595+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9000090110 con 0x7f90000712e0 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.595+0000 7f8ff6ffd640 1 -- 172.21.15.22:0/2996897436 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc0070d0 con 0x7f90000712e0 2024-09-19T11:22:44.598 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.595+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001c0880 con 0x7f90000712e0 2024-09-19T11:22:44.599 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.596+0000 7f8ff6ffd640 1 -- 172.21.15.22:0/2996897436 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 50975+0+0 (secure 0 0 0) 0x7f8ffc01a070 con 0x7f90000712e0 2024-09-19T11:22:44.599 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.596+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90000008d0 con 0x7f90000712e0 2024-09-19T11:22:44.599 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.596+0000 7f8ff6ffd640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8fe403e630 0x7f8fe4040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:44.599 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.596+0000 7f8ff6ffd640 1 -- 172.21.15.22:0/2996897436 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1607+0+0 (secure 0 0 0) 0x7f8ffc051980 con 0x7f90000712e0 2024-09-19T11:22:44.599 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.596+0000 7f9005cd8640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8fe403e630 0x7f8fe4040af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:44.599 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.597+0000 7f9005cd8640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8fe403e630 0x7f8fe4040af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0034000 tx=0x7f8ff002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:44.602 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.601+0000 7f8ff6ffd640 1 -- 172.21.15.22:0/2996897436 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f8ffc010300 con 0x7f90000712e0 2024-09-19T11:22:44.739 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-19T11:22:44.738+0000 7f4ae9801640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T11:22:44.752 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-19T11:22:44.751+0000 7f4ae1620640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T11:22:44.793 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.792+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9000075760 con 0x7f90000712e0 2024-09-19T11:22:44.795 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.794+0000 7f8ff6ffd640 1 -- 172.21.15.22:0/2996897436 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) ==== 74+0+5122 (secure 0 0 0) 0x7f8ffc010e20 con 0x7f90000712e0 2024-09-19T11:22:44.795 INFO:teuthology.misc.health.smithi022.stdout: 2024-09-19T11:22:44.796 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":9,"fsid":"200d5f3b-d410-4b92-82cf-1cbe85431f50","created":"2024-09-19T11:22:36.212681+0000","modified":"2024-09-19T11:22:44.782271+0000","last_up_change":"2024-09-19T11:22:44.782271+0000","last_in_change":"2024-09-19T11:22:37.307013+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f0cbb56a-4180-4b0e-948a-cb474e8486a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6801","nonce":3144434255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6803","nonce":3144434255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6807","nonce":3144434255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6805","nonce":3144434255}]},"public_addr":"172.21.15.22:6801/3144434255","cluster_addr":"172.21.15.22:6803/3144434255","heartbeat_back_addr":"172.21.15.22:6807/3144434255","heartbeat_front_addr":"172.21.15.22:6805/3144434255","state":["exists","up"]},{"osd":1,"uuid":"f8538703-1f8e-48d3-8d2e-d423b3fb007a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e738a240-46c1-4d91-98ec-1e2d64c3f437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6801","nonce":2071226075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6803","nonce":2071226075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6807","nonce":2071226075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6805","nonce":2071226075}]},"public_addr":"172.21.15.28:6801/2071226075","cluster_addr":"172.21.15.28:6803/2071226075","heartbeat_back_addr":"172.21.15.28:6807/2071226075","heartbeat_front_addr":"172.21.15.28:6805/2071226075","state":["exists","up"]},{"osd":3,"uuid":"a884f1f1-c119-4e37-b10b-9038bb154d55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T11:22:44.798 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8fe403e630 msgr2=0x7f8fe4040af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:44.798 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8fe403e630 0x7f8fe4040af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0034000 tx=0x7f8ff002e000 comp rx=0 tx=0).stop 2024-09-19T11:22:44.798 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 msgr2=0x7f90001c0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:44.798 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001c0340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c3e0 tx=0x7f8ffc00c8a0 comp rx=0 tx=0).stop 2024-09-19T11:22:44.798 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 shutdown_connections 2024-09-19T11:22:44.798 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8fe403e630 0x7f8fe4040af0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:44.799 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 --2- 172.21.15.22:0/2996897436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90000712e0 0x7f90001c0340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:44.799 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 >> 172.21.15.22:0/2996897436 conn(0x7f900009b950 msgr2=0x7f9000072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:44.799 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 shutdown_connections 2024-09-19T11:22:44.799 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:44.796+0000 7f900873d640 1 -- 172.21.15.22:0/2996897436 wait complete. 2024-09-19T11:22:44.807 DEBUG:teuthology.misc:2 of 4 OSDs are up 2024-09-19T11:22:44.912 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-19T11:22:44.911+0000 7f9604272640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T11:22:44.924 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-19T11:22:44.923+0000 7f95fb87d640 -1 osd.3 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T11:22:45.569 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-19T11:22:45.566+0000 7f3ca89f1640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T11:22:45.579 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-19T11:22:45.578+0000 7f3ca0810640 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T11:22:51.212 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T11:22:51.317 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f66466dd640 1 Processor -- start 2024-09-19T11:22:51.317 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f66466dd640 1 -- start start 2024-09-19T11:22:51.318 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f66466dd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f664012fbe0 0x7f664012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:51.318 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f66466dd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6640070430 con 0x7f6640130500 2024-09-19T11:22:51.318 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f66466dd640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f664006fc10 con 0x7f664012fbe0 2024-09-19T11:22:51.318 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f663ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f664012fbe0 0x7f664012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:51.319 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f663f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6640130500 0x7f664006e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51774/0 (socket says 172.21.15.22:51774) 2024-09-19T11:22:51.319 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.316+0000 7f663f7fe640 1 -- 172.21.15.22:0/697635247 learned_addr learned my addr 172.21.15.22:0/697635247 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:51.319 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.317+0000 7f663ffff640 1 -- 172.21.15.22:0/697635247 >> v1:172.21.15.22:6789/0 conn(0x7f6640130500 legacy=0x7f664006e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:51.319 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.317+0000 7f663ffff640 1 -- 172.21.15.22:0/697635247 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6640130950 con 0x7f664012fbe0 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.317+0000 7f663ffff640 1 --2- 172.21.15.22:0/697635247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f664012ffc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6630009720 tx=0x7f663002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5308df090ab36f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f663effd640 1 -- 172.21.15.22:0/697635247 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6630037070 con 0x7f664012fbe0 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f663effd640 1 -- 172.21.15.22:0/697635247 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663002ad90 con 0x7f664012fbe0 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f663effd640 1 -- 172.21.15.22:0/697635247 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6630032590 con 0x7f664012fbe0 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f66466dd640 1 -- 172.21.15.22:0/697635247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 msgr2=0x7f664012ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f66466dd640 1 --2- 172.21.15.22:0/697635247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f664012ffc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6630009720 tx=0x7f663002a110 comp rx=0 tx=0).stop 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f66466dd640 1 -- 172.21.15.22:0/697635247 shutdown_connections 2024-09-19T11:22:51.320 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f66466dd640 1 --2- 172.21.15.22:0/697635247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f664012ffc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:51.321 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f66466dd640 1 -- 172.21.15.22:0/697635247 >> 172.21.15.22:0/697635247 conn(0x7f664009b950 msgr2=0x7f664009bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:51.321 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f66466dd640 1 -- 172.21.15.22:0/697635247 shutdown_connections 2024-09-19T11:22:51.321 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.318+0000 7f66466dd640 1 -- 172.21.15.22:0/697635247 wait complete. 2024-09-19T11:22:51.322 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.319+0000 7f66466dd640 1 Processor -- start 2024-09-19T11:22:51.322 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.319+0000 7f66466dd640 1 -- start start 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.320+0000 7f66466dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f66400959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.320+0000 7f66466dd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f664006ee70 con 0x7f664012fbe0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.320+0000 7f663ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f66400959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.320+0000 7f663ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f66400959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54434/0 (socket says 172.21.15.22:54434) 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.320+0000 7f663ffff640 1 -- 172.21.15.22:0/3439646319 learned_addr learned my addr 172.21.15.22:0/3439646319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.320+0000 7f663ffff640 1 -- 172.21.15.22:0/3439646319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66400975d0 con 0x7f664012fbe0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.320+0000 7f663ffff640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f66400959f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6630035040 tx=0x7f66300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.321+0000 7f663cff9640 1 -- 172.21.15.22:0/3439646319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6630041070 con 0x7f664012fbe0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.321+0000 7f663cff9640 1 -- 172.21.15.22:0/3439646319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663003c460 con 0x7f664012fbe0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.321+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6640124280 con 0x7f664012fbe0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.321+0000 7f663cff9640 1 -- 172.21.15.22:0/3439646319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6630037040 con 0x7f664012fbe0 2024-09-19T11:22:51.323 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.321+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6640095f30 con 0x7f664012fbe0 2024-09-19T11:22:51.324 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.322+0000 7f663cff9640 1 -- 172.21.15.22:0/3439646319 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6630044070 con 0x7f664012fbe0 2024-09-19T11:22:51.324 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.322+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6640096fb0 con 0x7f664012fbe0 2024-09-19T11:22:51.325 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.322+0000 7f663cff9640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f662c03e710 0x7f662c040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:51.325 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.322+0000 7f663cff9640 1 -- 172.21.15.22:0/3439646319 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3129+0+0 (secure 0 0 0) 0x7f6630073330 con 0x7f664012fbe0 2024-09-19T11:22:51.325 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.322+0000 7f663f7fe640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f662c03e710 0x7f662c040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:51.325 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.323+0000 7f663f7fe640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f662c03e710 0x7f662c040bd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6634007fd0 tx=0x7f6634007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:51.328 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.327+0000 7f663cff9640 1 -- 172.21.15.22:0/3439646319 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f663003c780 con 0x7f664012fbe0 2024-09-19T11:22:51.516 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.513+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f664006f3b0 con 0x7f664012fbe0 2024-09-19T11:22:51.516 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.514+0000 7f663cff9640 1 -- 172.21.15.22:0/3439646319 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7965 (secure 0 0 0) 0x7f663003cca0 con 0x7f664012fbe0 2024-09-19T11:22:51.516 INFO:teuthology.misc.health.smithi022.stdout: 2024-09-19T11:22:51.516 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":12,"fsid":"200d5f3b-d410-4b92-82cf-1cbe85431f50","created":"2024-09-19T11:22:36.212681+0000","modified":"2024-09-19T11:22:47.793479+0000","last_up_change":"2024-09-19T11:22:45.784999+0000","last_in_change":"2024-09-19T11:22:37.307013+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T11:22:44.794902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0cbb56a-4180-4b0e-948a-cb474e8486a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6801","nonce":3144434255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6803","nonce":3144434255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6807","nonce":3144434255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6805","nonce":3144434255}]},"public_addr":"172.21.15.22:6801/3144434255","cluster_addr":"172.21.15.22:6803/3144434255","heartbeat_back_addr":"172.21.15.22:6807/3144434255","heartbeat_front_addr":"172.21.15.22:6805/3144434255","state":["exists","up"]},{"osd":1,"uuid":"f8538703-1f8e-48d3-8d2e-d423b3fb007a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6817","nonce":656119574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6819","nonce":656119574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6823","nonce":656119574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6821","nonce":656119574}]},"public_addr":"172.21.15.28:6817/656119574","cluster_addr":"172.21.15.28:6819/656119574","heartbeat_back_addr":"172.21.15.28:6823/656119574","heartbeat_front_addr":"172.21.15.28:6821/656119574","state":["exists","up"]},{"osd":2,"uuid":"e738a240-46c1-4d91-98ec-1e2d64c3f437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6801","nonce":2071226075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6803","nonce":2071226075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6807","nonce":2071226075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6805","nonce":2071226075}]},"public_addr":"172.21.15.28:6801/2071226075","cluster_addr":"172.21.15.28:6803/2071226075","heartbeat_back_addr":"172.21.15.28:6807/2071226075","heartbeat_front_addr":"172.21.15.28:6805/2071226075","state":["exists","up"]},{"osd":3,"uuid":"a884f1f1-c119-4e37-b10b-9038bb154d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6809","nonce":3027134945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6811","nonce":3027134945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6815","nonce":3027134945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6813","nonce":3027134945}]},"public_addr":"172.21.15.28:6809/3027134945","cluster_addr":"172.21.15.28:6811/3027134945","heartbeat_back_addr":"172.21.15.28:6815/3027134945","heartbeat_front_addr":"172.21.15.28:6813/3027134945","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.039577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:44.013759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:42.894612+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.051197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T11:22:51.518 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f662c03e710 msgr2=0x7f662c040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:51.518 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f662c03e710 0x7f662c040bd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6634007fd0 tx=0x7f6634007aa0 comp rx=0 tx=0).stop 2024-09-19T11:22:51.518 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 msgr2=0x7f66400959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:51.518 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f66400959f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6630035040 tx=0x7f66300046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:51.519 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 shutdown_connections 2024-09-19T11:22:51.519 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f662c03e710 0x7f662c040bd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:51.519 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 --2- 172.21.15.22:0/3439646319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f664012fbe0 0x7f66400959f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:51.519 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 >> 172.21.15.22:0/3439646319 conn(0x7f664009b950 msgr2=0x7f66401b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:51.519 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 shutdown_connections 2024-09-19T11:22:51.519 INFO:teuthology.misc.health.smithi022.stderr:2024-09-19T11:22:51.516+0000 7f66466dd640 1 -- 172.21.15.22:0/3439646319 wait complete. 2024-09-19T11:22:51.527 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-19T11:22:51.528 INFO:tasks.ceph:Creating RBD pool 2024-09-19T11:22:51.528 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-19T11:22:51.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.627+0000 7f96897da640 1 Processor -- start 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.627+0000 7f96897da640 1 -- start start 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f96897da640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f9684130010 0x7f9684134860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f96897da640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9684130420 con 0x7f968412f520 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f96897da640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f968406f6b0 con 0x7f9684130010 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f9682ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f968412f520 0x7f968412f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51786/0 (socket says 172.21.15.22:51786) 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f9682ffd640 1 -- 172.21.15.22:0/561172073 learned_addr learned my addr 172.21.15.22:0/561172073 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f96827fc640 1 --2- 172.21.15.22:0/561172073 >> v2:172.21.15.22:3300/0 conn(0x7f9684130010 0x7f9684134860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f9681ffb640 1 -- 172.21.15.22:0/561172073 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201071864 0 0) 0x7f9684130420 con 0x7f968412f520 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f9681ffb640 1 -- 172.21.15.22:0/561172073 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f966c003520 con 0x7f968412f520 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f96827fc640 1 -- 172.21.15.22:0/561172073 >> v1:172.21.15.22:6789/0 conn(0x7f968412f520 legacy=0x7f968412f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:51.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.628+0000 7f96827fc640 1 -- 172.21.15.22:0/561172073 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968406fd20 con 0x7f9684130010 2024-09-19T11:22:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.629+0000 7f96827fc640 1 --2- 172.21.15.22:0/561172073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9684130010 0x7f9684134860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9678009720 tx=0x7f967802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7533b7df5bee3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.629+0000 7f9681ffb640 1 -- 172.21.15.22:0/561172073 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9678037070 con 0x7f9684130010 2024-09-19T11:22:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.629+0000 7f9681ffb640 1 -- 172.21.15.22:0/561172073 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967802ad90 con 0x7f9684130010 2024-09-19T11:22:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.629+0000 7f9681ffb640 1 -- 172.21.15.22:0/561172073 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9678032570 con 0x7f9684130010 2024-09-19T11:22:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.630+0000 7f96897da640 1 -- 172.21.15.22:0/561172073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9684130010 msgr2=0x7f9684134860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:51.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.630+0000 7f96897da640 1 --2- 172.21.15.22:0/561172073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9684130010 0x7f9684134860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9678009720 tx=0x7f967802a110 comp rx=0 tx=0).stop 2024-09-19T11:22:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.630+0000 7f96897da640 1 -- 172.21.15.22:0/561172073 shutdown_connections 2024-09-19T11:22:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.630+0000 7f96897da640 1 --2- 172.21.15.22:0/561172073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9684130010 0x7f9684134860 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.630+0000 7f96897da640 1 -- 172.21.15.22:0/561172073 >> 172.21.15.22:0/561172073 conn(0x7f96840a2080 msgr2=0x7f96840a2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.630+0000 7f96897da640 1 -- 172.21.15.22:0/561172073 shutdown_connections 2024-09-19T11:22:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.630+0000 7f96897da640 1 -- 172.21.15.22:0/561172073 wait complete. 2024-09-19T11:22:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f96897da640 1 Processor -- start 2024-09-19T11:22:51.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f96897da640 1 -- start start 2024-09-19T11:22:51.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f96897da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968412f520 0x7f9684127f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:51.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f96897da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9684134f20 con 0x7f968412f520 2024-09-19T11:22:51.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f9682ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968412f520 0x7f9684127f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:51.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f9682ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968412f520 0x7f9684127f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54462/0 (socket says 172.21.15.22:54462) 2024-09-19T11:22:51.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f9682ffd640 1 -- 172.21.15.22:0/2680044779 learned_addr learned my addr 172.21.15.22:0/2680044779 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:51.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.631+0000 7f9682ffd640 1 -- 172.21.15.22:0/2680044779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9684129560 con 0x7f968412f520 2024-09-19T11:22:51.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.632+0000 7f9682ffd640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968412f520 0x7f9684127f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f967400c480 tx=0x7f967400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.632+0000 7f966b7fe640 1 -- 172.21.15.22:0/2680044779 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9674017070 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.632+0000 7f966b7fe640 1 -- 172.21.15.22:0/2680044779 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9674005cd0 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.632+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968412b240 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.632+0000 7f966b7fe640 1 -- 172.21.15.22:0/2680044779 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9674007200 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.632+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96841284b0 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.633+0000 7f966b7fe640 1 -- 172.21.15.22:0/2680044779 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f967401a070 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.633+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968412f900 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.633+0000 7f966b7fe640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f967003e710 0x7f9670040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.633+0000 7f966b7fe640 1 -- 172.21.15.22:0/2680044779 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3129+0+0 (secure 0 0 0) 0x7f9674051e50 con 0x7f968412f520 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.633+0000 7f96827fc640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f967003e710 0x7f9670040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:51.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.634+0000 7f96827fc640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f967003e710 0x7f9670040bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9678009720 tx=0x7f967802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:51.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.638+0000 7f966b7fe640 1 -- 172.21.15.22:0/2680044779 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f967401b3b0 con 0x7f968412f520 2024-09-19T11:22:51.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:51.826+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f968412a160 con 0x7f968412f520 2024-09-19T11:22:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.804+0000 7f966b7fe640 1 -- 172.21.15.22:0/2680044779 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) ==== 109+0+0 (secure 0 0 0) 0x7f967401b730 con 0x7f968412f520 2024-09-19T11:22:52.805 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'rbd' created 2024-09-19T11:22:52.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f967003e710 msgr2=0x7f9670040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:52.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f967003e710 0x7f9670040bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9678009720 tx=0x7f967802e000 comp rx=0 tx=0).stop 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968412f520 msgr2=0x7f9684127f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968412f520 0x7f9684127f70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f967400c480 tx=0x7f967400c940 comp rx=0 tx=0).stop 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 shutdown_connections 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f967003e710 0x7f9670040bd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 --2- 172.21.15.22:0/2680044779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f968412f520 0x7f9684127f70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 >> 172.21.15.22:0/2680044779 conn(0x7f96840a2080 msgr2=0x7f968412e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 shutdown_connections 2024-09-19T11:22:52.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.807+0000 7f96897da640 1 -- 172.21.15.22:0/2680044779 wait complete. 2024-09-19T11:22:52.821 DEBUG:teuthology.orchestra.run.smithi022:> rbd --cluster ceph pool init rbd 2024-09-19T11:22:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:22:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:22:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.887+0000 7fe75b63d140 1 Processor -- start 2024-09-19T11:22:52.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.887+0000 7fe75b63d140 1 -- start start 2024-09-19T11:22:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe75b63d140 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561a8afdea00 0x561a8afdede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe75b63d140 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x561a8add0d90 con 0x561a8afdf320 2024-09-19T11:22:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe75b63d140 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x561a8ae20a80 con 0x561a8afdea00 2024-09-19T11:22:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe75965c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561a8afdea00 0x561a8afdede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:52.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe758e5b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x561a8afdf320 0x561a8af33be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:51788/0 (socket says 172.21.15.22:51788) 2024-09-19T11:22:52.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe75965c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561a8afdea00 0x561a8afdede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54468/0 (socket says 172.21.15.22:54468) 2024-09-19T11:22:52.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe758e5b640 1 -- 172.21.15.22:0/2395929929 learned_addr learned my addr 172.21.15.22:0/2395929929 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:52.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe75965c640 1 -- 172.21.15.22:0/2395929929 >> v1:172.21.15.22:6789/0 conn(0x561a8afdf320 legacy=0x561a8af33be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:52.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe75965c640 1 -- 172.21.15.22:0/2395929929 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8ade8410 con 0x561a8afdea00 2024-09-19T11:22:52.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.888+0000 7fe74bfff640 1 -- 172.21.15.22:0/2395929929 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1164955543 0 0) 0x561a8add0d90 con 0x561a8afdf320 2024-09-19T11:22:52.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.890+0000 7fe75965c640 1 --2- 172.21.15.22:0/2395929929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afdede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00a470 tx=0x7fe74c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3fa667288e7397 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:52.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.890+0000 7fe74bfff640 1 -- 172.21.15.22:0/2395929929 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c053570 con 0x561a8afdea00 2024-09-19T11:22:52.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.890+0000 7fe74bfff640 1 -- 172.21.15.22:0/2395929929 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c058070 con 0x561a8afdea00 2024-09-19T11:22:52.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.890+0000 7fe74bfff640 1 -- 172.21.15.22:0/2395929929 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c053880 con 0x561a8afdea00 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.891+0000 7fe75b63d140 1 -- 172.21.15.22:0/2395929929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 msgr2=0x561a8afdede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.891+0000 7fe75b63d140 1 --2- 172.21.15.22:0/2395929929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afdede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00a470 tx=0x7fe74c045a20 comp rx=0 tx=0).stop 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.891+0000 7fe75b63d140 1 -- 172.21.15.22:0/2395929929 shutdown_connections 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.891+0000 7fe75b63d140 1 --2- 172.21.15.22:0/2395929929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afdede0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.891+0000 7fe75b63d140 1 -- 172.21.15.22:0/2395929929 >> 172.21.15.22:0/2395929929 conn(0x561a8af503b0 msgr2=0x561a8af4a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.891+0000 7fe75b63d140 1 -- 172.21.15.22:0/2395929929 shutdown_connections 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.891+0000 7fe75b63d140 1 -- 172.21.15.22:0/2395929929 wait complete. 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.892+0000 7fe75b63d140 1 Processor -- start 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.892+0000 7fe75b63d140 1 -- start start 2024-09-19T11:22:52.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.892+0000 7fe75b63d140 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afe9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.892+0000 7fe75b63d140 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a8ae24890 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.892+0000 7fe75965c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afe9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.892+0000 7fe75965c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afe9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54484/0 (socket says 172.21.15.22:54484) 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.892+0000 7fe75965c640 1 -- 172.21.15.22:0/1389657161 learned_addr learned my addr 172.21.15.22:0/1389657161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.893+0000 7fe75965c640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8ad9f720 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.893+0000 7fe75965c640 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afe9b60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe74c055040 tx=0x7fe74c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.893+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c053b30 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.893+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c053cd0 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.893+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c0673f0 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.893+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561a8afca100 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.893+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a8afcaeb0 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.894+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe74c06e060 con 0x561a8afdea00 2024-09-19T11:22:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.895+0000 7fe7497fa640 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe73c03d000 0x7fe73c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:52.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.895+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3471+0+0 (secure 0 0 0) 0x7fe74c0959c0 con 0x561a8afdea00 2024-09-19T11:22:52.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.895+0000 7fe758e5b640 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe73c03d000 0x7fe73c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:52.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.895+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) -- 0x561a8afea0a0 con 0x561a8afdea00 2024-09-19T11:22:52.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:52.895+0000 7fe758e5b640 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe73c03d000 0x7fe73c03f4c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe7500012f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.804+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v14) ==== 141+0+0 (secure 0 0 0) 0x7fe74c062070 con 0x561a8afdea00 2024-09-19T11:22:53.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.804+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe74c058070 con 0x561a8afdea00 2024-09-19T11:22:53.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.807+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7fe74c05ce30 con 0x561a8afdea00 2024-09-19T11:22:53.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.807+0000 7fe74b7fe640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=14}) -- 0x7fe734002ae0 con 0x561a8afdea00 2024-09-19T11:22:53.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.809+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 8 ==== osd_map(14..14 src has 1..14) ==== 650+0+0 (secure 0 0 0) 0x7fe74c006b80 con 0x561a8afdea00 2024-09-19T11:22:53.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.810+0000 7fe75b63d140 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a8afdf320 0x561a8afed780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:53.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.810+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14) -- 0x561a8aff2140 con 0x561a8afdf320 2024-09-19T11:22:53.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.810+0000 7fe75a8be640 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a8afdf320 0x561a8afed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:53.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.812+0000 7fe75a8be640 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a8afdf320 0x561a8afed780 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:53.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.813+0000 7fe75a8be640 1 -- 172.21.15.22:0/1389657161 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(1 rbd_trash [create] v14'1 uv1 ondisk = 0) ==== 153+0+0 (crc 0 0 0) 0x7fe754002040 con 0x561a8afdf320 2024-09-19T11:22:53.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.816+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) -- 0x561a8b0445d0 con 0x561a8afdf320 2024-09-19T11:22:53.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.817+0000 7fe75a8be640 1 -- 172.21.15.22:0/1389657161 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7fe754002040 con 0x561a8afdf320 2024-09-19T11:22:53.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.817+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e14) -- 0x561a8b0436e0 con 0x561a8afdf320 2024-09-19T11:22:53.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.818+0000 7fe75a8be640 1 -- 172.21.15.22:0/1389657161 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7fe754002040 con 0x561a8afdf320 2024-09-19T11:22:53.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:53.818+0000 7fe74affd640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) -- 0x7fe738001f00 con 0x561a8afdea00 2024-09-19T11:22:54.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:54.811+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v15) ==== 55+0+0 (secure 0 0 0) 0x7fe74c018c50 con 0x561a8afdea00 2024-09-19T11:22:54.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:54.811+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=15}) -- 0x7fe73c042bb0 con 0x561a8afdea00 2024-09-19T11:22:54.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:54.811+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 10 ==== osd_map(15..15 src has 1..15) ==== 626+0+0 (secure 0 0 0) 0x7fe74c087020 con 0x561a8afdea00 2024-09-19T11:22:54.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:54.811+0000 7fe74b7fe640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15) -- 0x7fe7340057e0 con 0x561a8afdf320 2024-09-19T11:22:54.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:54.815+0000 7fe75a8be640 1 -- 172.21.15.22:0/1389657161 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v15'1 uv1 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fe754002040 con 0x561a8afdf320 2024-09-19T11:22:54.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:54.815+0000 7fe74affd640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v15) -- 0x7fe738003350 con 0x561a8afdea00 2024-09-19T11:22:55.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.812+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 11 ==== pool_op_reply(tid 6 (0) Success v16) ==== 43+0+0 (secure 0 0 0) 0x7fe74c01aed0 con 0x561a8afdea00 2024-09-19T11:22:55.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.812+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=16}) -- 0x7fe73c042e30 con 0x561a8afdea00 2024-09-19T11:22:55.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.812+0000 7fe7497fa640 1 -- 172.21.15.22:0/1389657161 <== mon.0 v2:172.21.15.22:3300/0 12 ==== osd_map(16..16 src has 1..16) ==== 654+0+0 (secure 0 0 0) 0x7fe74c06f850 con 0x561a8afdea00 2024-09-19T11:22:55.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.812+0000 7fe74b7fe640 1 -- 172.21.15.22:0/1389657161 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) -- 0x7fe734005ba0 con 0x561a8afdf320 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.817+0000 7fe75a8be640 1 -- 172.21.15.22:0/1389657161 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(7 rbd_info [write 0~19] v16'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe754002040 con 0x561a8afdf320 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a8afdf320 msgr2=0x561a8afed780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a8afdf320 0x561a8afed780 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe73c03d000 msgr2=0x7fe73c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe73c03d000 0x7fe73c03f4c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe750000c00 tx=0x7fe7500012f0 comp rx=0 tx=0).stop 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 msgr2=0x561a8afe9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afe9b60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe74c055040 tx=0x7fe74c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 shutdown_connections 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a8afdf320 0x561a8afed780 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:55.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe73c03d000 0x7fe73c03f4c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:55.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 --2- 172.21.15.22:0/1389657161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a8afdea00 0x561a8afe9b60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:55.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 >> 172.21.15.22:0/1389657161 conn(0x561a8af503b0 msgr2=0x561a8afd79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:55.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 shutdown_connections 2024-09-19T11:22:55.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.818+0000 7fe75b63d140 1 -- 172.21.15.22:0/1389657161 wait complete. 2024-09-19T11:22:55.825 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-19T11:22:55.825 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-19T11:22:55.825 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-19T11:22:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f8401517640 1 Processor -- start 2024-09-19T11:22:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f8401517640 1 -- start start 2024-09-19T11:22:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f8401517640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f83fc12c0b0 0x7f83fc12c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f8401517640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83fc09a290 con 0x7f83fc12c9d0 2024-09-19T11:22:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f8401517640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f83fc071860 con 0x7f83fc12c0b0 2024-09-19T11:22:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f83faffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f83fc12c0b0 0x7f83fc12c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:55.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f83f27fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f83fc12c9d0 0x7f83fc097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49102/0 (socket says 172.21.15.22:49102) 2024-09-19T11:22:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.937+0000 7f83f27fc640 1 -- 172.21.15.22:0/3568413431 learned_addr learned my addr 172.21.15.22:0/3568413431 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.938+0000 7f83faffd640 1 -- 172.21.15.22:0/3568413431 >> v1:172.21.15.22:6789/0 conn(0x7f83fc12c9d0 legacy=0x7f83fc097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:55.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.938+0000 7f83faffd640 1 -- 172.21.15.22:0/3568413431 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc099e20 con 0x7f83fc12c0b0 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.938+0000 7f83faffd640 1 --2- 172.21.15.22:0/3568413431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc12c490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00b9f0 tx=0x7f83ec02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b619cad3d662b85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f83fa7fc640 1 -- 172.21.15.22:0/3568413431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec03a070 con 0x7f83fc12c0b0 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f83fa7fc640 1 -- 172.21.15.22:0/3568413431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec031070 con 0x7f83fc12c0b0 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f83fa7fc640 1 -- 172.21.15.22:0/3568413431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec032620 con 0x7f83fc12c0b0 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f8401517640 1 -- 172.21.15.22:0/3568413431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 msgr2=0x7f83fc12c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f8401517640 1 --2- 172.21.15.22:0/3568413431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc12c490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00b9f0 tx=0x7f83ec02c810 comp rx=0 tx=0).stop 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f8401517640 1 -- 172.21.15.22:0/3568413431 shutdown_connections 2024-09-19T11:22:55.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f8401517640 1 --2- 172.21.15.22:0/3568413431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc12c490 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:55.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f8401517640 1 -- 172.21.15.22:0/3568413431 >> 172.21.15.22:0/3568413431 conn(0x7f83fc09abc0 msgr2=0x7f83fc098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:55.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f8401517640 1 -- 172.21.15.22:0/3568413431 shutdown_connections 2024-09-19T11:22:55.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.939+0000 7f8401517640 1 -- 172.21.15.22:0/3568413431 wait complete. 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.940+0000 7f8401517640 1 Processor -- start 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.940+0000 7f8401517640 1 -- start start 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f8401517640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc127480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f8401517640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc12ce00 con 0x7f83fc12c0b0 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83faffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc127480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83faffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc127480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59842/0 (socket says 172.21.15.22:59842) 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83faffd640 1 -- 172.21.15.22:0/2504296260 learned_addr learned my addr 172.21.15.22:0/2504296260 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83faffd640 1 -- 172.21.15.22:0/2504296260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc128b70 con 0x7f83fc12c0b0 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83faffd640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc127480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00b9f0 tx=0x7f83ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:55.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83f3fff640 1 -- 172.21.15.22:0/2504296260 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec03a040 con 0x7f83fc12c0b0 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83f3fff640 1 -- 172.21.15.22:0/2504296260 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec045070 con 0x7f83fc12c0b0 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f83f3fff640 1 -- 172.21.15.22:0/2504296260 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83ec035820 con 0x7f83fc12c0b0 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.941+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc0904d0 con 0x7f83fc12c0b0 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.942+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc127ba0 con 0x7f83fc12c0b0 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.942+0000 7f83f3fff640 1 -- 172.21.15.22:0/2504296260 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f83ec033070 con 0x7f83fc12c0b0 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.942+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc129ab0 con 0x7f83fc12c0b0 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.943+0000 7f83f3fff640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f83dc03e710 0x7f83dc040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:55.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.943+0000 7f83f3fff640 1 -- 172.21.15.22:0/2504296260 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f83ec0738c0 con 0x7f83fc12c0b0 2024-09-19T11:22:55.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.943+0000 7f83f27fc640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f83dc03e710 0x7f83dc040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:55.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.943+0000 7f83f27fc640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f83dc03e710 0x7f83dc040bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f83e4007fd0 tx=0x7f83e4007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:55.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:55.948+0000 7f83f3fff640 1 -- 172.21.15.22:0/2504296260 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f83ec049580 con 0x7f83fc12c0b0 2024-09-19T11:22:56.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.137+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f83fc12c490 con 0x7f83fc12c0b0 2024-09-19T11:22:56.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.137+0000 7f83f3fff640 1 -- 172.21.15.22:0/2504296260 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v5) ==== 86+0+159 (secure 0 0 0) 0x7f83ec04a980 con 0x7f83fc12c0b0 2024-09-19T11:22:56.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.139+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f83dc03e710 msgr2=0x7f83dc040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.139+0000 7f8401517640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f83dc03e710 0x7f83dc040bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f83e4007fd0 tx=0x7f83e4007aa0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 msgr2=0x7f83fc127480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc127480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00b9f0 tx=0x7f83ec004730 comp rx=0 tx=0).stop 2024-09-19T11:22:56.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 shutdown_connections 2024-09-19T11:22:56.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f83dc03e710 0x7f83dc040bd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 --2- 172.21.15.22:0/2504296260 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83fc12c0b0 0x7f83fc127480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 >> 172.21.15.22:0/2504296260 conn(0x7f83fc09abc0 msgr2=0x7f83fc097550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:56.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 shutdown_connections 2024-09-19T11:22:56.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.140+0000 7f8401517640 1 -- 172.21.15.22:0/2504296260 wait complete. 2024-09-19T11:22:56.152 INFO:teuthology.orchestra.run.smithi022.stdout:[{"version":5,"timestamp":"2024-09-19T11:22:45.567990+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"6834.675241"}]}] 2024-09-19T11:22:56.152 INFO:tasks.ceph_manager:config epoch is 5 2024-09-19T11:22:56.152 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T11:22:56.152 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-19T11:22:56.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-19T11:22:56.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.252+0000 7f281af90640 1 Processor -- start 2024-09-19T11:22:56.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.252+0000 7f281af90640 1 -- start start 2024-09-19T11:22:56.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f281af90640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f281412cba0 0x7f28141315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f281af90640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f281409a290 con 0x7f281412c0b0 2024-09-19T11:22:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f281af90640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2814071860 con 0x7f281412cba0 2024-09-19T11:22:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f2813fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f281412cba0 0x7f28141315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f2818d2c640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f281412c0b0 0x7f281412c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49110/0 (socket says 172.21.15.22:49110) 2024-09-19T11:22:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f2818d2c640 1 -- 172.21.15.22:0/981392162 learned_addr learned my addr 172.21.15.22:0/981392162 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f2813fff640 1 -- 172.21.15.22:0/981392162 >> v1:172.21.15.22:6789/0 conn(0x7f281412c0b0 legacy=0x7f281412c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.253+0000 7f2813fff640 1 -- 172.21.15.22:0/981392162 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2814099e20 con 0x7f281412cba0 2024-09-19T11:22:56.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f2813fff640 1 --2- 172.21.15.22:0/981392162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412cba0 0x7f28141315e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2808009750 tx=0x7f280802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41000acfcfc2305 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f28137fe640 1 -- 172.21.15.22:0/981392162 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2808037070 con 0x7f281412cba0 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f28137fe640 1 -- 172.21.15.22:0/981392162 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280802ad80 con 0x7f281412cba0 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f28137fe640 1 -- 172.21.15.22:0/981392162 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2808032590 con 0x7f281412cba0 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f281af90640 1 -- 172.21.15.22:0/981392162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412cba0 msgr2=0x7f28141315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f281af90640 1 --2- 172.21.15.22:0/981392162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412cba0 0x7f28141315e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2808009750 tx=0x7f280802a140 comp rx=0 tx=0).stop 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f281af90640 1 -- 172.21.15.22:0/981392162 shutdown_connections 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f281af90640 1 --2- 172.21.15.22:0/981392162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412cba0 0x7f28141315e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f281af90640 1 -- 172.21.15.22:0/981392162 >> 172.21.15.22:0/981392162 conn(0x7f281409abc0 msgr2=0x7f2814098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f281af90640 1 -- 172.21.15.22:0/981392162 shutdown_connections 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.254+0000 7f281af90640 1 -- 172.21.15.22:0/981392162 wait complete. 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.255+0000 7f281af90640 1 Processor -- start 2024-09-19T11:22:56.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.255+0000 7f281af90640 1 -- start start 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f281af90640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412c0b0 0x7f28141262b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f281af90640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2814070c40 con 0x7f281412c0b0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f2818d2c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412c0b0 0x7f28141262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f2818d2c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412c0b0 0x7f28141262b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59856/0 (socket says 172.21.15.22:59856) 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f2818d2c640 1 -- 172.21.15.22:0/3223102975 learned_addr learned my addr 172.21.15.22:0/3223102975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f2818d2c640 1 -- 172.21.15.22:0/3223102975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28141290b0 con 0x7f281412c0b0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f2818d2c640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412c0b0 0x7f28141262b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f280400c450 tx=0x7f280400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.256+0000 7f28117fa640 1 -- 172.21.15.22:0/3223102975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2804017070 con 0x7f281412c0b0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.257+0000 7f28117fa640 1 -- 172.21.15.22:0/3223102975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2804005c10 con 0x7f281412c0b0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.257+0000 7f28117fa640 1 -- 172.21.15.22:0/3223102975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28040070d0 con 0x7f281412c0b0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.257+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2814128940 con 0x7f281412c0b0 2024-09-19T11:22:56.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.257+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141269d0 con 0x7f281412c0b0 2024-09-19T11:22:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.258+0000 7f28117fa640 1 -- 172.21.15.22:0/3223102975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f280401a070 con 0x7f281412c0b0 2024-09-19T11:22:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.258+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281412c490 con 0x7f281412c0b0 2024-09-19T11:22:56.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.258+0000 7f28117fa640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f280003e710 0x7f2800040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.258+0000 7f28117fa640 1 -- 172.21.15.22:0/3223102975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f2804052080 con 0x7f281412c0b0 2024-09-19T11:22:56.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.258+0000 7f2813fff640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f280003e710 0x7f2800040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.259+0000 7f2813fff640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f280003e710 0x7f2800040bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f280802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.263+0000 7f28117fa640 1 -- 172.21.15.22:0/3223102975 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f280401ad80 con 0x7f281412c0b0 2024-09-19T11:22:56.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.496+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2814129450 con 0x7f281412c0b0 2024-09-19T11:22:56.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.498+0000 7f28117fa640 1 -- 172.21.15.22:0/3223102975 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+98087 (secure 0 0 0) 0x7f2804038f40 con 0x7f281412c0b0 2024-09-19T11:22:56.501 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f280003e710 msgr2=0x7f2800040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f280003e710 0x7f2800040bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f280802e000 comp rx=0 tx=0).stop 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412c0b0 msgr2=0x7f28141262b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412c0b0 0x7f28141262b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f280400c450 tx=0x7f280400c910 comp rx=0 tx=0).stop 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 shutdown_connections 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f280003e710 0x7f2800040bd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 --2- 172.21.15.22:0/3223102975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f281412c0b0 0x7f28141262b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 >> 172.21.15.22:0/3223102975 conn(0x7f281409abc0 msgr2=0x7f28141b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 shutdown_connections 2024-09-19T11:22:56.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.500+0000 7f281af90640 1 -- 172.21.15.22:0/3223102975 wait complete. 2024-09-19T11:22:56.513 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":2061368145},{"type":"v1","addr":"172.21.15.28:6825","nonce":2061368145}]},"active_addr":"172.21.15.28:6825/2061368145","active_change":"2024-09-19T11:22:40.774599+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":3222695723}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":3777568511}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":596106999}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":1592635848}]}]} 2024-09-19T11:22:56.516 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-19T11:22:56.516 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T11:22:56.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T11:22:56.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f59649f8640 1 Processor -- start 2024-09-19T11:22:56.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f59649f8640 1 -- start start 2024-09-19T11:22:56.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f59649f8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f596006dfb0 0x7f596006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f59649f8640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5960071660 con 0x7f596006e8d0 2024-09-19T11:22:56.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f59649f8640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5960072550 con 0x7f596006dfb0 2024-09-19T11:22:56.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f595e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f596006dfb0 0x7f596006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f5957fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f596006e8d0 0x7f5960097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49114/0 (socket says 172.21.15.22:49114) 2024-09-19T11:22:56.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f5957fff640 1 -- 172.21.15.22:0/216166960 learned_addr learned my addr 172.21.15.22:0/216166960 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:56.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.616+0000 7f595e59c640 1 --2- 172.21.15.22:0/216166960 >> v2:172.21.15.22:3300/0 conn(0x7f596006dfb0 0x7f596006e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59868/0 (socket says 172.21.15.22:59868) 2024-09-19T11:22:56.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.617+0000 7f595e59c640 1 -- 172.21.15.22:0/216166960 >> v1:172.21.15.22:6789/0 conn(0x7f596006e8d0 legacy=0x7f5960097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.617+0000 7f595e59c640 1 -- 172.21.15.22:0/216166960 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5960099e20 con 0x7f596006dfb0 2024-09-19T11:22:56.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.617+0000 7f595e59c640 1 --2- 172.21.15.22:0/216166960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f596006e390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5950009720 tx=0x7f595002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66b942ac7ed9dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.617+0000 7f595dd9b640 1 -- 172.21.15.22:0/216166960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5950037070 con 0x7f596006dfb0 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.617+0000 7f595dd9b640 1 -- 172.21.15.22:0/216166960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595002ae20 con 0x7f596006dfb0 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.617+0000 7f595dd9b640 1 -- 172.21.15.22:0/216166960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5950032590 con 0x7f596006dfb0 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.618+0000 7f59649f8640 1 -- 172.21.15.22:0/216166960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 msgr2=0x7f596006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.618+0000 7f59649f8640 1 --2- 172.21.15.22:0/216166960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f596006e390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5950009720 tx=0x7f595002a1a0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.618+0000 7f59649f8640 1 -- 172.21.15.22:0/216166960 shutdown_connections 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.618+0000 7f59649f8640 1 --2- 172.21.15.22:0/216166960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f596006e390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.618+0000 7f59649f8640 1 -- 172.21.15.22:0/216166960 >> 172.21.15.22:0/216166960 conn(0x7f596009abc0 msgr2=0x7f5960098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.618+0000 7f59649f8640 1 -- 172.21.15.22:0/216166960 shutdown_connections 2024-09-19T11:22:56.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.618+0000 7f59649f8640 1 -- 172.21.15.22:0/216166960 wait complete. 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.619+0000 7f59649f8640 1 Processor -- start 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.619+0000 7f59649f8640 1 -- start start 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.619+0000 7f59649f8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f59601c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.619+0000 7f59649f8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f596006ece0 con 0x7f596006dfb0 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.619+0000 7f595e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f59601c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f595e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f59601c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59880/0 (socket says 172.21.15.22:59880) 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f595e59c640 1 -- 172.21.15.22:0/2800494248 learned_addr learned my addr 172.21.15.22:0/2800494248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f595e59c640 1 -- 172.21.15.22:0/2800494248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5960074110 con 0x7f596006dfb0 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f595e59c640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f59601c4560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f595002a1e0 tx=0x7f59500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f5956ffd640 1 -- 172.21.15.22:0/2800494248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5950042070 con 0x7f596006dfb0 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f5956ffd640 1 -- 172.21.15.22:0/2800494248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595003e590 con 0x7f596006dfb0 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f5956ffd640 1 -- 172.21.15.22:0/2800494248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5950037040 con 0x7f596006dfb0 2024-09-19T11:22:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5960070da0 con 0x7f596006dfb0 2024-09-19T11:22:56.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.620+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59600701f0 con 0x7f596006dfb0 2024-09-19T11:22:56.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.621+0000 7f5956ffd640 1 -- 172.21.15.22:0/2800494248 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5950045070 con 0x7f596006dfb0 2024-09-19T11:22:56.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.621+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596006f0c0 con 0x7f596006dfb0 2024-09-19T11:22:56.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.621+0000 7f5956ffd640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f594003e710 0x7f5940040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.621+0000 7f5956ffd640 1 -- 172.21.15.22:0/2800494248 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f5950073860 con 0x7f596006dfb0 2024-09-19T11:22:56.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.622+0000 7f5957fff640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f594003e710 0x7f5940040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.622+0000 7f5957fff640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f594003e710 0x7f5940040bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5948007fd0 tx=0x7f5948007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.626+0000 7f5956ffd640 1 -- 172.21.15.22:0/2800494248 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f595003c430 con 0x7f596006dfb0 2024-09-19T11:22:56.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.813+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f596006e390 con 0x7f596006dfb0 2024-09-19T11:22:56.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.816+0000 7f5956ffd640 1 -- 172.21.15.22:0/2800494248 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+9865 (secure 0 0 0) 0x7f595002ae20 con 0x7f596006dfb0 2024-09-19T11:22:56.817 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:56.817 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"fsid":"200d5f3b-d410-4b92-82cf-1cbe85431f50","created":"2024-09-19T11:22:36.212681+0000","modified":"2024-09-19T11:22:55.810737+0000","last_up_change":"2024-09-19T11:22:45.784999+0000","last_in_change":"2024-09-19T11:22:37.307013+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T11:22:44.794902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T11:22:51.830554+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0cbb56a-4180-4b0e-948a-cb474e8486a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6801","nonce":3144434255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6803","nonce":3144434255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6807","nonce":3144434255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6805","nonce":3144434255}]},"public_addr":"172.21.15.22:6801/3144434255","cluster_addr":"172.21.15.22:6803/3144434255","heartbeat_back_addr":"172.21.15.22:6807/3144434255","heartbeat_front_addr":"172.21.15.22:6805/3144434255","state":["exists","up"]},{"osd":1,"uuid":"f8538703-1f8e-48d3-8d2e-d423b3fb007a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6817","nonce":656119574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6819","nonce":656119574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6823","nonce":656119574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6821","nonce":656119574}]},"public_addr":"172.21.15.28:6817/656119574","cluster_addr":"172.21.15.28:6819/656119574","heartbeat_back_addr":"172.21.15.28:6823/656119574","heartbeat_front_addr":"172.21.15.28:6821/656119574","state":["exists","up"]},{"osd":2,"uuid":"e738a240-46c1-4d91-98ec-1e2d64c3f437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6801","nonce":2071226075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6803","nonce":2071226075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6807","nonce":2071226075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6805","nonce":2071226075}]},"public_addr":"172.21.15.28:6801/2071226075","cluster_addr":"172.21.15.28:6803/2071226075","heartbeat_back_addr":"172.21.15.28:6807/2071226075","heartbeat_front_addr":"172.21.15.28:6805/2071226075","state":["exists","up"]},{"osd":3,"uuid":"a884f1f1-c119-4e37-b10b-9038bb154d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6809","nonce":3027134945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6811","nonce":3027134945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6815","nonce":3027134945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6813","nonce":3027134945}]},"public_addr":"172.21.15.28:6809/3027134945","cluster_addr":"172.21.15.28:6811/3027134945","heartbeat_back_addr":"172.21.15.28:6815/3027134945","heartbeat_front_addr":"172.21.15.28:6813/3027134945","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.039577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:44.013759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:42.894612+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.051197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T11:22:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f594003e710 msgr2=0x7f5940040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f594003e710 0x7f5940040bd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5948007fd0 tx=0x7f5948007aa0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 msgr2=0x7f59601c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f59601c4560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f595002a1e0 tx=0x7f59500046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 shutdown_connections 2024-09-19T11:22:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f594003e710 0x7f5940040bd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 --2- 172.21.15.22:0/2800494248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f596006dfb0 0x7f59601c4560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 >> 172.21.15.22:0/2800494248 conn(0x7f596009abc0 msgr2=0x7f596009c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:56.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 shutdown_connections 2024-09-19T11:22:56.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.818+0000 7f59649f8640 1 -- 172.21.15.22:0/2800494248 wait complete. 2024-09-19T11:22:56.831 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T11:22:56.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.932+0000 7f487e816640 1 Processor -- start 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.932+0000 7f487e816640 1 -- start start 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f487e816640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f487806e530 0x7f487806e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f487e816640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48780708a0 con 0x7f487806ee50 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f487e816640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f48780704d0 con 0x7f487806e530 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f4877fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f487806e530 0x7f487806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f486ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f487806ee50 0x7f487812f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49120/0 (socket says 172.21.15.22:49120) 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f486ffff640 1 -- 172.21.15.22:0/1540998095 learned_addr learned my addr 172.21.15.22:0/1540998095 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f4877fff640 1 -- 172.21.15.22:0/1540998095 >> v1:172.21.15.22:6789/0 conn(0x7f487806ee50 legacy=0x7f487812f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.933+0000 7f4877fff640 1 -- 172.21.15.22:0/1540998095 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487806f260 con 0x7f487806e530 2024-09-19T11:22:56.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.934+0000 7f4877fff640 1 --2- 172.21.15.22:0/1540998095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f487806e910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4868009720 tx=0x7f486802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc963d97e5301e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.934+0000 7f48777fe640 1 -- 172.21.15.22:0/1540998095 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4868037070 con 0x7f487806e530 2024-09-19T11:22:56.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.934+0000 7f48777fe640 1 -- 172.21.15.22:0/1540998095 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486802ad90 con 0x7f487806e530 2024-09-19T11:22:56.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.934+0000 7f48777fe640 1 -- 172.21.15.22:0/1540998095 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4868032590 con 0x7f487806e530 2024-09-19T11:22:56.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.935+0000 7f487e816640 1 -- 172.21.15.22:0/1540998095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 msgr2=0x7f487806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:56.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.935+0000 7f487e816640 1 --2- 172.21.15.22:0/1540998095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f487806e910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4868009720 tx=0x7f486802a110 comp rx=0 tx=0).stop 2024-09-19T11:22:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.935+0000 7f487e816640 1 -- 172.21.15.22:0/1540998095 shutdown_connections 2024-09-19T11:22:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.935+0000 7f487e816640 1 --2- 172.21.15.22:0/1540998095 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f487806e910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.935+0000 7f487e816640 1 -- 172.21.15.22:0/1540998095 >> 172.21.15.22:0/1540998095 conn(0x7f48780a22c0 msgr2=0x7f48780a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.935+0000 7f487e816640 1 -- 172.21.15.22:0/1540998095 shutdown_connections 2024-09-19T11:22:56.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.935+0000 7f487e816640 1 -- 172.21.15.22:0/1540998095 wait complete. 2024-09-19T11:22:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f487e816640 1 Processor -- start 2024-09-19T11:22:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f487e816640 1 -- start start 2024-09-19T11:22:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f487e816640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f48780903f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f487e816640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4878130010 con 0x7f487806e530 2024-09-19T11:22:56.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f4877fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f48780903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f4877fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f48780903f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59892/0 (socket says 172.21.15.22:59892) 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f4877fff640 1 -- 172.21.15.22:0/3308790175 learned_addr learned my addr 172.21.15.22:0/3308790175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f4877fff640 1 -- 172.21.15.22:0/3308790175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4878129f90 con 0x7f487806e530 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.936+0000 7f4877fff640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f48780903f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4868009840 tx=0x7f48680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.937+0000 7f48757fa640 1 -- 172.21.15.22:0/3308790175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4868041020 con 0x7f487806e530 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.937+0000 7f48757fa640 1 -- 172.21.15.22:0/3308790175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486803d550 con 0x7f487806e530 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.937+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4878073990 con 0x7f487806e530 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.937+0000 7f48757fa640 1 -- 172.21.15.22:0/3308790175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4868037040 con 0x7f487806e530 2024-09-19T11:22:56.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.937+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48780933f0 con 0x7f487806e530 2024-09-19T11:22:56.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.937+0000 7f48757fa640 1 -- 172.21.15.22:0/3308790175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4868044020 con 0x7f487806e530 2024-09-19T11:22:56.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.938+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487806e910 con 0x7f487806e530 2024-09-19T11:22:56.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.938+0000 7f48757fa640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f485803e710 0x7f4858040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:56.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.938+0000 7f48757fa640 1 -- 172.21.15.22:0/3308790175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f48680733d0 con 0x7f487806e530 2024-09-19T11:22:56.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.938+0000 7f486ffff640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f485803e710 0x7f4858040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:56.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.939+0000 7f486ffff640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f485803e710 0x7f4858040bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4860007fd0 tx=0x7f4860007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:56.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:56.942+0000 7f48757fa640 1 -- 172.21.15.22:0/3308790175 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f486804b030 con 0x7f487806e530 2024-09-19T11:22:57.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.133+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4878091f40 con 0x7f487806e530 2024-09-19T11:22:57.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.133+0000 7f48757fa640 1 -- 172.21.15.22:0/3308790175 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+9865 (secure 0 0 0) 0x7f4878129f90 con 0x7f487806e530 2024-09-19T11:22:57.135 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:57.135 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"fsid":"200d5f3b-d410-4b92-82cf-1cbe85431f50","created":"2024-09-19T11:22:36.212681+0000","modified":"2024-09-19T11:22:55.810737+0000","last_up_change":"2024-09-19T11:22:45.784999+0000","last_in_change":"2024-09-19T11:22:37.307013+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T11:22:44.794902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T11:22:51.830554+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0cbb56a-4180-4b0e-948a-cb474e8486a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6801","nonce":3144434255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6803","nonce":3144434255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6807","nonce":3144434255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6805","nonce":3144434255}]},"public_addr":"172.21.15.22:6801/3144434255","cluster_addr":"172.21.15.22:6803/3144434255","heartbeat_back_addr":"172.21.15.22:6807/3144434255","heartbeat_front_addr":"172.21.15.22:6805/3144434255","state":["exists","up"]},{"osd":1,"uuid":"f8538703-1f8e-48d3-8d2e-d423b3fb007a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6817","nonce":656119574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6819","nonce":656119574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6823","nonce":656119574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6821","nonce":656119574}]},"public_addr":"172.21.15.28:6817/656119574","cluster_addr":"172.21.15.28:6819/656119574","heartbeat_back_addr":"172.21.15.28:6823/656119574","heartbeat_front_addr":"172.21.15.28:6821/656119574","state":["exists","up"]},{"osd":2,"uuid":"e738a240-46c1-4d91-98ec-1e2d64c3f437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6801","nonce":2071226075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6803","nonce":2071226075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6807","nonce":2071226075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6805","nonce":2071226075}]},"public_addr":"172.21.15.28:6801/2071226075","cluster_addr":"172.21.15.28:6803/2071226075","heartbeat_back_addr":"172.21.15.28:6807/2071226075","heartbeat_front_addr":"172.21.15.28:6805/2071226075","state":["exists","up"]},{"osd":3,"uuid":"a884f1f1-c119-4e37-b10b-9038bb154d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6809","nonce":3027134945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6811","nonce":3027134945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6815","nonce":3027134945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6813","nonce":3027134945}]},"public_addr":"172.21.15.28:6809/3027134945","cluster_addr":"172.21.15.28:6811/3027134945","heartbeat_back_addr":"172.21.15.28:6815/3027134945","heartbeat_front_addr":"172.21.15.28:6813/3027134945","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.039577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:44.013759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:42.894612+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.051197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f485803e710 msgr2=0x7f4858040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f485803e710 0x7f4858040bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4860007fd0 tx=0x7f4860007aa0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 msgr2=0x7f48780903f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f48780903f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4868009840 tx=0x7f48680046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 shutdown_connections 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f485803e710 0x7f4858040bd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 --2- 172.21.15.22:0/3308790175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f487806e530 0x7f48780903f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 >> 172.21.15.22:0/3308790175 conn(0x7f48780a22c0 msgr2=0x7f487809da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 shutdown_connections 2024-09-19T11:22:57.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.136+0000 7f487e816640 1 -- 172.21.15.22:0/3308790175 wait complete. 2024-09-19T11:22:57.148 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T11:22:57.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T11:22:57.149 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T11:22:57.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T11:22:57.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.255+0000 7fa1af6d4640 1 Processor -- start 2024-09-19T11:22:57.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.255+0000 7fc7274c6640 1 Processor -- start 2024-09-19T11:22:57.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.255+0000 7fc5865db640 1 Processor -- start 2024-09-19T11:22:57.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.255+0000 7fa13ad5d640 1 Processor -- start 2024-09-19T11:22:57.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc5865db640 1 -- start start 2024-09-19T11:22:57.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa1af6d4640 1 -- start start 2024-09-19T11:22:57.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa13ad5d640 1 -- start start 2024-09-19T11:22:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc7274c6640 1 -- start start 2024-09-19T11:22:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa1af6d4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa1a812c0b0 0x7fa1a812c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa1af6d4640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a809a290 con 0x7fa1a812c9d0 2024-09-19T11:22:57.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa1af6d4640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a8071860 con 0x7fa1a812c0b0 2024-09-19T11:22:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa13ad5d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa13412c0b0 0x7fa13412c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa13ad5d640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13409a290 con 0x7fa13412c9d0 2024-09-19T11:22:57.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa13ad5d640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa134071860 con 0x7fa13412c0b0 2024-09-19T11:22:57.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc5865db640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc58012be50 0x7fc58012c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc7274c6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc72009b4a0 0x7fc72009b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa133fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa13412c9d0 0x7fa134097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49140/0 (socket says 172.21.15.22:49140) 2024-09-19T11:22:57.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa133fff640 1 -- 172.21.15.22:0/1812294468 learned_addr learned my addr 172.21.15.22:0/1812294468 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:57.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc7274c6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7200706a0 con 0x7fc72009bdc0 2024-09-19T11:22:57.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc7274c6640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc72009f400 con 0x7fc72009b4a0 2024-09-19T11:22:57.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa1acc6f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa1a812c9d0 0x7fa1a8097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49128/0 (socket says 172.21.15.22:49128) 2024-09-19T11:22:57.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa1acc6f640 1 -- 172.21.15.22:0/444576659 learned_addr learned my addr 172.21.15.22:0/444576659 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc5865db640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc58006f9f0 con 0x7fc58012c770 2024-09-19T11:22:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fc5865db640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc58006f580 con 0x7fc58012be50 2024-09-19T11:22:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa1ad470640 1 --2- 172.21.15.22:0/444576659 >> v2:172.21.15.22:3300/0 conn(0x7fa1a812c0b0 0x7fa1a812c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc725262640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc72009b4a0 0x7fc72009b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc724a61640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc72009bdc0 0x7fc72006efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49156/0 (socket says 172.21.15.22:49156) 2024-09-19T11:22:57.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc724a61640 1 -- 172.21.15.22:0/4180765193 learned_addr learned my addr 172.21.15.22:0/4180765193 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.256+0000 7fa138af9640 1 --2- 172.21.15.22:0/1812294468 >> v2:172.21.15.22:3300/0 conn(0x7fa13412c0b0 0x7fa13412c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc57ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc58012be50 0x7fc58012c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc57f7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc58012c770 0x7fc58006dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49160/0 (socket says 172.21.15.22:49160) 2024-09-19T11:22:57.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc57f7fe640 1 -- 172.21.15.22:0/136677840 learned_addr learned my addr 172.21.15.22:0/136677840 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2265770604 0 0) 0x7fa13409a290 con 0x7fa13412c9d0 2024-09-19T11:22:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa124003520 con 0x7fa13412c9d0 2024-09-19T11:22:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc725262640 1 -- 172.21.15.22:0/4180765193 >> v1:172.21.15.22:6789/0 conn(0x7fc72009bdc0 legacy=0x7fc72006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc725262640 1 -- 172.21.15.22:0/4180765193 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72009f950 con 0x7fc72009b4a0 2024-09-19T11:22:57.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1ad470640 1 -- 172.21.15.22:0/444576659 >> v1:172.21.15.22:6789/0 conn(0x7fa1a812c9d0 legacy=0x7fa1a8097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T11:22:57.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1ad470640 1 -- 172.21.15.22:0/444576659 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a8099e20 con 0x7fa1a812c0b0 2024-09-19T11:22:57.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 <== mon.0 v1:172.21.15.22:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3583732207 0 0) 0x7fa124003520 con 0x7fa13412c9d0 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 >> v2:172.21.15.22:3300/0 conn(0x7fa13412c0b0 msgr2=0x7fa13412c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 --2- 172.21.15.22:0/1812294468 >> v2:172.21.15.22:3300/0 conn(0x7fa13412c0b0 0x7fa13412c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 --> v1:172.21.15.22:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa134099e20 con 0x7fa13412c9d0 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 <== mon.0 v1:172.21.15.22:6789/0 3 ==== mon_map magic: 0 ==== 205+0+0 (unknown 144046082 0 0) 0x7fa120001620 con 0x7fa13412c9d0 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 >> v1:172.21.15.22:6789/0 conn(0x7fa13412c9d0 legacy=0x7fa134097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 --2- 172.21.15.22:0/1812294468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa124003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa134071860 con 0x7fa124003a50 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc57effd640 1 -- 172.21.15.22:0/136677840 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180715419 0 0) 0x7fc58006f9f0 con 0x7fc58012c770 2024-09-19T11:22:57.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc57effd640 1 -- 172.21.15.22:0/136677840 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc568003440 con 0x7fc58012c770 2024-09-19T11:22:57.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fa138af9640 1 --2- 172.21.15.22:0/1812294468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa124003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.257+0000 7fc57ffff640 1 -- 172.21.15.22:0/136677840 >> v1:172.21.15.22:6789/0 conn(0x7fc58012c770 legacy=0x7fc58006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fc57ffff640 1 -- 172.21.15.22:0/136677840 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc580070f50 con 0x7fc58012be50 2024-09-19T11:22:57.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fa138af9640 1 -- 172.21.15.22:0/1812294468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa124003520 con 0x7fa124003a50 2024-09-19T11:22:57.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fc725262640 1 --2- 172.21.15.22:0/4180765193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009b4a0 0x7fc72009b880 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009720 tx=0x7fc71c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57fa02f45e1f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fa1ad470640 1 --2- 172.21.15.22:0/444576659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c0b0 0x7fa1a812c490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009750 tx=0x7fa1a402a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb30a1173eaad55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fc717fff640 1 -- 172.21.15.22:0/4180765193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc71c037070 con 0x7fc72009b4a0 2024-09-19T11:22:57.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fc717fff640 1 -- 172.21.15.22:0/4180765193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c02ad90 con 0x7fc72009b4a0 2024-09-19T11:22:57.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fa19ffff640 1 -- 172.21.15.22:0/444576659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a4037070 con 0x7fa1a812c0b0 2024-09-19T11:22:57.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fc717fff640 1 -- 172.21.15.22:0/4180765193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc71c032590 con 0x7fc72009b4a0 2024-09-19T11:22:57.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fa19ffff640 1 -- 172.21.15.22:0/444576659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a402ace0 con 0x7fa1a812c0b0 2024-09-19T11:22:57.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.258+0000 7fa19ffff640 1 -- 172.21.15.22:0/444576659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a4032440 con 0x7fa1a812c0b0 2024-09-19T11:22:57.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc57ffff640 1 --2- 172.21.15.22:0/136677840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58012c230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc570009720 tx=0x7fc57002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8374b50eb88f227 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa138af9640 1 --2- 172.21.15.22:0/1812294468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa124003e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa128009a40 tx=0x7fa124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f62230b6a76bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128036070 con 0x7fa124003a50 2024-09-19T11:22:57.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc57effd640 1 -- 172.21.15.22:0/136677840 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc570037070 con 0x7fc58012be50 2024-09-19T11:22:57.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc57effd640 1 -- 172.21.15.22:0/136677840 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57002ad90 con 0x7fc58012be50 2024-09-19T11:22:57.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc57effd640 1 -- 172.21.15.22:0/136677840 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc570032590 con 0x7fc58012be50 2024-09-19T11:22:57.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa128031530 con 0x7fa124003a50 2024-09-19T11:22:57.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1337fe640 1 -- 172.21.15.22:0/1812294468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128009690 con 0x7fa124003a50 2024-09-19T11:22:57.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1af6d4640 1 -- 172.21.15.22:0/444576659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c0b0 msgr2=0x7fa1a812c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/444576659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c0b0 0x7fa1a812c490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009750 tx=0x7fa1a402a0a0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc7274c6640 1 -- 172.21.15.22:0/4180765193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009b4a0 msgr2=0x7fc72009b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc7274c6640 1 --2- 172.21.15.22:0/4180765193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009b4a0 0x7fc72009b880 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009720 tx=0x7fc71c02a110 comp rx=0 tx=0).stop 2024-09-19T11:22:57.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1af6d4640 1 -- 172.21.15.22:0/444576659 shutdown_connections 2024-09-19T11:22:57.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/444576659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c0b0 0x7fa1a812c490 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009750 tx=0x7fa1a402a0a0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc7274c6640 1 -- 172.21.15.22:0/4180765193 shutdown_connections 2024-09-19T11:22:57.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc7274c6640 1 --2- 172.21.15.22:0/4180765193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009b4a0 0x7fc72009b880 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009720 tx=0x7fc71c02a110 comp rx=0 tx=0).stop 2024-09-19T11:22:57.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc7274c6640 1 -- 172.21.15.22:0/4180765193 >> 172.21.15.22:0/4180765193 conn(0x7fc7200a25b0 msgr2=0x7fc72009ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1af6d4640 1 -- 172.21.15.22:0/444576659 >> 172.21.15.22:0/444576659 conn(0x7fa1a809abc0 msgr2=0x7fa1a8098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc7274c6640 1 -- 172.21.15.22:0/4180765193 shutdown_connections 2024-09-19T11:22:57.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1af6d4640 1 -- 172.21.15.22:0/444576659 shutdown_connections 2024-09-19T11:22:57.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fc7274c6640 1 -- 172.21.15.22:0/4180765193 wait complete. 2024-09-19T11:22:57.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.259+0000 7fa1af6d4640 1 -- 172.21.15.22:0/444576659 wait complete. 2024-09-19T11:22:57.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc5865db640 1 -- 172.21.15.22:0/136677840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 msgr2=0x7fc58012c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc5865db640 1 --2- 172.21.15.22:0/136677840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58012c230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc570009720 tx=0x7fc57002a110 comp rx=0 tx=0).stop 2024-09-19T11:22:57.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc5865db640 1 -- 172.21.15.22:0/136677840 shutdown_connections 2024-09-19T11:22:57.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc5865db640 1 --2- 172.21.15.22:0/136677840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58012c230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc5865db640 1 -- 172.21.15.22:0/136677840 >> 172.21.15.22:0/136677840 conn(0x7fc58009abc0 msgr2=0x7fc580098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc5865db640 1 -- 172.21.15.22:0/136677840 shutdown_connections 2024-09-19T11:22:57.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc5865db640 1 -- 172.21.15.22:0/136677840 wait complete. 2024-09-19T11:22:57.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 -- 172.21.15.22:0/1812294468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 msgr2=0x7fa124003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/1812294468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa124003e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa128009a40 tx=0x7fa124005830 comp rx=0 tx=0).stop 2024-09-19T11:22:57.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 -- 172.21.15.22:0/1812294468 shutdown_connections 2024-09-19T11:22:57.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/1812294468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa124003e50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/1812294468 >> v2:172.21.15.22:3300/0 conn(0x7fa13412c0b0 0x7fa13412c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 -- 172.21.15.22:0/1812294468 >> 172.21.15.22:0/1812294468 conn(0x7fa13409abc0 msgr2=0x7fa134098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 -- 172.21.15.22:0/1812294468 shutdown_connections 2024-09-19T11:22:57.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa13ad5d640 1 -- 172.21.15.22:0/1812294468 wait complete. 2024-09-19T11:22:57.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc7274c6640 1 Processor -- start 2024-09-19T11:22:57.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fc7274c6640 1 -- start start 2024-09-19T11:22:57.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.260+0000 7fa1af6d4640 1 Processor -- start 2024-09-19T11:22:57.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fa1af6d4640 1 -- start start 2024-09-19T11:22:57.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc7274c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009bdc0 0x7fc72012d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc7274c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc72006f6d0 con 0x7fc72009bdc0 2024-09-19T11:22:57.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc725262640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009bdc0 0x7fc72012d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc5865db640 1 Processor -- start 2024-09-19T11:22:57.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc725262640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009bdc0 0x7fc72012d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59956/0 (socket says 172.21.15.22:59956) 2024-09-19T11:22:57.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc725262640 1 -- 172.21.15.22:0/884181960 learned_addr learned my addr 172.21.15.22:0/884181960 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fa1af6d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c9d0 0x7fa1a81c0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc5865db640 1 -- start start 2024-09-19T11:22:57.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fa1af6d4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a812ce00 con 0x7fa1a812c9d0 2024-09-19T11:22:57.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fc725262640 1 -- 172.21.15.22:0/884181960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72012a070 con 0x7fc72009bdc0 2024-09-19T11:22:57.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fa13ad5d640 1 Processor -- start 2024-09-19T11:22:57.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fa1ad470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c9d0 0x7fa1a81c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa13ad5d640 1 -- start start 2024-09-19T11:22:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.261+0000 7fa1ad470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c9d0 0x7fa1a81c0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59970/0 (socket says 172.21.15.22:59970) 2024-09-19T11:22:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa1ad470640 1 -- 172.21.15.22:0/2331333822 learned_addr learned my addr 172.21.15.22:0/2331333822 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc725262640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009bdc0 0x7fc72012d4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc71c02ad60 tx=0x7fc71c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc5865db640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58008efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc5865db640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc580099f40 con 0x7fc58012be50 2024-09-19T11:22:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa1ad470640 1 -- 172.21.15.22:0/2331333822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a808c070 con 0x7fa1a812c9d0 2024-09-19T11:22:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc71c037050 con 0x7fc72009bdc0 2024-09-19T11:22:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc57ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58008efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c038040 con 0x7fc72009bdc0 2024-09-19T11:22:57.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc71c03c910 con 0x7fc72009bdc0 2024-09-19T11:22:57.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc57ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58008efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59982/0 (socket says 172.21.15.22:59982) 2024-09-19T11:22:57.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc57ffff640 1 -- 172.21.15.22:0/135946968 learned_addr learned my addr 172.21.15.22:0/135946968 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa1ad470640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c9d0 0x7fa1a81c0620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1a402a0e0 tx=0x7fa1a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc57ffff640 1 -- 172.21.15.22:0/135946968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc580090900 con 0x7fc58012be50 2024-09-19T11:22:57.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa13ad5d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa1341271d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc720072d10 con 0x7fc72009bdc0 2024-09-19T11:22:57.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72012da00 con 0x7fc72009bdc0 2024-09-19T11:22:57.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa13ad5d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa13412ce00 con 0x7fa124003a50 2024-09-19T11:22:57.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a4041070 con 0x7fa1a812c9d0 2024-09-19T11:22:57.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a403c460 con 0x7fa1a812c9d0 2024-09-19T11:22:57.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fa138af9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa1341271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fa138af9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa1341271d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59984/0 (socket says 172.21.15.22:59984) 2024-09-19T11:22:57.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fa138af9640 1 -- 172.21.15.22:0/782427622 learned_addr learned my addr 172.21.15.22:0/782427622 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fc57ffff640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58008efe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc570009840 tx=0x7fc5700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fa138af9640 1 -- 172.21.15.22:0/782427622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa134128fe0 con 0x7fa124003a50 2024-09-19T11:22:57.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc570042020 con 0x7fc58012be50 2024-09-19T11:22:57.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57003d470 con 0x7fc58012be50 2024-09-19T11:22:57.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc570037040 con 0x7fc58012be50 2024-09-19T11:22:57.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.262+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1a4037040 con 0x7fa1a812c9d0 2024-09-19T11:22:57.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a8128fc0 con 0x7fa1a812c9d0 2024-09-19T11:22:57.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc72006fbb0 con 0x7fc72009bdc0 2024-09-19T11:22:57.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81298a0 con 0x7fa1a812c9d0 2024-09-19T11:22:57.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5800737f0 con 0x7fc58012be50 2024-09-19T11:22:57.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.263+0000 7fa138af9640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa1341271d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa128034040 tx=0x7fa128004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58008f520 con 0x7fc58012be50 2024-09-19T11:22:57.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1a812c490 con 0x7fa1a812c9d0 2024-09-19T11:22:57.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc71c03cab0 con 0x7fc72009bdc0 2024-09-19T11:22:57.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128040070 con 0x7fa124003a50 2024-09-19T11:22:57.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa134074090 con 0x7fa124003a50 2024-09-19T11:22:57.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fc715ffb640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70403e710 0x7fc704040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fc71c0736e0 con 0x7fc72009bdc0 2024-09-19T11:22:57.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1341291e0 con 0x7fa124003a50 2024-09-19T11:22:57.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa128031530 con 0x7fa124003a50 2024-09-19T11:22:57.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa128049060 con 0x7fa124003a50 2024-09-19T11:22:57.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc58012c230 con 0x7fc58012be50 2024-09-19T11:22:57.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fc715ffb640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fc704043e90 0x7fc704046310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc71c02ad90 con 0x7fc704043e90 2024-09-19T11:22:57.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.264+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc71c07a050 con 0x7fc72009bdc0 2024-09-19T11:22:57.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fc724a61640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70403e710 0x7fc704040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fc725a63640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fc704043e90 0x7fc704046310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa1a4044070 con 0x7fa1a812c9d0 2024-09-19T11:22:57.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa114000f80 con 0x7fa124003a50 2024-09-19T11:22:57.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc570045020 con 0x7fc58012be50 2024-09-19T11:22:57.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fa19dffb640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa18c03e7b0 0x7fa18c040c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fc725a63640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fc704043e90 0x7fc704046310 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.265+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fa1a4073f10 con 0x7fa1a812c9d0 2024-09-19T11:22:57.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa19dffb640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa18c043fb0 0x7fa18c046430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1a402ace0 con 0x7fa18c043fb0 2024-09-19T11:22:57.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fa1a4074280 con 0x7fa1a812c9d0 2024-09-19T11:22:57.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc57cff9640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc56c03e7e0 0x7fc56c040ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fc5700735b0 con 0x7fc58012be50 2024-09-19T11:22:57.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa1adc71640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa18c043fb0 0x7fa18c046430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc724a61640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70403e710 0x7fc704040bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc710007fd0 tx=0x7fc710007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa1acc6f640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa18c03e7b0 0x7fa18c040c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc57cff9640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fc56c043fe0 0x7fc56c046460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc57002ad90 con 0x7fc56c043fe0 2024-09-19T11:22:57.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fc57007a050 con 0x7fc58012be50 2024-09-19T11:22:57.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc57f7fe640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc56c03e7e0 0x7fc56c040ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa128039070 con 0x7fa124003a50 2024-09-19T11:22:57.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc584b78640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fc56c043fe0 0x7fc56c046460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa1adc71640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa18c043fb0 0x7fa18c046430 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa1acc6f640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa18c03e7b0 0x7fa18c040c70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa198007fd0 tx=0x7fa198007b40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa1317fa640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa11803e710 0x7fa118040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fa12800f040 con 0x7fa124003a50 2024-09-19T11:22:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.266+0000 7fc57f7fe640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc56c03e7e0 0x7fc56c040ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc574001610 tx=0x7fc5740078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fa1317fa640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa118043e90 0x7fa118046310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa128007e80 con 0x7fa118043e90 2024-09-19T11:22:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fa12807a050 con 0x7fa124003a50 2024-09-19T11:22:57.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fa133fff640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa11803e710 0x7fa118040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fc584b78640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fc56c043fe0 0x7fc56c046460 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc71c02ad90 con 0x7fc704043e90 2024-09-19T11:22:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fa1392fa640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa118043e90 0x7fa118046310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1a402ace0 con 0x7fa18c043fb0 2024-09-19T11:22:57.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fa1392fa640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa118043e90 0x7fa118046310 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.268+0000 7fa133fff640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa11803e710 0x7fa118040bd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa120000f60 tx=0x7fa1200014b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.267+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc57002ad90 con 0x7fc56c043fe0 2024-09-19T11:22:57.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.268+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa128007e80 con 0x7fa118043e90 2024-09-19T11:22:57.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.281+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc72012f5e0 con 0x7fc704043e90 2024-09-19T11:22:57.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.281+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5800915e0 con 0x7fc56c043fe0 2024-09-19T11:22:57.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.281+0000 7fc715ffb640 1 -- 172.21.15.22:0/884181960 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc72012f5e0 con 0x7fc704043e90 2024-09-19T11:22:57.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.281+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fc704043e90 msgr2=0x7fc704046310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.281+0000 7fc7274c6640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fc704043e90 0x7fc704046310 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.281+0000 7fc57cff9640 1 -- 172.21.15.22:0/135946968 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc5800915e0 con 0x7fc56c043fe0 2024-09-19T11:22:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70403e710 msgr2=0x7fc704040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fc56c043fe0 msgr2=0x7fc56c046460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fc56c043fe0 0x7fc56c046460 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70403e710 0x7fc704040bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc710007fd0 tx=0x7fc710007aa0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009bdc0 msgr2=0x7fc72012d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc56c03e7e0 msgr2=0x7fc56c040ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc56c03e7e0 0x7fc56c040ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc574001610 tx=0x7fc5740078f0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 msgr2=0x7fc58008efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58008efe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc570009840 tx=0x7fc5700046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 shutdown_connections 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fc56c043fe0 0x7fc56c046460 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc56c03e7e0 0x7fc56c040ca0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 --2- 172.21.15.22:0/135946968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc58012be50 0x7fc58008efe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 >> 172.21.15.22:0/135946968 conn(0x7fc58009abc0 msgr2=0x7fc58009bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009bdc0 0x7fc72012d4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc71c02ad60 tx=0x7fc71c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 shutdown_connections 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70403e710 0x7fc704040bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fc704043e90 0x7fc704046310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 --2- 172.21.15.22:0/884181960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc72009bdc0 0x7fc72012d4c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 >> 172.21.15.22:0/884181960 conn(0x7fc7200a25b0 msgr2=0x7fc72009c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 shutdown_connections 2024-09-19T11:22:57.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc7274c6640 1 -- 172.21.15.22:0/884181960 wait complete. 2024-09-19T11:22:57.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 shutdown_connections 2024-09-19T11:22:57.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.282+0000 7fc5865db640 1 -- 172.21.15.22:0/135946968 wait complete. 2024-09-19T11:22:57.311 INFO:teuthology.orchestra.run.smithi022.stdout:42949672965 2024-09-19T11:22:57.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T11:22:57.312 INFO:teuthology.orchestra.run.smithi022.stdout:42949672965 2024-09-19T11:22:57.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T11:22:57.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.298+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1a812a310 con 0x7fa18c043fb0 2024-09-19T11:22:57.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.298+0000 7fa19dffb640 1 -- 172.21.15.22:0/2331333822 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa1a812a310 con 0x7fa18c043fb0 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa18c043fb0 msgr2=0x7fa18c046430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa18c043fb0 0x7fa18c046430 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa18c03e7b0 msgr2=0x7fa18c040c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa18c03e7b0 0x7fa18c040c70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa198007fd0 tx=0x7fa198007b40 comp rx=0 tx=0).stop 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c9d0 msgr2=0x7fa1a81c0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c9d0 0x7fa1a81c0620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1a402a0e0 tx=0x7fa1a40046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 shutdown_connections 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa18c043fb0 0x7fa18c046430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa18c03e7b0 0x7fa18c040c70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 --2- 172.21.15.22:0/2331333822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1a812c9d0 0x7fa1a81c0620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 >> 172.21.15.22:0/2331333822 conn(0x7fa1a809abc0 msgr2=0x7fa1a80975c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa114002c70 con 0x7fa118043e90 2024-09-19T11:22:57.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 shutdown_connections 2024-09-19T11:22:57.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1af6d4640 1 -- 172.21.15.22:0/2331333822 wait complete. 2024-09-19T11:22:57.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.299+0000 7fa1317fa640 1 -- 172.21.15.22:0/782427622 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa114002c70 con 0x7fa118043e90 2024-09-19T11:22:57.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa118043e90 msgr2=0x7fa118046310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa118043e90 0x7fa118046310 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa11803e710 msgr2=0x7fa118040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa11803e710 0x7fa118040bd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa120000f60 tx=0x7fa1200014b0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 msgr2=0x7fa1341271d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa1341271d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa128034040 tx=0x7fa128004730 comp rx=0 tx=0).stop 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 shutdown_connections 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa11803e710 0x7fa118040bd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa118043e90 0x7fa118046310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 --2- 172.21.15.22:0/782427622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa124003a50 0x7fa1341271d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 >> 172.21.15.22:0/782427622 conn(0x7fa13409abc0 msgr2=0x7fa134072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 shutdown_connections 2024-09-19T11:22:57.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.300+0000 7fa13ad5d640 1 -- 172.21.15.22:0/782427622 wait complete. 2024-09-19T11:22:57.318 INFO:teuthology.orchestra.run.smithi022.stdout:38654705669 2024-09-19T11:22:57.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T11:22:57.319 INFO:teuthology.orchestra.run.smithi022.stdout:38654705669 2024-09-19T11:22:57.320 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T11:22:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.421+0000 7f194607f640 1 Processor -- start 2024-09-19T11:22:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.421+0000 7f6226a40640 1 Processor -- start 2024-09-19T11:22:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.421+0000 7f6226a40640 1 -- start start 2024-09-19T11:22:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.421+0000 7f194607f640 1 -- start start 2024-09-19T11:22:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f194607f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f194006fab0 0x7f1940134820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f6226a40640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f622012cba0 0x7f62201315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f194607f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1940070330 con 0x7f194006efc0 2024-09-19T11:22:57.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f194607f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f194009f310 con 0x7f194006fab0 2024-09-19T11:22:57.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f6226a40640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f622009a290 con 0x7f622012c0b0 2024-09-19T11:22:57.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f6226a40640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6220071860 con 0x7f622012cba0 2024-09-19T11:22:57.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f193effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f194006fab0 0x7f1940134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f193effd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f194006fab0 0x7f1940134820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59994/0 (socket says 172.21.15.22:59994) 2024-09-19T11:22:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f193effd640 1 -- 172.21.15.22:0/1431167968 learned_addr learned my addr 172.21.15.22:0/1431167968 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f621f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f622012cba0 0x7f62201315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f621f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f622012cba0 0x7f62201315e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59988/0 (socket says 172.21.15.22:59988) 2024-09-19T11:22:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f621f7fe640 1 -- 172.21.15.22:0/3934487194 learned_addr learned my addr 172.21.15.22:0/3934487194 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f193effd640 1 -- 172.21.15.22:0/1431167968 >> v1:172.21.15.22:6789/0 conn(0x7f194006efc0 legacy=0x7f194006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.422+0000 7f193effd640 1 -- 172.21.15.22:0/1431167968 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1940070640 con 0x7f194006fab0 2024-09-19T11:22:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.423+0000 7f621f7fe640 1 -- 172.21.15.22:0/3934487194 >> v1:172.21.15.22:6789/0 conn(0x7f622012c0b0 legacy=0x7f622012c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.423+0000 7f621f7fe640 1 -- 172.21.15.22:0/3934487194 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6220099e20 con 0x7f622012cba0 2024-09-19T11:22:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.423+0000 7f193effd640 1 --2- 172.21.15.22:0/1431167968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006fab0 0x7f1940134820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19340097f0 tx=0x7f193402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd87eae51680ca91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.423+0000 7f193e7fc640 1 -- 172.21.15.22:0/1431167968 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1934037070 con 0x7f194006fab0 2024-09-19T11:22:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.423+0000 7f193e7fc640 1 -- 172.21.15.22:0/1431167968 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1934032440 con 0x7f194006fab0 2024-09-19T11:22:57.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.423+0000 7f193e7fc640 1 -- 172.21.15.22:0/1431167968 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f193402ae60 con 0x7f194006fab0 2024-09-19T11:22:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f621f7fe640 1 --2- 172.21.15.22:0/3934487194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012cba0 0x7f62201315e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6214009820 tx=0x7f621402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb030ffba0135fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f621effd640 1 -- 172.21.15.22:0/3934487194 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6214037070 con 0x7f622012cba0 2024-09-19T11:22:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f621effd640 1 -- 172.21.15.22:0/3934487194 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f621402ae50 con 0x7f622012cba0 2024-09-19T11:22:57.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f621effd640 1 -- 172.21.15.22:0/3934487194 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6214032590 con 0x7f622012cba0 2024-09-19T11:22:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f194607f640 1 -- 172.21.15.22:0/1431167968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006fab0 msgr2=0x7f1940134820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f194607f640 1 --2- 172.21.15.22:0/1431167968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006fab0 0x7f1940134820 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19340097f0 tx=0x7f193402a1e0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f194607f640 1 -- 172.21.15.22:0/1431167968 shutdown_connections 2024-09-19T11:22:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f194607f640 1 --2- 172.21.15.22:0/1431167968 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006fab0 0x7f1940134820 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f194607f640 1 -- 172.21.15.22:0/1431167968 >> 172.21.15.22:0/1431167968 conn(0x7f19400a2460 msgr2=0x7f194009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f194607f640 1 -- 172.21.15.22:0/1431167968 shutdown_connections 2024-09-19T11:22:57.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f194607f640 1 -- 172.21.15.22:0/1431167968 wait complete. 2024-09-19T11:22:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.424+0000 7f6226a40640 1 -- 172.21.15.22:0/3934487194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012cba0 msgr2=0x7f62201315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f6226a40640 1 --2- 172.21.15.22:0/3934487194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012cba0 0x7f62201315e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6214009820 tx=0x7f621402a210 comp rx=0 tx=0).stop 2024-09-19T11:22:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f6226a40640 1 -- 172.21.15.22:0/3934487194 shutdown_connections 2024-09-19T11:22:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f6226a40640 1 --2- 172.21.15.22:0/3934487194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012cba0 0x7f62201315e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f6226a40640 1 -- 172.21.15.22:0/3934487194 >> 172.21.15.22:0/3934487194 conn(0x7f622009abc0 msgr2=0x7f6220098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f6226a40640 1 -- 172.21.15.22:0/3934487194 shutdown_connections 2024-09-19T11:22:57.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f6226a40640 1 -- 172.21.15.22:0/3934487194 wait complete. 2024-09-19T11:22:57.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f194607f640 1 Processor -- start 2024-09-19T11:22:57.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.425+0000 7f6226a40640 1 Processor -- start 2024-09-19T11:22:57.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f194607f640 1 -- start start 2024-09-19T11:22:57.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f6226a40640 1 -- start start 2024-09-19T11:22:57.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f194607f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006efc0 0x7f1940098c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f194607f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1940134ee0 con 0x7f194006efc0 2024-09-19T11:22:57.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f6226a40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012c0b0 0x7f622012d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f193f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006efc0 0x7f1940098c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f6226a40640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6220070c40 con 0x7f622012c0b0 2024-09-19T11:22:57.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f621ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012c0b0 0x7f622012d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f621ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012c0b0 0x7f622012d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60016/0 (socket says 172.21.15.22:60016) 2024-09-19T11:22:57.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f621ffff640 1 -- 172.21.15.22:0/3399345625 learned_addr learned my addr 172.21.15.22:0/3399345625 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f193f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006efc0 0x7f1940098c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60004/0 (socket says 172.21.15.22:60004) 2024-09-19T11:22:57.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.426+0000 7f193f7fe640 1 -- 172.21.15.22:0/2666683297 learned_addr learned my addr 172.21.15.22:0/2666683297 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f621ffff640 1 -- 172.21.15.22:0/3399345625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6220126ac0 con 0x7f622012c0b0 2024-09-19T11:22:57.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f193f7fe640 1 -- 172.21.15.22:0/2666683297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194009c2f0 con 0x7f194006efc0 2024-09-19T11:22:57.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f621ffff640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012c0b0 0x7f622012d5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f621000c480 tx=0x7f621000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f621cff9640 1 -- 172.21.15.22:0/3399345625 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6210017070 con 0x7f622012c0b0 2024-09-19T11:22:57.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f193f7fe640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006efc0 0x7f1940098c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f193000c480 tx=0x7f193000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f621cff9640 1 -- 172.21.15.22:0/3399345625 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6210005200 con 0x7f622012c0b0 2024-09-19T11:22:57.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6220097b70 con 0x7f622012c0b0 2024-09-19T11:22:57.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f621cff9640 1 -- 172.21.15.22:0/3399345625 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62100035f0 con 0x7f622012c0b0 2024-09-19T11:22:57.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622012dd10 con 0x7f622012c0b0 2024-09-19T11:22:57.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f1927fff640 1 -- 172.21.15.22:0/2666683297 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1930017070 con 0x7f194006efc0 2024-09-19T11:22:57.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f1927fff640 1 -- 172.21.15.22:0/2666683297 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1930005200 con 0x7f194006efc0 2024-09-19T11:22:57.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194012ca30 con 0x7f194006efc0 2024-09-19T11:22:57.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.427+0000 7f1927fff640 1 -- 172.21.15.22:0/2666683297 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19300035f0 con 0x7f194006efc0 2024-09-19T11:22:57.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.428+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194009b6b0 con 0x7f194006efc0 2024-09-19T11:22:57.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.429+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622012c490 con 0x7f622012c0b0 2024-09-19T11:22:57.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.429+0000 7f621cff9640 1 -- 172.21.15.22:0/3399345625 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f621001a070 con 0x7f622012c0b0 2024-09-19T11:22:57.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.429+0000 7f1927fff640 1 -- 172.21.15.22:0/2666683297 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f193001a070 con 0x7f194006efc0 2024-09-19T11:22:57.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.429+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f194009b9f0 con 0x7f194006efc0 2024-09-19T11:22:57.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.429+0000 7f621cff9640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620c03e710 0x7f620c040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.429+0000 7f1927fff640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f192c03e760 0x7f192c040c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.429+0000 7f621cff9640 1 -- 172.21.15.22:0/3399345625 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f6210052110 con 0x7f622012c0b0 2024-09-19T11:22:57.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.430+0000 7f1927fff640 1 -- 172.21.15.22:0/2666683297 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f19300520e0 con 0x7f194006efc0 2024-09-19T11:22:57.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.430+0000 7f621f7fe640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620c03e710 0x7f620c040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.430+0000 7f193effd640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f192c03e760 0x7f192c040c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.430+0000 7f193effd640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f192c03e760 0x7f192c040c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1934034000 tx=0x7f193402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.430+0000 7f621f7fe640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620c03e710 0x7f620c040bd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6214034000 tx=0x7f621402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.434+0000 7f621cff9640 1 -- 172.21.15.22:0/3399345625 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f6210011590 con 0x7f622012c0b0 2024-09-19T11:22:57.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.439+0000 7f1927fff640 1 -- 172.21.15.22:0/2666683297 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f194009b9f0 con 0x7f194006efc0 2024-09-19T11:22:57.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.468+0000 7f0374a23640 1 Processor -- start 2024-09-19T11:22:57.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f0374a23640 1 -- start start 2024-09-19T11:22:57.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f77fb532640 1 Processor -- start 2024-09-19T11:22:57.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f77fb532640 1 -- start start 2024-09-19T11:22:57.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f0374a23640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0370071e00 0x7f0370132800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f0374a23640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03700724c0 con 0x7f0370071310 2024-09-19T11:22:57.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f0374a23640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f037009c2e0 con 0x7f0370071e00 2024-09-19T11:22:57.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f77fb532640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f77f4095b60 0x7f77f4134960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f036dd9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0370071e00 0x7f0370132800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f036dd9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f0370071e00 0x7f0370132800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60024/0 (socket says 172.21.15.22:60024) 2024-09-19T11:22:57.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f036dd9b640 1 -- 172.21.15.22:0/2718712432 learned_addr learned my addr 172.21.15.22:0/2718712432 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f77fb532640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77f4073410 con 0x7f77f4095070 2024-09-19T11:22:57.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f77fb532640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f77f4073a00 con 0x7f77f4095b60 2024-09-19T11:22:57.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f77f92ce640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f77f4095070 0x7f77f4095450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49186/0 (socket says 172.21.15.22:49186) 2024-09-19T11:22:57.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.469+0000 7f77f92ce640 1 -- 172.21.15.22:0/432434731 learned_addr learned my addr 172.21.15.22:0/432434731 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:57.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.470+0000 7f77f8acd640 1 --2- 172.21.15.22:0/432434731 >> v2:172.21.15.22:3300/0 conn(0x7f77f4095b60 0x7f77f4134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.470+0000 7f036dd9b640 1 -- 172.21.15.22:0/2718712432 >> v1:172.21.15.22:6789/0 conn(0x7f0370071310 legacy=0x7f03700716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T11:22:57.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.470+0000 7f036dd9b640 1 -- 172.21.15.22:0/2718712432 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037009be60 con 0x7f0370071e00 2024-09-19T11:22:57.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.470+0000 7f77ebfff640 1 -- 172.21.15.22:0/432434731 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4274132887 0 0) 0x7f77f4073410 con 0x7f77f4095070 2024-09-19T11:22:57.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.470+0000 7f77ebfff640 1 -- 172.21.15.22:0/432434731 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77e0003520 con 0x7f77f4095070 2024-09-19T11:22:57.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.470+0000 7f77f8acd640 1 -- 172.21.15.22:0/432434731 >> v1:172.21.15.22:6789/0 conn(0x7f77f4095070 legacy=0x7f77f4095450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.470+0000 7f77f8acd640 1 -- 172.21.15.22:0/432434731 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f4074020 con 0x7f77f4095b60 2024-09-19T11:22:57.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f036dd9b640 1 --2- 172.21.15.22:0/2718712432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071e00 0x7f0370132800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03640097f0 tx=0x7f036402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fd9425401e65a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f036d59a640 1 -- 172.21.15.22:0/2718712432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0364037070 con 0x7f0370071e00 2024-09-19T11:22:57.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f036d59a640 1 -- 172.21.15.22:0/2718712432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0364032440 con 0x7f0370071e00 2024-09-19T11:22:57.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f036d59a640 1 -- 172.21.15.22:0/2718712432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036402ae60 con 0x7f0370071e00 2024-09-19T11:22:57.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f77f8acd640 1 --2- 172.21.15.22:0/432434731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095b60 0x7f77f4134960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009720 tx=0x7f77e402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125614e659945297 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f77ebfff640 1 -- 172.21.15.22:0/432434731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77e4037070 con 0x7f77f4095b60 2024-09-19T11:22:57.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f77ebfff640 1 -- 172.21.15.22:0/432434731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e402ad90 con 0x7f77f4095b60 2024-09-19T11:22:57.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.471+0000 7f77ebfff640 1 -- 172.21.15.22:0/432434731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77e4032590 con 0x7f77f4095b60 2024-09-19T11:22:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f0374a23640 1 -- 172.21.15.22:0/2718712432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071e00 msgr2=0x7f0370132800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f0374a23640 1 --2- 172.21.15.22:0/2718712432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071e00 0x7f0370132800 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f03640097f0 tx=0x7f036402a1e0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f0374a23640 1 -- 172.21.15.22:0/2718712432 shutdown_connections 2024-09-19T11:22:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f0374a23640 1 --2- 172.21.15.22:0/2718712432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071e00 0x7f0370132800 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f0374a23640 1 -- 172.21.15.22:0/2718712432 >> 172.21.15.22:0/2718712432 conn(0x7f037009cb40 msgr2=0x7f037009cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f0374a23640 1 -- 172.21.15.22:0/2718712432 shutdown_connections 2024-09-19T11:22:57.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f0374a23640 1 -- 172.21.15.22:0/2718712432 wait complete. 2024-09-19T11:22:57.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f77fb532640 1 -- 172.21.15.22:0/432434731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095b60 msgr2=0x7f77f4134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f77fb532640 1 --2- 172.21.15.22:0/432434731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095b60 0x7f77f4134960 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009720 tx=0x7f77e402a110 comp rx=0 tx=0).stop 2024-09-19T11:22:57.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f77fb532640 1 -- 172.21.15.22:0/432434731 shutdown_connections 2024-09-19T11:22:57.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f77fb532640 1 --2- 172.21.15.22:0/432434731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095b60 0x7f77f4134960 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f77fb532640 1 -- 172.21.15.22:0/432434731 >> 172.21.15.22:0/432434731 conn(0x7f77f409f320 msgr2=0x7f77f409b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f77fb532640 1 -- 172.21.15.22:0/432434731 shutdown_connections 2024-09-19T11:22:57.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.472+0000 7f77fb532640 1 -- 172.21.15.22:0/432434731 wait complete. 2024-09-19T11:22:57.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.473+0000 7f0374a23640 1 Processor -- start 2024-09-19T11:22:57.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.473+0000 7f0374a23640 1 -- start start 2024-09-19T11:22:57.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.473+0000 7f77fb532640 1 Processor -- start 2024-09-19T11:22:57.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.473+0000 7f77fb532640 1 -- start start 2024-09-19T11:22:57.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.473+0000 7f0374a23640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071310 0x7f03700966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.473+0000 7f0374a23640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f03701330d0 con 0x7f0370071310 2024-09-19T11:22:57.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f036e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071310 0x7f03700966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f036e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071310 0x7f03700966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60044/0 (socket says 172.21.15.22:60044) 2024-09-19T11:22:57.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f036e59c640 1 -- 172.21.15.22:0/4016194711 learned_addr learned my addr 172.21.15.22:0/4016194711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f77fb532640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095070 0x7f77f41c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f77fb532640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77f4134ea0 con 0x7f77f4095070 2024-09-19T11:22:57.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f77f92ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095070 0x7f77f41c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f77f92ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095070 0x7f77f41c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60048/0 (socket says 172.21.15.22:60048) 2024-09-19T11:22:57.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f77f92ce640 1 -- 172.21.15.22:0/1528891514 learned_addr learned my addr 172.21.15.22:0/1528891514 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:57.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f036e59c640 1 -- 172.21.15.22:0/4016194711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0370127d10 con 0x7f0370071310 2024-09-19T11:22:57.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f77f92ce640 1 -- 172.21.15.22:0/1528891514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f40742d0 con 0x7f77f4095070 2024-09-19T11:22:57.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f036e59c640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071310 0x7f03700966c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f036000c3b0 tx=0x7f036000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f77f92ce640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095070 0x7f77f41c41c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c480 tx=0x7f77f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.474+0000 7f035effd640 1 -- 172.21.15.22:0/4016194711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03600116c0 con 0x7f0370071310 2024-09-19T11:22:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f035effd640 1 -- 172.21.15.22:0/4016194711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0360017070 con 0x7f0370071310 2024-09-19T11:22:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0370070ad0 con 0x7f0370071310 2024-09-19T11:22:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f035effd640 1 -- 172.21.15.22:0/4016194711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03600119d0 con 0x7f0370071310 2024-09-19T11:22:57.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0370098a20 con 0x7f0370071310 2024-09-19T11:22:57.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f77e9ffb640 1 -- 172.21.15.22:0/1528891514 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f0017070 con 0x7f77f4095070 2024-09-19T11:22:57.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f77e9ffb640 1 -- 172.21.15.22:0/1528891514 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f0005cd0 con 0x7f77f4095070 2024-09-19T11:22:57.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f77e9ffb640 1 -- 172.21.15.22:0/1528891514 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77f0007200 con 0x7f77f4095070 2024-09-19T11:22:57.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f408fa70 con 0x7f77f4095070 2024-09-19T11:22:57.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.475+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f41c48e0 con 0x7f77f4095070 2024-09-19T11:22:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.476+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03700a0d10 con 0x7f0370071310 2024-09-19T11:22:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.476+0000 7f035effd640 1 -- 172.21.15.22:0/4016194711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0360011c80 con 0x7f0370071310 2024-09-19T11:22:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.477+0000 7f035effd640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f035003e6c0 0x7f0350040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.477+0000 7f035effd640 1 -- 172.21.15.22:0/4016194711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f0360052c00 con 0x7f0370071310 2024-09-19T11:22:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.476+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f4095450 con 0x7f77f4095070 2024-09-19T11:22:57.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.477+0000 7f77e9ffb640 1 -- 172.21.15.22:0/1528891514 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f77f001a070 con 0x7f77f4095070 2024-09-19T11:22:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.477+0000 7f036dd9b640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f035003e6c0 0x7f0350040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.477+0000 7f77e9ffb640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f77d803e760 0x7f77d8040c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.477+0000 7f77e9ffb640 1 -- 172.21.15.22:0/1528891514 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f77f00520b0 con 0x7f77f4095070 2024-09-19T11:22:57.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.477+0000 7f77f8acd640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f77d803e760 0x7f77d8040c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.478+0000 7f036dd9b640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f035003e6c0 0x7f0350040b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0364034000 tx=0x7f036402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.478+0000 7f77f8acd640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f77d803e760 0x7f77d8040c20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009720 tx=0x7f77e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.481+0000 7f035effd640 1 -- 172.21.15.22:0/4016194711 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f0360012420 con 0x7f0370071310 2024-09-19T11:22:57.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.486+0000 7f77e9ffb640 1 -- 172.21.15.22:0/1528891514 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f77f001b3b0 con 0x7f77f4095070 2024-09-19T11:22:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.668+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0370099270 con 0x7f0370071310 2024-09-19T11:22:57.670 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.669+0000 7f035effd640 1 -- 172.21.15.22:0/4016194711 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f036001ac80 con 0x7f0370071310 2024-09-19T11:22:57.670 INFO:teuthology.orchestra.run.smithi022.stdout:38654705668 2024-09-19T11:22:57.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.669+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6220074090 con 0x7f622012c0b0 2024-09-19T11:22:57.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.669+0000 7f621cff9640 1 -- 172.21.15.22:0/3399345625 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f621001bdd0 con 0x7f622012c0b0 2024-09-19T11:22:57.671 INFO:teuthology.orchestra.run.smithi022.stdout:42949672963 2024-09-19T11:22:57.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f035003e6c0 msgr2=0x7f0350040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f035003e6c0 0x7f0350040b80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0364034000 tx=0x7f036402e000 comp rx=0 tx=0).stop 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071310 msgr2=0x7f03700966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071310 0x7f03700966c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f036000c3b0 tx=0x7f036000c870 comp rx=0 tx=0).stop 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 shutdown_connections 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f035003e6c0 0x7f0350040b80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 --2- 172.21.15.22:0/4016194711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0370071310 0x7f03700966c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 >> 172.21.15.22:0/4016194711 conn(0x7f037009cb40 msgr2=0x7f037012d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 shutdown_connections 2024-09-19T11:22:57.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.671+0000 7f0374a23640 1 -- 172.21.15.22:0/4016194711 wait complete. 2024-09-19T11:22:57.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620c03e710 msgr2=0x7f620c040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620c03e710 0x7f620c040bd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6214034000 tx=0x7f621402e000 comp rx=0 tx=0).stop 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012c0b0 msgr2=0x7f622012d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012c0b0 0x7f622012d5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f621000c480 tx=0x7f621000c940 comp rx=0 tx=0).stop 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 shutdown_connections 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620c03e710 0x7f620c040bd0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 --2- 172.21.15.22:0/3399345625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622012c0b0 0x7f622012d5f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 >> 172.21.15.22:0/3399345625 conn(0x7f622009abc0 msgr2=0x7f622006dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 shutdown_connections 2024-09-19T11:22:57.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.672+0000 7f6226a40640 1 -- 172.21.15.22:0/3399345625 wait complete. 2024-09-19T11:22:57.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.674+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f194006f3a0 con 0x7f194006efc0 2024-09-19T11:22:57.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.675+0000 7f1927fff640 1 -- 172.21.15.22:0/2666683297 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1930011590 con 0x7f194006efc0 2024-09-19T11:22:57.677 INFO:teuthology.orchestra.run.smithi022.stdout:42949672963 2024-09-19T11:22:57.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f192c03e760 msgr2=0x7f192c040c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f192c03e760 0x7f192c040c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1934034000 tx=0x7f193402e000 comp rx=0 tx=0).stop 2024-09-19T11:22:57.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006efc0 msgr2=0x7f1940098c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006efc0 0x7f1940098c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f193000c480 tx=0x7f193000c940 comp rx=0 tx=0).stop 2024-09-19T11:22:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 shutdown_connections 2024-09-19T11:22:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f192c03e760 0x7f192c040c20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 --2- 172.21.15.22:0/2666683297 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f194006efc0 0x7f1940098c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 >> 172.21.15.22:0/2666683297 conn(0x7f19400a2460 msgr2=0x7f194012f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 shutdown_connections 2024-09-19T11:22:57.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.677+0000 7f194607f640 1 -- 172.21.15.22:0/2666683297 wait complete. 2024-09-19T11:22:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.678+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f77f408f790 con 0x7f77f4095070 2024-09-19T11:22:57.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.678+0000 7f77e9ffb640 1 -- 172.21.15.22:0/1528891514 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f77f001b730 con 0x7f77f4095070 2024-09-19T11:22:57.681 INFO:teuthology.orchestra.run.smithi022.stdout:38654705668 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f77d803e760 msgr2=0x7f77d8040c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f77d803e760 0x7f77d8040c20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009720 tx=0x7f77e402e000 comp rx=0 tx=0).stop 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095070 msgr2=0x7f77f41c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095070 0x7f77f41c41c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c480 tx=0x7f77f000c940 comp rx=0 tx=0).stop 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 shutdown_connections 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f77d803e760 0x7f77d8040c20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 --2- 172.21.15.22:0/1528891514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77f4095070 0x7f77f41c41c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 >> 172.21.15.22:0/1528891514 conn(0x7f77f409f320 msgr2=0x7f77f412f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 shutdown_connections 2024-09-19T11:22:57.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:57.682+0000 7f77fb532640 1 -- 172.21.15.22:0/1528891514 wait complete. 2024-09-19T11:22:57.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672963 for osd.3 2024-09-19T11:22:57.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2024-09-19T11:22:57.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672963 for osd.1 2024-09-19T11:22:57.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-09-19T11:22:58.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T11:22:58.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T11:22:58.693 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T11:22:58.696 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T11:22:58.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f6471a71640 1 Processor -- start 2024-09-19T11:22:58.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdd05e16640 1 Processor -- start 2024-09-19T11:22:58.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdd05e16640 1 -- start start 2024-09-19T11:22:58.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f6471a71640 1 -- start start 2024-09-19T11:22:58.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdd05e16640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdd0006fab0 0x7fdd00134820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdd05e16640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd00070330 con 0x7fdd0006efc0 2024-09-19T11:22:58.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdd05e16640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0009f310 con 0x7fdd0006fab0 2024-09-19T11:22:58.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f6471a71640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f646c06efc0 0x7f646c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f6471a71640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646c070330 con 0x7f646c06f8e0 2024-09-19T11:22:58.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f6471a71640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f646c09f310 con 0x7f646c06efc0 2024-09-19T11:22:58.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdcfeffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdd0006fab0 0x7fdd00134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdcfeffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdd0006fab0 0x7fdd00134820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60064/0 (socket says 172.21.15.22:60064) 2024-09-19T11:22:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f646b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f646c06efc0 0x7f646c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f646affd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f646c06f8e0 0x7f646c09b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49194/0 (socket says 172.21.15.22:49194) 2024-09-19T11:22:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f646affd640 1 -- 172.21.15.22:0/3580389077 learned_addr learned my addr 172.21.15.22:0/3580389077 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7f646b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f646c06efc0 0x7f646c06f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60068/0 (socket says 172.21.15.22:60068) 2024-09-19T11:22:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.807+0000 7fdcfeffd640 1 -- 172.21.15.22:0/3027708711 learned_addr learned my addr 172.21.15.22:0/3027708711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.808+0000 7f646b7fe640 1 -- 172.21.15.22:0/3580389077 >> v1:172.21.15.22:6789/0 conn(0x7f646c06f8e0 legacy=0x7f646c09b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.808+0000 7f646b7fe640 1 -- 172.21.15.22:0/3580389077 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c070640 con 0x7f646c06efc0 2024-09-19T11:22:58.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.808+0000 7fdcfeffd640 1 -- 172.21.15.22:0/3027708711 >> v1:172.21.15.22:6789/0 conn(0x7fdd0006efc0 legacy=0x7fdd0006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.808+0000 7fdcfeffd640 1 -- 172.21.15.22:0/3027708711 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00070640 con 0x7fdd0006fab0 2024-09-19T11:22:58.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7f646b7fe640 1 --2- 172.21.15.22:0/3580389077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c06f3a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f645c009720 tx=0x7f645c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e2eeb241bf0c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7fdcfeffd640 1 --2- 172.21.15.22:0/3027708711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006fab0 0x7fdd00134820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40097f0 tx=0x7fdcf402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719386945284a12c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7fdcfe7fc640 1 -- 172.21.15.22:0/3027708711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf4037070 con 0x7fdd0006fab0 2024-09-19T11:22:58.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7f646a7fc640 1 -- 172.21.15.22:0/3580389077 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f645c037070 con 0x7f646c06efc0 2024-09-19T11:22:58.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7fdcfe7fc640 1 -- 172.21.15.22:0/3027708711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf4032440 con 0x7fdd0006fab0 2024-09-19T11:22:58.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7fdcfe7fc640 1 -- 172.21.15.22:0/3027708711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf4032740 con 0x7fdd0006fab0 2024-09-19T11:22:58.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7f646a7fc640 1 -- 172.21.15.22:0/3580389077 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645c02ad90 con 0x7f646c06efc0 2024-09-19T11:22:58.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.809+0000 7f646a7fc640 1 -- 172.21.15.22:0/3580389077 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f645c032590 con 0x7f646c06efc0 2024-09-19T11:22:58.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7fdd05e16640 1 -- 172.21.15.22:0/3027708711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006fab0 msgr2=0x7fdd00134820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7fdd05e16640 1 --2- 172.21.15.22:0/3027708711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006fab0 0x7fdd00134820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40097f0 tx=0x7fdcf402a1f0 comp rx=0 tx=0).stop 2024-09-19T11:22:58.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7f6471a71640 1 -- 172.21.15.22:0/3580389077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 msgr2=0x7f646c06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7f6471a71640 1 --2- 172.21.15.22:0/3580389077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c06f3a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f645c009720 tx=0x7f645c02a110 comp rx=0 tx=0).stop 2024-09-19T11:22:58.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7f6471a71640 1 -- 172.21.15.22:0/3580389077 shutdown_connections 2024-09-19T11:22:58.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7fdd05e16640 1 -- 172.21.15.22:0/3027708711 shutdown_connections 2024-09-19T11:22:58.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7fdd05e16640 1 --2- 172.21.15.22:0/3027708711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006fab0 0x7fdd00134820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:58.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7f6471a71640 1 --2- 172.21.15.22:0/3580389077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c06f3a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:58.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7f6471a71640 1 -- 172.21.15.22:0/3580389077 >> 172.21.15.22:0/3580389077 conn(0x7f646c0a2460 msgr2=0x7f646c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:58.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7fdd05e16640 1 -- 172.21.15.22:0/3027708711 >> 172.21.15.22:0/3027708711 conn(0x7fdd000a2460 msgr2=0x7fdd0009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:58.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7f6471a71640 1 -- 172.21.15.22:0/3580389077 shutdown_connections 2024-09-19T11:22:58.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7fdd05e16640 1 -- 172.21.15.22:0/3027708711 shutdown_connections 2024-09-19T11:22:58.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7f6471a71640 1 -- 172.21.15.22:0/3580389077 wait complete. 2024-09-19T11:22:58.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.810+0000 7fdd05e16640 1 -- 172.21.15.22:0/3027708711 wait complete. 2024-09-19T11:22:58.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.811+0000 7f6471a71640 1 Processor -- start 2024-09-19T11:22:58.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.811+0000 7fdd05e16640 1 Processor -- start 2024-09-19T11:22:58.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7f6471a71640 1 -- start start 2024-09-19T11:22:58.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7fdd05e16640 1 -- start start 2024-09-19T11:22:58.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7f6471a71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c0903f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7f6471a71640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f646c09f870 con 0x7f646c06efc0 2024-09-19T11:22:58.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7fdd05e16640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006efc0 0x7fdd00127de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7fdd05e16640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00134ee0 con 0x7fdd0006efc0 2024-09-19T11:22:58.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7f646b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c0903f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7f646b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c0903f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60074/0 (socket says 172.21.15.22:60074) 2024-09-19T11:22:58.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7f646b7fe640 1 -- 172.21.15.22:0/687911975 learned_addr learned my addr 172.21.15.22:0/687911975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7fdcff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006efc0 0x7fdd00127de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7fdcff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006efc0 0x7fdd00127de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60086/0 (socket says 172.21.15.22:60086) 2024-09-19T11:22:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.812+0000 7fdcff7fe640 1 -- 172.21.15.22:0/3289538051 learned_addr learned my addr 172.21.15.22:0/3289538051 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7fdcff7fe640 1 -- 172.21.15.22:0/3289538051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00073fe0 con 0x7fdd0006efc0 2024-09-19T11:22:58.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7fdcff7fe640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006efc0 0x7fdd00127de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c480 tx=0x7fdcf000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7fdce7fff640 1 -- 172.21.15.22:0/3289538051 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf0017070 con 0x7fdd0006efc0 2024-09-19T11:22:58.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7fdce7fff640 1 -- 172.21.15.22:0/3289538051 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcf0005200 con 0x7fdd0006efc0 2024-09-19T11:22:58.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7f646b7fe640 1 -- 172.21.15.22:0/687911975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c091c50 con 0x7f646c06efc0 2024-09-19T11:22:58.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7fdce7fff640 1 -- 172.21.15.22:0/3289538051 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdcf00035f0 con 0x7fdd0006efc0 2024-09-19T11:22:58.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd001293a0 con 0x7fdd0006efc0 2024-09-19T11:22:58.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0012a6d0 con 0x7fdd0006efc0 2024-09-19T11:22:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.813+0000 7f646b7fe640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c0903f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f645c009840 tx=0x7f645c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.814+0000 7f6453fff640 1 -- 172.21.15.22:0/687911975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f645c041020 con 0x7f646c06efc0 2024-09-19T11:22:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.814+0000 7f6453fff640 1 -- 172.21.15.22:0/687911975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645c03d550 con 0x7f646c06efc0 2024-09-19T11:22:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.814+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c09b730 con 0x7f646c06efc0 2024-09-19T11:22:58.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.815+0000 7fdce7fff640 1 -- 172.21.15.22:0/3289538051 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdcf001a070 con 0x7fdd0006efc0 2024-09-19T11:22:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.815+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c090b10 con 0x7f646c06efc0 2024-09-19T11:22:58.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.815+0000 7fdce7fff640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdcec03e760 0x7fdcec040c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.815+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd000a2f20 con 0x7fdd0006efc0 2024-09-19T11:22:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.815+0000 7fdce7fff640 1 -- 172.21.15.22:0/3289538051 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fdcf00521e0 con 0x7fdd0006efc0 2024-09-19T11:22:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.815+0000 7f6453fff640 1 -- 172.21.15.22:0/687911975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f645c041020 con 0x7f646c06efc0 2024-09-19T11:22:58.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.815+0000 7fdcfeffd640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdcec03e760 0x7fdcec040c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.816+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c06f3a0 con 0x7f646c06efc0 2024-09-19T11:22:58.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.816+0000 7fdcfeffd640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdcec03e760 0x7fdcec040c20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4034000 tx=0x7fdcf402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.816+0000 7f6453fff640 1 -- 172.21.15.22:0/687911975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f645c044020 con 0x7f646c06efc0 2024-09-19T11:22:58.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.817+0000 7f6453fff640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f645803eaa0 0x7f6458040f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.817+0000 7f646affd640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f645803eaa0 0x7f6458040f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.817+0000 7f646affd640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f645803eaa0 0x7f6458040f60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6460007fd0 tx=0x7f6460007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.826+0000 7f6453fff640 1 -- 172.21.15.22:0/687911975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f645c03ad80 con 0x7f646c06efc0 2024-09-19T11:22:58.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.826+0000 7fdce7fff640 1 -- 172.21.15.22:0/3289538051 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fdcf0011590 con 0x7fdd0006efc0 2024-09-19T11:22:58.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.834+0000 7f6453fff640 1 -- 172.21.15.22:0/687911975 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f645c04b030 con 0x7f646c06efc0 2024-09-19T11:22:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.853+0000 7f6378317640 1 Processor -- start 2024-09-19T11:22:58.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.853+0000 7f6378317640 1 -- start start 2024-09-19T11:22:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.853+0000 7f6378317640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f637009c370 0x7f63700979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.853+0000 7f6378317640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6370097ef0 con 0x7f637009be50 2024-09-19T11:22:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.853+0000 7f6378317640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f63700980c0 con 0x7f637009c370 2024-09-19T11:22:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f63758b2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f637009c370 0x7f63700979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f63760b3640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f637009be50 0x7f63700972a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49206/0 (socket says 172.21.15.22:49206) 2024-09-19T11:22:58.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f63760b3640 1 -- 172.21.15.22:0/1423827432 learned_addr learned my addr 172.21.15.22:0/1423827432 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f1ff78dd640 1 Processor -- start 2024-09-19T11:22:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f1ff78dd640 1 -- start start 2024-09-19T11:22:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f63758b2640 1 -- 172.21.15.22:0/1423827432 >> v1:172.21.15.22:6789/0 conn(0x7f637009be50 legacy=0x7f63700972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f63758b2640 1 -- 172.21.15.22:0/1423827432 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6370135ee0 con 0x7f637009c370 2024-09-19T11:22:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f1ff78dd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ff0096a20 0x7f1ff00972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f1ff78dd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff0096e00 con 0x7f1ff00977e0 2024-09-19T11:22:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f1ff78dd640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff00988a0 con 0x7f1ff0096a20 2024-09-19T11:22:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.854+0000 7f1ff5679640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ff0096a20 0x7f1ff00972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f1ff5679640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1ff0096a20 0x7f1ff00972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60114/0 (socket says 172.21.15.22:60114) 2024-09-19T11:22:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f1ff5679640 1 -- 172.21.15.22:0/1054872714 learned_addr learned my addr 172.21.15.22:0/1054872714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f1ff5679640 1 -- 172.21.15.22:0/1054872714 >> v1:172.21.15.22:6789/0 conn(0x7f1ff00977e0 legacy=0x7f1ff0133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f1ff5679640 1 -- 172.21.15.22:0/1054872714 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff009c0c0 con 0x7f1ff0096a20 2024-09-19T11:22:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f63758b2640 1 --2- 172.21.15.22:0/1423827432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009c370 0x7f63700979b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6360009720 tx=0x7f636002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439d38dfff8e5cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f63750b1640 1 -- 172.21.15.22:0/1423827432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6360037070 con 0x7f637009c370 2024-09-19T11:22:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f63750b1640 1 -- 172.21.15.22:0/1423827432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636002ae20 con 0x7f637009c370 2024-09-19T11:22:58.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.855+0000 7f63750b1640 1 -- 172.21.15.22:0/1423827432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6360032590 con 0x7f637009c370 2024-09-19T11:22:58.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f1ff5679640 1 --2- 172.21.15.22:0/1054872714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff00972a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0097f0 tx=0x7f1fec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f886346e8674a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f1fe7fff640 1 -- 172.21.15.22:0/1054872714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec037070 con 0x7f1ff0096a20 2024-09-19T11:22:58.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f1fe7fff640 1 -- 172.21.15.22:0/1054872714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec032440 con 0x7f1ff0096a20 2024-09-19T11:22:58.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f1fe7fff640 1 -- 172.21.15.22:0/1054872714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec02ae60 con 0x7f1ff0096a20 2024-09-19T11:22:58.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f6378317640 1 -- 172.21.15.22:0/1423827432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009c370 msgr2=0x7f63700979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f6378317640 1 --2- 172.21.15.22:0/1423827432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009c370 0x7f63700979b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6360009720 tx=0x7f636002a1a0 comp rx=0 tx=0).stop 2024-09-19T11:22:58.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f6378317640 1 -- 172.21.15.22:0/1423827432 shutdown_connections 2024-09-19T11:22:58.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f6378317640 1 --2- 172.21.15.22:0/1423827432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009c370 0x7f63700979b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:58.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f6378317640 1 -- 172.21.15.22:0/1423827432 >> 172.21.15.22:0/1423827432 conn(0x7f637009ce60 msgr2=0x7f637009a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:58.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f6378317640 1 -- 172.21.15.22:0/1423827432 shutdown_connections 2024-09-19T11:22:58.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.856+0000 7f6378317640 1 -- 172.21.15.22:0/1423827432 wait complete. 2024-09-19T11:22:58.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f1ff78dd640 1 -- 172.21.15.22:0/1054872714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 msgr2=0x7f1ff00972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:58.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f1ff78dd640 1 --2- 172.21.15.22:0/1054872714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff00972a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0097f0 tx=0x7f1fec02a1e0 comp rx=0 tx=0).stop 2024-09-19T11:22:58.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f1ff78dd640 1 -- 172.21.15.22:0/1054872714 shutdown_connections 2024-09-19T11:22:58.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f1ff78dd640 1 --2- 172.21.15.22:0/1054872714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff00972a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:58.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f1ff78dd640 1 -- 172.21.15.22:0/1054872714 >> 172.21.15.22:0/1054872714 conn(0x7f1ff009ce60 msgr2=0x7f1ff009a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:58.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f1ff78dd640 1 -- 172.21.15.22:0/1054872714 shutdown_connections 2024-09-19T11:22:58.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f1ff78dd640 1 -- 172.21.15.22:0/1054872714 wait complete. 2024-09-19T11:22:58.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f6378317640 1 Processor -- start 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.857+0000 7f6378317640 1 -- start start 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.858+0000 7f6378317640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009be50 0x7f6370090bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.858+0000 7f6378317640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6370135260 con 0x7f637009be50 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.858+0000 7f63760b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009be50 0x7f6370090bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.858+0000 7f63760b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009be50 0x7f6370090bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60128/0 (socket says 172.21.15.22:60128) 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.858+0000 7f63760b3640 1 -- 172.21.15.22:0/1792004332 learned_addr learned my addr 172.21.15.22:0/1792004332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.858+0000 7f63760b3640 1 -- 172.21.15.22:0/1792004332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6370092260 con 0x7f637009be50 2024-09-19T11:22:58.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.858+0000 7f1ff78dd640 1 Processor -- start 2024-09-19T11:22:58.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f63760b3640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009be50 0x7f6370090bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f636c00c480 tx=0x7f636c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f1ff78dd640 1 -- start start 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f6366ffd640 1 -- 172.21.15.22:0/1792004332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f636c017070 con 0x7f637009be50 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f6366ffd640 1 -- 172.21.15.22:0/1792004332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636c005c10 con 0x7f637009be50 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63700962e0 con 0x7f637009be50 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f6366ffd640 1 -- 172.21.15.22:0/1792004332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f636c007070 con 0x7f637009be50 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6370093930 con 0x7f637009be50 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f1ff78dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff01c0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f1ff78dd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff0097f90 con 0x7f1ff0096a20 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.859+0000 7f1ff5679640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff01c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f1ff5679640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff01c0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60142/0 (socket says 172.21.15.22:60142) 2024-09-19T11:22:58.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f1ff5679640 1 -- 172.21.15.22:0/3537940417 learned_addr learned my addr 172.21.15.22:0/3537940417 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:58.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f6366ffd640 1 -- 172.21.15.22:0/1792004332 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f636c01a070 con 0x7f637009be50 2024-09-19T11:22:58.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f6366ffd640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f635403e710 0x7f6354040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f1ff5679640 1 -- 172.21.15.22:0/3537940417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff0095930 con 0x7f1ff0096a20 2024-09-19T11:22:58.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63701357a0 con 0x7f637009be50 2024-09-19T11:22:58.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f63758b2640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f635403e710 0x7f6354040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f6366ffd640 1 -- 172.21.15.22:0/1792004332 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f636c050d00 con 0x7f637009be50 2024-09-19T11:22:58.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.860+0000 7f1ff5679640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff01c0990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fec035040 tx=0x7f1fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.861+0000 7f63758b2640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f635403e710 0x7f6354040bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6360034000 tx=0x7f636002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.865+0000 7f1fe5ffb640 1 -- 172.21.15.22:0/3537940417 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec041020 con 0x7f1ff0096a20 2024-09-19T11:22:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.865+0000 7f1fe5ffb640 1 -- 172.21.15.22:0/3537940417 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec030a10 con 0x7f1ff0096a20 2024-09-19T11:22:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.865+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff01abf80 con 0x7f1ff0096a20 2024-09-19T11:22:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.865+0000 7f1fe5ffb640 1 -- 172.21.15.22:0/3537940417 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fec037040 con 0x7f1ff0096a20 2024-09-19T11:22:58.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.865+0000 7f6366ffd640 1 -- 172.21.15.22:0/1792004332 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f636c012400 con 0x7f637009be50 2024-09-19T11:22:58.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.865+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff01ac2b0 con 0x7f1ff0096a20 2024-09-19T11:22:58.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.866+0000 7f1fe5ffb640 1 -- 172.21.15.22:0/3537940417 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1fec044020 con 0x7f1ff0096a20 2024-09-19T11:22:58.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.867+0000 7f1fe5ffb640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1fd403e710 0x7f1fd4040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:58.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.867+0000 7f1fe5ffb640 1 -- 172.21.15.22:0/3537940417 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f1fec073e60 con 0x7f1ff0096a20 2024-09-19T11:22:58.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.867+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff009eaa0 con 0x7f1ff0096a20 2024-09-19T11:22:58.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.867+0000 7f1ff4e78640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1fd403e710 0x7f1fd4040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:58.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.868+0000 7f1ff4e78640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1fd403e710 0x7f1fd4040bd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0001600 tx=0x7f1fe00079f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:58.872+0000 7f1fe5ffb640 1 -- 172.21.15.22:0/3537940417 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f1ff009eaa0 con 0x7f1ff0096a20 2024-09-19T11:22:59.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.025+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f646c073fe0 con 0x7f646c06efc0 2024-09-19T11:22:59.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.025+0000 7f6453fff640 1 -- 172.21.15.22:0/687911975 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f645c03c380 con 0x7f646c06efc0 2024-09-19T11:22:59.027 INFO:teuthology.orchestra.run.smithi022.stdout:38654705669 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f645803eaa0 msgr2=0x7f6458040f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f645803eaa0 0x7f6458040f60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6460007fd0 tx=0x7f6460007aa0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 msgr2=0x7f646c0903f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c0903f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f645c009840 tx=0x7f645c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 shutdown_connections 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f645803eaa0 0x7f6458040f60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 --2- 172.21.15.22:0/687911975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f646c06efc0 0x7f646c0903f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 >> 172.21.15.22:0/687911975 conn(0x7f646c0a2460 msgr2=0x7f646c09cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 shutdown_connections 2024-09-19T11:22:59.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.028+0000 7f6471a71640 1 -- 172.21.15.22:0/687911975 wait complete. 2024-09-19T11:22:59.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2024-09-19T11:22:59.041 DEBUG:teuthology.parallel:result is None 2024-09-19T11:22:59.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.047+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fdd0012a3d0 con 0x7fdd0006efc0 2024-09-19T11:22:59.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.047+0000 7fdce7fff640 1 -- 172.21.15.22:0/3289538051 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdcf0040260 con 0x7fdd0006efc0 2024-09-19T11:22:59.049 INFO:teuthology.orchestra.run.smithi022.stdout:42949672965 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdcec03e760 msgr2=0x7fdcec040c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdcec03e760 0x7fdcec040c20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4034000 tx=0x7fdcf402e000 comp rx=0 tx=0).stop 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006efc0 msgr2=0x7fdd00127de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006efc0 0x7fdd00127de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c480 tx=0x7fdcf000c940 comp rx=0 tx=0).stop 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 shutdown_connections 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdcec03e760 0x7fdcec040c20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 --2- 172.21.15.22:0/3289538051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd0006efc0 0x7fdd00127de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 >> 172.21.15.22:0/3289538051 conn(0x7fdd000a2460 msgr2=0x7fdd0009cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 shutdown_connections 2024-09-19T11:22:59.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.049+0000 7fdd05e16640 1 -- 172.21.15.22:0/3289538051 wait complete. 2024-09-19T11:22:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.056+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f63700744a0 con 0x7f637009be50 2024-09-19T11:22:59.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.056+0000 7f6366ffd640 1 -- 172.21.15.22:0/1792004332 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f636c012780 con 0x7f637009be50 2024-09-19T11:22:59.057 INFO:teuthology.orchestra.run.smithi022.stdout:42949672965 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f635403e710 msgr2=0x7f6354040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f635403e710 0x7f6354040bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6360034000 tx=0x7f636002e000 comp rx=0 tx=0).stop 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009be50 msgr2=0x7f6370090bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009be50 0x7f6370090bc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f636c00c480 tx=0x7f636c00c940 comp rx=0 tx=0).stop 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 shutdown_connections 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f635403e710 0x7f6354040bd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 --2- 172.21.15.22:0/1792004332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f637009be50 0x7f6370090bc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 >> 172.21.15.22:0/1792004332 conn(0x7f637009ce60 msgr2=0x7f6370133ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 shutdown_connections 2024-09-19T11:22:59.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.058+0000 7f6378317640 1 -- 172.21.15.22:0/1792004332 wait complete. 2024-09-19T11:22:59.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.3 2024-09-19T11:22:59.062 DEBUG:teuthology.parallel:result is None 2024-09-19T11:22:59.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672965 got 42949672965 for osd.1 2024-09-19T11:22:59.070 DEBUG:teuthology.parallel:result is None 2024-09-19T11:22:59.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.075+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1ff00972a0 con 0x7f1ff0096a20 2024-09-19T11:22:59.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.075+0000 7f1fe5ffb640 1 -- 172.21.15.22:0/3537940417 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1fec04b030 con 0x7f1ff0096a20 2024-09-19T11:22:59.077 INFO:teuthology.orchestra.run.smithi022.stdout:38654705669 2024-09-19T11:22:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.077+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1fd403e710 msgr2=0x7f1fd4040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.077+0000 7f1ff78dd640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1fd403e710 0x7f1fd4040bd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0001600 tx=0x7f1fe00079f0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.077+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 msgr2=0x7f1ff01c0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.078+0000 7f1ff78dd640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff01c0990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1fec035040 tx=0x7f1fec0046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.078+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 shutdown_connections 2024-09-19T11:22:59.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.078+0000 7f1ff78dd640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1fd403e710 0x7f1fd4040bd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.078+0000 7f1ff78dd640 1 --2- 172.21.15.22:0/3537940417 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1ff0096a20 0x7f1ff01c0990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.078+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 >> 172.21.15.22:0/3537940417 conn(0x7f1ff009ce60 msgr2=0x7f1ff0071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.078+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 shutdown_connections 2024-09-19T11:22:59.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.078+0000 7f1ff78dd640 1 -- 172.21.15.22:0/3537940417 wait complete. 2024-09-19T11:22:59.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-09-19T11:22:59.089 DEBUG:teuthology.parallel:result is None 2024-09-19T11:22:59.089 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T11:22:59.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.229+0000 7fc4d43d3640 1 Processor -- start 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.229+0000 7fc4d43d3640 1 -- start start 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d43d3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc4cc06fab0 0x7fc4cc134820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d43d3640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4cc070330 con 0x7fc4cc06efc0 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d43d3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc4cc09f310 con 0x7fc4cc06fab0 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d216f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc4cc06efc0 0x7fc4cc06f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49216/0 (socket says 172.21.15.22:49216) 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d216f640 1 -- 172.21.15.22:0/1876536872 learned_addr learned my addr 172.21.15.22:0/1876536872 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d196e640 1 --2- 172.21.15.22:0/1876536872 >> v2:172.21.15.22:3300/0 conn(0x7fc4cc06fab0 0x7fc4cc134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d116d640 1 -- 172.21.15.22:0/1876536872 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488674269 0 0) 0x7fc4cc070330 con 0x7fc4cc06efc0 2024-09-19T11:22:59.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d116d640 1 -- 172.21.15.22:0/1876536872 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4b8003520 con 0x7fc4cc06efc0 2024-09-19T11:22:59.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d196e640 1 -- 172.21.15.22:0/1876536872 >> v1:172.21.15.22:6789/0 conn(0x7fc4cc06efc0 legacy=0x7fc4cc06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.230+0000 7fc4d196e640 1 -- 172.21.15.22:0/1876536872 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc070640 con 0x7fc4cc06fab0 2024-09-19T11:22:59.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.231+0000 7fc4d196e640 1 --2- 172.21.15.22:0/1876536872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06fab0 0x7fc4cc134820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009720 tx=0x7fc4bc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85e0931a9e6ace2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.231+0000 7fc4d116d640 1 -- 172.21.15.22:0/1876536872 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4bc037070 con 0x7fc4cc06fab0 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.231+0000 7fc4d116d640 1 -- 172.21.15.22:0/1876536872 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4bc02ad90 con 0x7fc4cc06fab0 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.231+0000 7fc4d116d640 1 -- 172.21.15.22:0/1876536872 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4bc032570 con 0x7fc4cc06fab0 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.232+0000 7fc4d43d3640 1 -- 172.21.15.22:0/1876536872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06fab0 msgr2=0x7fc4cc134820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.232+0000 7fc4d43d3640 1 --2- 172.21.15.22:0/1876536872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06fab0 0x7fc4cc134820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009720 tx=0x7fc4bc02a110 comp rx=0 tx=0).stop 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.232+0000 7fc4d43d3640 1 -- 172.21.15.22:0/1876536872 shutdown_connections 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.232+0000 7fc4d43d3640 1 --2- 172.21.15.22:0/1876536872 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06fab0 0x7fc4cc134820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.232+0000 7fc4d43d3640 1 -- 172.21.15.22:0/1876536872 >> 172.21.15.22:0/1876536872 conn(0x7fc4cc0a2460 msgr2=0x7fc4cc09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.232+0000 7fc4d43d3640 1 -- 172.21.15.22:0/1876536872 shutdown_connections 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.232+0000 7fc4d43d3640 1 -- 172.21.15.22:0/1876536872 wait complete. 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d43d3640 1 Processor -- start 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d43d3640 1 -- start start 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d43d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06efc0 0x7fc4cc09b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d43d3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc134ee0 con 0x7fc4cc06efc0 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d216f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06efc0 0x7fc4cc09b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d216f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06efc0 0x7fc4cc09b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60158/0 (socket says 172.21.15.22:60158) 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d216f640 1 -- 172.21.15.22:0/3246144571 learned_addr learned my addr 172.21.15.22:0/3246144571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d216f640 1 -- 172.21.15.22:0/3246144571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc129860 con 0x7fc4cc06efc0 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.233+0000 7fc4d216f640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06efc0 0x7fc4cc09b4a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800c480 tx=0x7fc4c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.234+0000 7fc4c2ffd640 1 -- 172.21.15.22:0/3246144571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4c8017070 con 0x7fc4cc06efc0 2024-09-19T11:22:59.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.234+0000 7fc4c2ffd640 1 -- 172.21.15.22:0/3246144571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c8005cd0 con 0x7fc4cc06efc0 2024-09-19T11:22:59.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.234+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc12b250 con 0x7fc4cc06efc0 2024-09-19T11:22:59.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.234+0000 7fc4c2ffd640 1 -- 172.21.15.22:0/3246144571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4c8007200 con 0x7fc4cc06efc0 2024-09-19T11:22:59.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.234+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc09b9e0 con 0x7fc4cc06efc0 2024-09-19T11:22:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.235+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc06f3a0 con 0x7fc4cc06efc0 2024-09-19T11:22:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.235+0000 7fc4c2ffd640 1 -- 172.21.15.22:0/3246144571 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc4c801a070 con 0x7fc4cc06efc0 2024-09-19T11:22:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.235+0000 7fc4c2ffd640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc4b003e710 0x7fc4b0040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.235+0000 7fc4c2ffd640 1 -- 172.21.15.22:0/3246144571 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fc4c8052040 con 0x7fc4cc06efc0 2024-09-19T11:22:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.235+0000 7fc4d196e640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc4b003e710 0x7fc4b0040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.236+0000 7fc4d196e640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc4b003e710 0x7fc4b0040bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc034000 tx=0x7fc4bc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.240+0000 7fc4c2ffd640 1 -- 172.21.15.22:0/3246144571 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fc4c801bdc0 con 0x7fc4cc06efc0 2024-09-19T11:22:59.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.423+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4cc12a8a0 con 0x7fc4b003e710 2024-09-19T11:22:59.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.425+0000 7fc4c2ffd640 1 -- 172.21.15.22:0/3246144571 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32168 (secure 0 0 0) 0x7fc4bc038040 con 0x7fc4b003e710 2024-09-19T11:22:59.426 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:59.426 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:22:59.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc4b003e710 msgr2=0x7fc4b0040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc4b003e710 0x7fc4b0040bd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc034000 tx=0x7fc4bc02e000 comp rx=0 tx=0).stop 2024-09-19T11:22:59.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06efc0 msgr2=0x7fc4cc09b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06efc0 0x7fc4cc09b4a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800c480 tx=0x7fc4c800c940 comp rx=0 tx=0).stop 2024-09-19T11:22:59.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 shutdown_connections 2024-09-19T11:22:59.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc4b003e710 0x7fc4b0040bd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 --2- 172.21.15.22:0/3246144571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc4cc06efc0 0x7fc4cc09b4a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.426+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 >> 172.21.15.22:0/3246144571 conn(0x7fc4cc0a2460 msgr2=0x7fc4cc12f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.427+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 shutdown_connections 2024-09-19T11:22:59.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.427+0000 7fc4d43d3640 1 -- 172.21.15.22:0/3246144571 wait complete. 2024-09-19T11:22:59.438 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-19T11:22:58.782261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":143132,"kb_used_data":1712,"kb_used_omap":6,"kb_used_meta":141305,"kb_avail":377344228,"statfs":{"total":386547056640,"available":386400489472,"internally_reserved":0,"allocated":1753088,"data_stored":1361998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":144697110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.976806"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.818050+0000","last_change":"2024-09-19T11:22:55.818050+0000","last_active":"2024-09-19T11:22:55.818050+0000","last_peered":"2024-09-19T11:22:55.818050+0000","last_clean":"2024-09-19T11:22:55.818050+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:22:55.818050+0000","last_undegraded":"2024-09-19T11:22:55.818050+0000","last_fullsized":"2024-09-19T11:22:55.818050+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.933095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.813197+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:22:55.813197+0000","last_peered":"2024-09-19T11:22:55.813197+0000","last_clean":"2024-09-19T11:22:55.813197+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:22:55.813197+0000","last_undegraded":"2024-09-19T11:22:55.813197+0000","last_fullsized":"2024-09-19T11:22:55.813197+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:08:18.682837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.818302+0000","last_change":"2024-09-19T11:22:55.818302+0000","last_active":"2024-09-19T11:22:55.818302+0000","last_peered":"2024-09-19T11:22:55.818302+0000","last_clean":"2024-09-19T11:22:55.818302+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:22:55.818302+0000","last_undegraded":"2024-09-19T11:22:55.818302+0000","last_fullsized":"2024-09-19T11:22:55.818302+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:51.911098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.813132+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:22:55.813132+0000","last_peered":"2024-09-19T11:22:55.813132+0000","last_clean":"2024-09-19T11:22:55.813132+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:22:55.813132+0000","last_undegraded":"2024-09-19T11:22:55.813132+0000","last_fullsized":"2024-09-19T11:22:55.813132+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:41.618944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.818733+0000","last_change":"2024-09-19T11:22:55.817947+0000","last_active":"2024-09-19T11:22:55.818733+0000","last_peered":"2024-09-19T11:22:55.818733+0000","last_clean":"2024-09-19T11:22:55.818733+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:22:55.818733+0000","last_undegraded":"2024-09-19T11:22:55.818733+0000","last_fullsized":"2024-09-19T11:22:55.818733+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:52.495235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.886122+0000","last_change":"2024-09-19T11:22:55.886390+0000","last_active":"2024-09-19T11:22:55.886122+0000","last_peered":"2024-09-19T11:22:55.886122+0000","last_clean":"2024-09-19T11:22:55.886122+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:22:55.886122+0000","last_undegraded":"2024-09-19T11:22:55.886122+0000","last_fullsized":"2024-09-19T11:22:55.886122+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:42.873153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.817444+0000","last_change":"2024-09-19T11:22:55.817444+0000","last_active":"2024-09-19T11:22:55.817444+0000","last_peered":"2024-09-19T11:22:55.817444+0000","last_clean":"2024-09-19T11:22:55.817444+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:22:55.817444+0000","last_undegraded":"2024-09-19T11:22:55.817444+0000","last_fullsized":"2024-09-19T11:22:55.817444+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:39.661543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":21,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.817718+0000","last_change":"2024-09-19T11:22:55.817718+0000","last_active":"2024-09-19T11:22:55.817718+0000","last_peered":"2024-09-19T11:22:55.817718+0000","last_clean":"2024-09-19T11:22:55.817718+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:22:55.817718+0000","last_undegraded":"2024-09-19T11:22:55.817718+0000","last_fullsized":"2024-09-19T11:22:55.817718+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:33.890847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'72","reported_seq":111,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:56.076183+0000","last_change":"2024-09-19T11:22:46.832286+0000","last_active":"2024-09-19T11:22:56.076183+0000","last_peered":"2024-09-19T11:22:56.076183+0000","last_clean":"2024-09-19T11:22:56.076183+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:22:56.076183+0000","last_undegraded":"2024-09-19T11:22:56.076183+0000","last_fullsized":"2024-09-19T11:22:56.076183+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_clean_scrub_stamp":"2024-09-19T11:22:45.784999+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:24.575130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35496,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":143360,"data_stored":45325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":733184,"data_stored":635674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36068,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335772,"statfs":{"total":96636764160,"available":96599830528,"internally_reserved":0,"allocated":733184,"data_stored":635674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35496,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":143360,"data_stored":45325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:22:59.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:22:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe90341f640 1 Processor -- start 2024-09-19T11:22:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe90341f640 1 -- start start 2024-09-19T11:22:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe90341f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe8fc12c0b0 0x7fe8fc12c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe90341f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8fc099e20 con 0x7fe8fc12c9d0 2024-09-19T11:22:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe90341f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe8fc0715a0 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe9011bb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe8fc12c0b0 0x7fe8fc12c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe9009ba640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe8fc12c9d0 0x7fe8fc097100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49230/0 (socket says 172.21.15.22:49230) 2024-09-19T11:22:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe9011bb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe8fc12c0b0 0x7fe8fc12c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60164/0 (socket says 172.21.15.22:60164) 2024-09-19T11:22:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.539+0000 7fe9009ba640 1 -- 172.21.15.22:0/1009324386 learned_addr learned my addr 172.21.15.22:0/1009324386 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.540+0000 7fe9011bb640 1 -- 172.21.15.22:0/1009324386 >> v1:172.21.15.22:6789/0 conn(0x7fe8fc12c9d0 legacy=0x7fe8fc097100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.540+0000 7fe9011bb640 1 -- 172.21.15.22:0/1009324386 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc0705b0 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.540+0000 7fe9011bb640 1 --2- 172.21.15.22:0/1009324386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc12c490 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009750 tx=0x7fe8ec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adabadad0d827a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.540+0000 7fe8fbfff640 1 -- 172.21.15.22:0/1009324386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec037070 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.540+0000 7fe8fbfff640 1 -- 172.21.15.22:0/1009324386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec02ad80 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.540+0000 7fe8fbfff640 1 -- 172.21.15.22:0/1009324386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec032590 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.541+0000 7fe90341f640 1 -- 172.21.15.22:0/1009324386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 msgr2=0x7fe8fc12c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.541+0000 7fe90341f640 1 --2- 172.21.15.22:0/1009324386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc12c490 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009750 tx=0x7fe8ec02a140 comp rx=0 tx=0).stop 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.541+0000 7fe90341f640 1 -- 172.21.15.22:0/1009324386 shutdown_connections 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.541+0000 7fe90341f640 1 --2- 172.21.15.22:0/1009324386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc12c490 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.541+0000 7fe90341f640 1 -- 172.21.15.22:0/1009324386 >> 172.21.15.22:0/1009324386 conn(0x7fe8fc09abc0 msgr2=0x7fe8fc098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.541+0000 7fe90341f640 1 -- 172.21.15.22:0/1009324386 shutdown_connections 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.541+0000 7fe90341f640 1 -- 172.21.15.22:0/1009324386 wait complete. 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.542+0000 7fe90341f640 1 Processor -- start 2024-09-19T11:22:59.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.542+0000 7fe90341f640 1 -- start start 2024-09-19T11:22:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.542+0000 7fe90341f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc128800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.542+0000 7fe90341f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc12ce60 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.542+0000 7fe9011bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc128800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.542+0000 7fe9011bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc128800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60172/0 (socket says 172.21.15.22:60172) 2024-09-19T11:22:59.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.542+0000 7fe9011bb640 1 -- 172.21.15.22:0/2420705348 learned_addr learned my addr 172.21.15.22:0/2420705348 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.543+0000 7fe9011bb640 1 -- 172.21.15.22:0/2420705348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc06e220 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.543+0000 7fe9011bb640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc128800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009870 tx=0x7fe8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.543+0000 7fe8f9ffb640 1 -- 172.21.15.22:0/2420705348 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec041070 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.543+0000 7fe8f9ffb640 1 -- 172.21.15.22:0/2420705348 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec03c590 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.543+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc097100 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.543+0000 7fe8f9ffb640 1 -- 172.21.15.22:0/2420705348 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8ec037040 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.543+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc08ea20 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.544+0000 7fe8f9ffb640 1 -- 172.21.15.22:0/2420705348 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe8ec044070 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.544+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc097df0 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.544+0000 7fe8f9ffb640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe8e803e710 0x7fe8e8040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.544+0000 7fe8f9ffb640 1 -- 172.21.15.22:0/2420705348 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fe8ec073660 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.545+0000 7fe9009ba640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe8e803e710 0x7fe8e8040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.545+0000 7fe9009ba640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe8e803e710 0x7fe8e8040bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0007fd0 tx=0x7fe8f0007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.549+0000 7fe8f9ffb640 1 -- 172.21.15.22:0/2420705348 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fe8ec03cdb0 con 0x7fe8fc12c0b0 2024-09-19T11:22:59.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.731+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8fc06e590 con 0x7fe8e803e710 2024-09-19T11:22:59.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.732+0000 7fe8f9ffb640 1 -- 172.21.15.22:0/2420705348 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+32168 (secure 0 0 0) 0x7fe8fc06e590 con 0x7fe8e803e710 2024-09-19T11:22:59.734 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:22:59.735 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:22:59.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe8e803e710 msgr2=0x7fe8e8040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe8e803e710 0x7fe8e8040bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0007fd0 tx=0x7fe8f0007aa0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 msgr2=0x7fe8fc128800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc128800 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009870 tx=0x7fe8ec0046c0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 shutdown_connections 2024-09-19T11:22:59.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe8e803e710 0x7fe8e8040bd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 --2- 172.21.15.22:0/2420705348 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8fc12c0b0 0x7fe8fc128800 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 >> 172.21.15.22:0/2420705348 conn(0x7fe8fc09abc0 msgr2=0x7fe8fc071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 shutdown_connections 2024-09-19T11:22:59.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.735+0000 7fe90341f640 1 -- 172.21.15.22:0/2420705348 wait complete. 2024-09-19T11:22:59.747 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-19T11:22:58.782261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":143132,"kb_used_data":1712,"kb_used_omap":6,"kb_used_meta":141305,"kb_avail":377344228,"statfs":{"total":386547056640,"available":386400489472,"internally_reserved":0,"allocated":1753088,"data_stored":1361998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":144697110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.976806"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.818050+0000","last_change":"2024-09-19T11:22:55.818050+0000","last_active":"2024-09-19T11:22:55.818050+0000","last_peered":"2024-09-19T11:22:55.818050+0000","last_clean":"2024-09-19T11:22:55.818050+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:22:55.818050+0000","last_undegraded":"2024-09-19T11:22:55.818050+0000","last_fullsized":"2024-09-19T11:22:55.818050+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.933095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.813197+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:22:55.813197+0000","last_peered":"2024-09-19T11:22:55.813197+0000","last_clean":"2024-09-19T11:22:55.813197+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:22:55.813197+0000","last_undegraded":"2024-09-19T11:22:55.813197+0000","last_fullsized":"2024-09-19T11:22:55.813197+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:08:18.682837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.818302+0000","last_change":"2024-09-19T11:22:55.818302+0000","last_active":"2024-09-19T11:22:55.818302+0000","last_peered":"2024-09-19T11:22:55.818302+0000","last_clean":"2024-09-19T11:22:55.818302+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:22:55.818302+0000","last_undegraded":"2024-09-19T11:22:55.818302+0000","last_fullsized":"2024-09-19T11:22:55.818302+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:51.911098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.813132+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:22:55.813132+0000","last_peered":"2024-09-19T11:22:55.813132+0000","last_clean":"2024-09-19T11:22:55.813132+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:22:55.813132+0000","last_undegraded":"2024-09-19T11:22:55.813132+0000","last_fullsized":"2024-09-19T11:22:55.813132+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:41.618944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.818733+0000","last_change":"2024-09-19T11:22:55.817947+0000","last_active":"2024-09-19T11:22:55.818733+0000","last_peered":"2024-09-19T11:22:55.818733+0000","last_clean":"2024-09-19T11:22:55.818733+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:22:55.818733+0000","last_undegraded":"2024-09-19T11:22:55.818733+0000","last_fullsized":"2024-09-19T11:22:55.818733+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:52.495235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.886122+0000","last_change":"2024-09-19T11:22:55.886390+0000","last_active":"2024-09-19T11:22:55.886122+0000","last_peered":"2024-09-19T11:22:55.886122+0000","last_clean":"2024-09-19T11:22:55.886122+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:22:55.886122+0000","last_undegraded":"2024-09-19T11:22:55.886122+0000","last_fullsized":"2024-09-19T11:22:55.886122+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:42.873153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.817444+0000","last_change":"2024-09-19T11:22:55.817444+0000","last_active":"2024-09-19T11:22:55.817444+0000","last_peered":"2024-09-19T11:22:55.817444+0000","last_clean":"2024-09-19T11:22:55.817444+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:22:55.817444+0000","last_undegraded":"2024-09-19T11:22:55.817444+0000","last_fullsized":"2024-09-19T11:22:55.817444+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:39.661543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":21,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:55.817718+0000","last_change":"2024-09-19T11:22:55.817718+0000","last_active":"2024-09-19T11:22:55.817718+0000","last_peered":"2024-09-19T11:22:55.817718+0000","last_clean":"2024-09-19T11:22:55.817718+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:22:55.817718+0000","last_undegraded":"2024-09-19T11:22:55.817718+0000","last_fullsized":"2024-09-19T11:22:55.817718+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:33.890847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'72","reported_seq":111,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-09-19T11:22:56.076183+0000","last_change":"2024-09-19T11:22:46.832286+0000","last_active":"2024-09-19T11:22:56.076183+0000","last_peered":"2024-09-19T11:22:56.076183+0000","last_clean":"2024-09-19T11:22:56.076183+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:22:56.076183+0000","last_undegraded":"2024-09-19T11:22:56.076183+0000","last_fullsized":"2024-09-19T11:22:56.076183+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_clean_scrub_stamp":"2024-09-19T11:22:45.784999+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:24.575130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672965,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35496,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":143360,"data_stored":45325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672965,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":733184,"data_stored":635674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36068,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335772,"statfs":{"total":96636764160,"available":96599830528,"internally_reserved":0,"allocated":733184,"data_stored":635674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35496,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336344,"statfs":{"total":96636764160,"available":96600416256,"internally_reserved":0,"allocated":143360,"data_stored":45325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:22:59.748 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T11:22:59.748 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T11:22:59.748 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-19T11:22:59.748 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.846+0000 7fa9164fe640 1 Processor -- start 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.846+0000 7fa9164fe640 1 -- start start 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa9164fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa91006fab0 0x7fa910134820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa9164fe640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa910070330 con 0x7fa91006efc0 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa9164fe640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fa91009f310 con 0x7fa91006fab0 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa90f7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fa91006fab0 0x7fa910134820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa90ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fa91006efc0 0x7fa91006f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:49234/0 (socket says 172.21.15.22:49234) 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa90ffff640 1 -- 172.21.15.22:0/1133360745 learned_addr learned my addr 172.21.15.22:0/1133360745 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa90f7fe640 1 -- 172.21.15.22:0/1133360745 >> v1:172.21.15.22:6789/0 conn(0x7fa91006efc0 legacy=0x7fa91006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.847+0000 7fa90f7fe640 1 -- 172.21.15.22:0/1133360745 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa910070640 con 0x7fa91006fab0 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.848+0000 7fa90f7fe640 1 --2- 172.21.15.22:0/1133360745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006fab0 0x7fa910134820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa904009720 tx=0x7fa90402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a52857df62bc5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.848+0000 7fa90effd640 1 -- 172.21.15.22:0/1133360745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa904037070 con 0x7fa91006fab0 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.848+0000 7fa90effd640 1 -- 172.21.15.22:0/1133360745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90402ad90 con 0x7fa91006fab0 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.848+0000 7fa90effd640 1 -- 172.21.15.22:0/1133360745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa904032590 con 0x7fa91006fab0 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.849+0000 7fa9164fe640 1 -- 172.21.15.22:0/1133360745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006fab0 msgr2=0x7fa910134820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.849+0000 7fa9164fe640 1 --2- 172.21.15.22:0/1133360745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006fab0 0x7fa910134820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa904009720 tx=0x7fa90402a110 comp rx=0 tx=0).stop 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.849+0000 7fa9164fe640 1 -- 172.21.15.22:0/1133360745 shutdown_connections 2024-09-19T11:22:59.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.849+0000 7fa9164fe640 1 --2- 172.21.15.22:0/1133360745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006fab0 0x7fa910134820 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:22:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.849+0000 7fa9164fe640 1 -- 172.21.15.22:0/1133360745 >> 172.21.15.22:0/1133360745 conn(0x7fa9100a2460 msgr2=0x7fa91009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:22:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.849+0000 7fa9164fe640 1 -- 172.21.15.22:0/1133360745 shutdown_connections 2024-09-19T11:22:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.849+0000 7fa9164fe640 1 -- 172.21.15.22:0/1133360745 wait complete. 2024-09-19T11:22:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.850+0000 7fa9164fe640 1 Processor -- start 2024-09-19T11:22:59.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.850+0000 7fa9164fe640 1 -- start start 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.850+0000 7fa9164fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006efc0 0x7fa91012bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.850+0000 7fa9164fe640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa910134ee0 con 0x7fa91006efc0 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.850+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006efc0 0x7fa91012bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.850+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006efc0 0x7fa91012bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60200/0 (socket says 172.21.15.22:60200) 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.850+0000 7fa90ffff640 1 -- 172.21.15.22:0/1088229333 learned_addr learned my addr 172.21.15.22:0/1088229333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.851+0000 7fa90ffff640 1 -- 172.21.15.22:0/1088229333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9101298f0 con 0x7fa91006efc0 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.851+0000 7fa90ffff640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006efc0 0x7fa91012bb50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c480 tx=0x7fa90000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.851+0000 7fa90cff9640 1 -- 172.21.15.22:0/1088229333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa900017070 con 0x7fa91006efc0 2024-09-19T11:22:59.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.851+0000 7fa90cff9640 1 -- 172.21.15.22:0/1088229333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa900005c10 con 0x7fa91006efc0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.851+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91012b5d0 con 0x7fa91006efc0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.851+0000 7fa90cff9640 1 -- 172.21.15.22:0/1088229333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa900007070 con 0x7fa91006efc0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.851+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91012c090 con 0x7fa91006efc0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.852+0000 7fa90cff9640 1 -- 172.21.15.22:0/1088229333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa90001a070 con 0x7fa91006efc0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.852+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91006f3a0 con 0x7fa91006efc0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.852+0000 7fa90cff9640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa8fc03e760 0x7fa8fc040c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.852+0000 7fa90cff9640 1 -- 172.21.15.22:0/1088229333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7fa900052040 con 0x7fa91006efc0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.852+0000 7fa90f7fe640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa8fc03e760 0x7fa8fc040c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:22:59.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.853+0000 7fa90f7fe640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa8fc03e760 0x7fa8fc040c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa904034000 tx=0x7fa90402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:22:59.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:22:59.857+0000 7fa90cff9640 1 -- 172.21.15.22:0/1088229333 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fa90001ac60 con 0x7fa91006efc0 2024-09-19T11:23:00.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.084+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa91012a4f0 con 0x7fa91006efc0 2024-09-19T11:23:00.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.085+0000 7fa90cff9640 1 -- 172.21.15.22:0/1088229333 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa900050e50 con 0x7fa91006efc0 2024-09-19T11:23:00.086 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:23:00.086 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T11:23:00.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.086+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa8fc03e760 msgr2=0x7fa8fc040c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.086+0000 7fa9164fe640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa8fc03e760 0x7fa8fc040c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa904034000 tx=0x7fa90402e000 comp rx=0 tx=0).stop 2024-09-19T11:23:00.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006efc0 msgr2=0x7fa91012bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006efc0 0x7fa91012bb50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa90000c480 tx=0x7fa90000c940 comp rx=0 tx=0).stop 2024-09-19T11:23:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 shutdown_connections 2024-09-19T11:23:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa8fc03e760 0x7fa8fc040c20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 --2- 172.21.15.22:0/1088229333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91006efc0 0x7fa91012bb50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 >> 172.21.15.22:0/1088229333 conn(0x7fa9100a2460 msgr2=0x7fa91012f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 shutdown_connections 2024-09-19T11:23:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:23:00.087+0000 7fa9164fe640 1 -- 172.21.15.22:0/1088229333 wait complete. 2024-09-19T11:23:00.099 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-19T11:23:00.099 INFO:teuthology.run_tasks:Running task rgw... 2024-09-19T11:23:00.113 DEBUG:tasks.rgw:config is {'client.0': None} 2024-09-19T11:23:00.113 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-09-19T11:23:00.113 INFO:tasks.rgw:Creating data pools 2024-09-19T11:23:00.113 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-09-19T11:23:00.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph osd erasure-code-profile set client.0 k=2 m=1 crush-failure-domain=osd --cluster ceph 2024-09-19T11:23:00.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.227+0000 7f8655552640 1 Processor -- start 2024-09-19T11:23:00.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.227+0000 7f8655552640 1 -- start start 2024-09-19T11:23:00.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f8655552640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f865011bb10 0x7f865011df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:00.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f8655552640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865011e4d0 con 0x7f8650117be0 2024-09-19T11:23:00.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f8655552640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f865011f6b0 con 0x7f865011bb10 2024-09-19T11:23:00.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f864e7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f865011bb10 0x7f865011df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:00.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f864effd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f8650117be0 0x7f8650117fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48968/0 (socket says 172.21.15.28:48968) 2024-09-19T11:23:00.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f864effd640 1 -- 172.21.15.28:0/847968848 learned_addr learned my addr 172.21.15.28:0/847968848 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:00.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f864e7fc640 1 -- 172.21.15.28:0/847968848 >> v1:172.21.15.22:6789/0 conn(0x7f8650117be0 legacy=0x7f8650117fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f864e7fc640 1 -- 172.21.15.28:0/847968848 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865011f830 con 0x7f865011bb10 2024-09-19T11:23:00.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.228+0000 7f864dffb640 1 -- 172.21.15.28:0/847968848 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462821884 0 0) 0x7f865011e4d0 con 0x7f8650117be0 2024-09-19T11:23:00.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.229+0000 7f864e7fc640 1 --2- 172.21.15.28:0/847968848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865011bb10 0x7f865011df00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8644009720 tx=0x7f864402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb6a38b0c7f1cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:00.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.229+0000 7f864dffb640 1 -- 172.21.15.28:0/847968848 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8644037070 con 0x7f865011bb10 2024-09-19T11:23:00.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.229+0000 7f864dffb640 1 -- 172.21.15.28:0/847968848 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864402ad90 con 0x7f865011bb10 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.229+0000 7f864dffb640 1 -- 172.21.15.28:0/847968848 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8644032590 con 0x7f865011bb10 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.230+0000 7f8655552640 1 -- 172.21.15.28:0/847968848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865011bb10 msgr2=0x7f865011df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.230+0000 7f8655552640 1 --2- 172.21.15.28:0/847968848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865011bb10 0x7f865011df00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8644009720 tx=0x7f864402a110 comp rx=0 tx=0).stop 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.230+0000 7f8655552640 1 -- 172.21.15.28:0/847968848 shutdown_connections 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.230+0000 7f8655552640 1 --2- 172.21.15.28:0/847968848 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f865011bb10 0x7f865011df00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.230+0000 7f8655552640 1 -- 172.21.15.28:0/847968848 >> 172.21.15.28:0/847968848 conn(0x7f865006f790 msgr2=0x7f865006fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.230+0000 7f8655552640 1 -- 172.21.15.28:0/847968848 shutdown_connections 2024-09-19T11:23:00.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.230+0000 7f8655552640 1 -- 172.21.15.28:0/847968848 wait complete. 2024-09-19T11:23:00.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.231+0000 7f8655552640 1 Processor -- start 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.231+0000 7f8655552640 1 -- start start 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.231+0000 7f8655552640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8650117be0 0x7f86501a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.231+0000 7f8655552640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f865011eb80 con 0x7f8650117be0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.231+0000 7f864effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8650117be0 0x7f86501a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.231+0000 7f864effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8650117be0 0x7f86501a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49310/0 (socket says 172.21.15.28:49310) 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.231+0000 7f864effd640 1 -- 172.21.15.28:0/1784921792 learned_addr learned my addr 172.21.15.28:0/1784921792 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.232+0000 7f864effd640 1 -- 172.21.15.28:0/1784921792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86501a7d20 con 0x7f8650117be0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.232+0000 7f864effd640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8650117be0 0x7f86501a7750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f864000c3d0 tx=0x7f864000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.232+0000 7f86377fe640 1 -- 172.21.15.28:0/1784921792 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86400074d0 con 0x7f8650117be0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.232+0000 7f86377fe640 1 -- 172.21.15.28:0/1784921792 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86400047e0 con 0x7f8650117be0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.232+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86501a8010 con 0x7f8650117be0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.232+0000 7f86377fe640 1 -- 172.21.15.28:0/1784921792 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8640007670 con 0x7f8650117be0 2024-09-19T11:23:00.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.232+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501ad1e0 con 0x7f8650117be0 2024-09-19T11:23:00.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.233+0000 7f86377fe640 1 -- 172.21.15.28:0/1784921792 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8640011070 con 0x7f8650117be0 2024-09-19T11:23:00.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.233+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86501a8350 con 0x7f8650117be0 2024-09-19T11:23:00.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.233+0000 7f86377fe640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863c03e740 0x7f863c040c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:00.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.233+0000 7f86377fe640 1 -- 172.21.15.28:0/1784921792 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 3538+0+0 (secure 0 0 0) 0x7f86400529d0 con 0x7f8650117be0 2024-09-19T11:23:00.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.233+0000 7f864e7fc640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863c03e740 0x7f863c040c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:00.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.234+0000 7f864e7fc640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863c03e740 0x7f863c040c00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8644034000 tx=0x7f864402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:00.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.238+0000 7f86377fe640 1 -- 172.21.15.28:0/1784921792 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f8640010d70 con 0x7f8650117be0 2024-09-19T11:23:00.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.434+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]} v 0) -- 0x7f865011f160 con 0x7f8650117be0 2024-09-19T11:23:00.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.819+0000 7f86377fe640 1 -- 172.21.15.28:0/1784921792 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]=0 v17) ==== 151+0+0 (secure 0 0 0) 0x7f8640004ce0 con 0x7f8650117be0 2024-09-19T11:23:00.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863c03e740 msgr2=0x7f863c040c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863c03e740 0x7f863c040c00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8644034000 tx=0x7f864402e000 comp rx=0 tx=0).stop 2024-09-19T11:23:00.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8650117be0 msgr2=0x7f86501a7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8650117be0 0x7f86501a7750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f864000c3d0 tx=0x7f864000c890 comp rx=0 tx=0).stop 2024-09-19T11:23:00.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 shutdown_connections 2024-09-19T11:23:00.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863c03e740 0x7f863c040c00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:00.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 --2- 172.21.15.28:0/1784921792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8650117be0 0x7f86501a7750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:00.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.821+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 >> 172.21.15.28:0/1784921792 conn(0x7f865006f790 msgr2=0x7f865011bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:00.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.822+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 shutdown_connections 2024-09-19T11:23:00.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.822+0000 7f8655552640 1 -- 172.21.15.28:0/1784921792 wait complete. 2024-09-19T11:23:00.835 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-09-19T11:23:00.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.941+0000 7f15377f3640 1 Processor -- start 2024-09-19T11:23:00.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.941+0000 7f15377f3640 1 -- start start 2024-09-19T11:23:00.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f15377f3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1530074d80 0x7f153010fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:00.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f15377f3640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1530110770 con 0x7f15301102e0 2024-09-19T11:23:00.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f15377f3640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f1530110940 con 0x7f1530074d80 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f153558f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f1530074d80 0x7f153010fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f1534d8e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f15301102e0 0x7f153011b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48978/0 (socket says 172.21.15.28:48978) 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f1534d8e640 1 -- 172.21.15.28:0/2317372699 learned_addr learned my addr 172.21.15.28:0/2317372699 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f153558f640 1 -- 172.21.15.28:0/2317372699 >> v1:172.21.15.22:6789/0 conn(0x7f15301102e0 legacy=0x7f153011b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.942+0000 7f153558f640 1 -- 172.21.15.28:0/2317372699 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1530110af0 con 0x7f1530074d80 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.943+0000 7f153558f640 1 --2- 172.21.15.28:0/2317372699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f153010fda0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f152c009720 tx=0x7f152c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed2a96cf5998ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.943+0000 7f1527fff640 1 -- 172.21.15.28:0/2317372699 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c037070 con 0x7f1530074d80 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.943+0000 7f1527fff640 1 -- 172.21.15.28:0/2317372699 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152c02ad90 con 0x7f1530074d80 2024-09-19T11:23:00.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.943+0000 7f1527fff640 1 -- 172.21.15.28:0/2317372699 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c032590 con 0x7f1530074d80 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.944+0000 7f15377f3640 1 -- 172.21.15.28:0/2317372699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 msgr2=0x7f153010fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.944+0000 7f15377f3640 1 --2- 172.21.15.28:0/2317372699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f153010fda0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f152c009720 tx=0x7f152c02a110 comp rx=0 tx=0).stop 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.944+0000 7f15377f3640 1 -- 172.21.15.28:0/2317372699 shutdown_connections 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.944+0000 7f15377f3640 1 --2- 172.21.15.28:0/2317372699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f153010fda0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.944+0000 7f15377f3640 1 -- 172.21.15.28:0/2317372699 >> 172.21.15.28:0/2317372699 conn(0x7f1530070280 msgr2=0x7f15300726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.944+0000 7f15377f3640 1 -- 172.21.15.28:0/2317372699 shutdown_connections 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.944+0000 7f15377f3640 1 -- 172.21.15.28:0/2317372699 wait complete. 2024-09-19T11:23:00.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.945+0000 7f15377f3640 1 Processor -- start 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.945+0000 7f15377f3640 1 -- start start 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.945+0000 7f15377f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f15301a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.945+0000 7f15377f3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f153011ccc0 con 0x7f1530074d80 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.945+0000 7f153558f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f15301a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.945+0000 7f153558f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f15301a56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49330/0 (socket says 172.21.15.28:49330) 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.945+0000 7f153558f640 1 -- 172.21.15.28:0/86270339 learned_addr learned my addr 172.21.15.28:0/86270339 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.946+0000 7f153558f640 1 -- 172.21.15.28:0/86270339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15301a5cb0 con 0x7f1530074d80 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.946+0000 7f153558f640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f15301a56e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f152c009720 tx=0x7f152c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.946+0000 7f1525ffb640 1 -- 172.21.15.28:0/86270339 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c041070 con 0x7f1530074d80 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.946+0000 7f1525ffb640 1 -- 172.21.15.28:0/86270339 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f152c03c460 con 0x7f1530074d80 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.946+0000 7f1525ffb640 1 -- 172.21.15.28:0/86270339 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c037040 con 0x7f1530074d80 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.946+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15301a5f40 con 0x7f1530074d80 2024-09-19T11:23:00.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.946+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15301aaf10 con 0x7f1530074d80 2024-09-19T11:23:00.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.947+0000 7f1525ffb640 1 -- 172.21.15.28:0/86270339 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f152c044070 con 0x7f1530074d80 2024-09-19T11:23:00.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.947+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153010fe20 con 0x7f1530074d80 2024-09-19T11:23:00.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.948+0000 7f1525ffb640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f151403e6f0 0x7f1514040bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:00.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.948+0000 7f1525ffb640 1 -- 172.21.15.28:0/86270339 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 3802+0+0 (secure 0 0 0) 0x7f152c0738f0 con 0x7f1530074d80 2024-09-19T11:23:00.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.948+0000 7f1534d8e640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f151403e6f0 0x7f1514040bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:00.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.948+0000 7f1534d8e640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f151403e6f0 0x7f1514040bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1520007fd0 tx=0x7f1520007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:00.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:00.953+0000 7f1525ffb640 1 -- 172.21.15.28:0/86270339 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f152c03d3f0 con 0x7f1530074d80 2024-09-19T11:23:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:01.142+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"} v 0) -- 0x7f15301102e0 con 0x7f1530074d80 2024-09-19T11:23:02.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.211+0000 7f1525ffb640 1 -- 172.21.15.28:0/86270339 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"}]=0 pool 'default.rgw.buckets.data' created v19) ==== 227+0+0 (secure 0 0 0) 0x7f152c03ac80 con 0x7f1530074d80 2024-09-19T11:23:02.213 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'default.rgw.buckets.data' created 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f151403e6f0 msgr2=0x7f1514040bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f151403e6f0 0x7f1514040bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1520007fd0 tx=0x7f1520007aa0 comp rx=0 tx=0).stop 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 msgr2=0x7f15301a56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f15301a56e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f152c009720 tx=0x7f152c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 shutdown_connections 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f151403e6f0 0x7f1514040bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 --2- 172.21.15.28:0/86270339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1530074d80 0x7f15301a56e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.215+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 >> 172.21.15.28:0/86270339 conn(0x7f1530070280 msgr2=0x7f1530118780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:02.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.216+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 shutdown_connections 2024-09-19T11:23:02.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.217+0000 7f15377f3640 1 -- 172.21.15.28:0/86270339 wait complete. 2024-09-19T11:23:02.239 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-09-19T11:23:02.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f2721c7f640 1 Processor -- start 2024-09-19T11:23:02.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f2721c7f640 1 -- start start 2024-09-19T11:23:02.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f2721c7f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f271c110900 0x7f271c0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:02.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f2721c7f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f271c074ca0 con 0x7f271c074810 2024-09-19T11:23:02.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f2721c7f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f271c074e70 con 0x7f271c110900 2024-09-19T11:23:02.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f271b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f271c110900 0x7f271c0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:02.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f271b7fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f271c110900 0x7f271c0742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49340/0 (socket says 172.21.15.28:49340) 2024-09-19T11:23:02.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.345+0000 7f271b7fe640 1 -- 172.21.15.28:0/2013616564 learned_addr learned my addr 172.21.15.28:0/2013616564 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:02.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.346+0000 7f271b7fe640 1 -- 172.21.15.28:0/2013616564 >> v1:172.21.15.22:6789/0 conn(0x7f271c074810 legacy=0x7f271c11b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:02.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.346+0000 7f271b7fe640 1 -- 172.21.15.28:0/2013616564 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c075020 con 0x7f271c110900 2024-09-19T11:23:02.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.347+0000 7f271b7fe640 1 --2- 172.21.15.28:0/2013616564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c110900 0x7f271c0742d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f270c0097f0 tx=0x7f270c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b2d6ad5f8de212 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:02.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.347+0000 7f271a7fc640 1 -- 172.21.15.28:0/2013616564 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f270c037070 con 0x7f271c110900 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.347+0000 7f271a7fc640 1 -- 172.21.15.28:0/2013616564 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c032440 con 0x7f271c110900 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.347+0000 7f271a7fc640 1 -- 172.21.15.28:0/2013616564 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f270c02ae60 con 0x7f271c110900 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.347+0000 7f2721c7f640 1 -- 172.21.15.28:0/2013616564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c110900 msgr2=0x7f271c0742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.347+0000 7f2721c7f640 1 --2- 172.21.15.28:0/2013616564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c110900 0x7f271c0742d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f270c0097f0 tx=0x7f270c02a1e0 comp rx=0 tx=0).stop 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.348+0000 7f2721c7f640 1 -- 172.21.15.28:0/2013616564 shutdown_connections 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.348+0000 7f2721c7f640 1 --2- 172.21.15.28:0/2013616564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c110900 0x7f271c0742d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.348+0000 7f2721c7f640 1 -- 172.21.15.28:0/2013616564 >> 172.21.15.28:0/2013616564 conn(0x7f271c0702f0 msgr2=0x7f271c072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.348+0000 7f2721c7f640 1 -- 172.21.15.28:0/2013616564 shutdown_connections 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.348+0000 7f2721c7f640 1 -- 172.21.15.28:0/2013616564 wait complete. 2024-09-19T11:23:02.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.348+0000 7f2721c7f640 1 Processor -- start 2024-09-19T11:23:02.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.348+0000 7f2721c7f640 1 -- start start 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.349+0000 7f2721c7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c074810 0x7f271c110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.349+0000 7f2721c7f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f271c11ccc0 con 0x7f271c074810 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.349+0000 7f271b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c074810 0x7f271c110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.349+0000 7f271b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c074810 0x7f271c110fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49350/0 (socket says 172.21.15.28:49350) 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.349+0000 7f271b7fe640 1 -- 172.21.15.28:0/1804850778 learned_addr learned my addr 172.21.15.28:0/1804850778 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.349+0000 7f271b7fe640 1 -- 172.21.15.28:0/1804850778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c112ed0 con 0x7f271c074810 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.349+0000 7f271b7fe640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c074810 0x7f271c110fd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f270c035040 tx=0x7f270c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.350+0000 7f2703fff640 1 -- 172.21.15.28:0/1804850778 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f270c041070 con 0x7f271c074810 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.350+0000 7f2703fff640 1 -- 172.21.15.28:0/1804850778 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270c03c7a0 con 0x7f271c074810 2024-09-19T11:23:02.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.350+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c111570 con 0x7f271c074810 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.350+0000 7f2703fff640 1 -- 172.21.15.28:0/1804850778 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f270c037040 con 0x7f271c074810 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.350+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c111b10 con 0x7f271c074810 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.351+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c111e50 con 0x7f271c074810 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.351+0000 7f2703fff640 1 -- 172.21.15.28:0/1804850778 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f270c044070 con 0x7f271c074810 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.351+0000 7f2703fff640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f270803e740 0x7f2708040c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.351+0000 7f2703fff640 1 -- 172.21.15.28:0/1804850778 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4249+0+0 (secure 0 0 0) 0x7f270c073bf0 con 0x7f271c074810 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.351+0000 7f271affd640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f270803e740 0x7f2708040c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:02.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.352+0000 7f271affd640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f270803e740 0x7f2708040c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2710001600 tx=0x7f27100079f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:02.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.356+0000 7f2703fff640 1 -- 172.21.15.28:0/1804850778 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f270c03d3f0 con 0x7f271c074810 2024-09-19T11:23:02.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:02.547+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) -- 0x7f271c074350 con 0x7f271c074810 2024-09-19T11:23:03.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.214+0000 7f2703fff640 1 -- 172.21.15.28:0/1804850778 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v20) ==== 185+0+0 (secure 0 0 0) 0x7f270c03ac80 con 0x7f271c074810 2024-09-19T11:23:03.216 INFO:teuthology.orchestra.run.smithi028.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-09-19T11:23:03.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.217+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f270803e740 msgr2=0x7f2708040c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:03.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.217+0000 7f2721c7f640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f270803e740 0x7f2708040c00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2710001600 tx=0x7f27100079f0 comp rx=0 tx=0).stop 2024-09-19T11:23:03.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.217+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c074810 msgr2=0x7f271c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:03.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.217+0000 7f2721c7f640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c074810 0x7f271c110fd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f270c035040 tx=0x7f270c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:23:03.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.218+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 shutdown_connections 2024-09-19T11:23:03.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.218+0000 7f2721c7f640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f270803e740 0x7f2708040c00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:03.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.218+0000 7f2721c7f640 1 --2- 172.21.15.28:0/1804850778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f271c074810 0x7f271c110fd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:03.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.218+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 >> 172.21.15.28:0/1804850778 conn(0x7f271c0702f0 msgr2=0x7f271c1187d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:03.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.220+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 shutdown_connections 2024-09-19T11:23:03.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.222+0000 7f2721c7f640 1 -- 172.21.15.28:0/1804850778 wait complete. 2024-09-19T11:23:03.256 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-09-19T11:23:03.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.363+0000 7f508b4fe640 1 Processor -- start 2024-09-19T11:23:03.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.363+0000 7f508b4fe640 1 -- start start 2024-09-19T11:23:03.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.363+0000 7f508b4fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f50841137c0 0x7f5084113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:03.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.363+0000 7f508b4fe640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50841142c0 con 0x7f5084074250 2024-09-19T11:23:03.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.363+0000 7f508b4fe640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f5084074df0 con 0x7f50841137c0 2024-09-19T11:23:03.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.364+0000 7f508929a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f50841137c0 0x7f5084113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:03.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.364+0000 7f5088a99640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f5084074250 0x7f50840746e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48996/0 (socket says 172.21.15.28:48996) 2024-09-19T11:23:03.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.364+0000 7f508929a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f50841137c0 0x7f5084113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49354/0 (socket says 172.21.15.28:49354) 2024-09-19T11:23:03.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.364+0000 7f5088a99640 1 -- 172.21.15.28:0/1168918346 learned_addr learned my addr 172.21.15.28:0/1168918346 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:03.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.364+0000 7f508929a640 1 -- 172.21.15.28:0/1168918346 >> v1:172.21.15.22:6789/0 conn(0x7f5084074250 legacy=0x7f50840746e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:03.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.364+0000 7f508929a640 1 -- 172.21.15.28:0/1168918346 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5084074fa0 con 0x7f50841137c0 2024-09-19T11:23:03.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.365+0000 7f508929a640 1 --2- 172.21.15.28:0/1168918346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50841137c0 0x7f5084113ba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5080009720 tx=0x7f508002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fd0c7822875beb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:03.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.365+0000 7f507bfff640 1 -- 172.21.15.28:0/1168918346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5080037070 con 0x7f50841137c0 2024-09-19T11:23:03.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.365+0000 7f507bfff640 1 -- 172.21.15.28:0/1168918346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508002ad90 con 0x7f50841137c0 2024-09-19T11:23:03.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.365+0000 7f507bfff640 1 -- 172.21.15.28:0/1168918346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5080032590 con 0x7f50841137c0 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.365+0000 7f508b4fe640 1 -- 172.21.15.28:0/1168918346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50841137c0 msgr2=0x7f5084113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.365+0000 7f508b4fe640 1 --2- 172.21.15.28:0/1168918346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50841137c0 0x7f5084113ba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5080009720 tx=0x7f508002a110 comp rx=0 tx=0).stop 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.366+0000 7f508b4fe640 1 -- 172.21.15.28:0/1168918346 shutdown_connections 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.366+0000 7f508b4fe640 1 --2- 172.21.15.28:0/1168918346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f50841137c0 0x7f5084113ba0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.366+0000 7f508b4fe640 1 -- 172.21.15.28:0/1168918346 >> 172.21.15.28:0/1168918346 conn(0x7f50840702f0 msgr2=0x7f5084072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.366+0000 7f508b4fe640 1 -- 172.21.15.28:0/1168918346 shutdown_connections 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.366+0000 7f508b4fe640 1 -- 172.21.15.28:0/1168918346 wait complete. 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.366+0000 7f508b4fe640 1 Processor -- start 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.366+0000 7f508b4fe640 1 -- start start 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.367+0000 7f508b4fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5084074250 0x7f50841a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.367+0000 7f508b4fe640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f508411bc10 con 0x7f5084074250 2024-09-19T11:23:03.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.367+0000 7f508929a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5084074250 0x7f50841a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:03.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.367+0000 7f508929a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5084074250 0x7f50841a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49356/0 (socket says 172.21.15.28:49356) 2024-09-19T11:23:03.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.367+0000 7f508929a640 1 -- 172.21.15.28:0/914945998 learned_addr learned my addr 172.21.15.28:0/914945998 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:03.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.367+0000 7f508929a640 1 -- 172.21.15.28:0/914945998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a5d50 con 0x7f5084074250 2024-09-19T11:23:03.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.368+0000 7f508929a640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5084074250 0x7f50841a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5080009720 tx=0x7f50800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.368+0000 7f5079ffb640 1 -- 172.21.15.28:0/914945998 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5080041070 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.368+0000 7f5079ffb640 1 -- 172.21.15.28:0/914945998 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508003c460 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.368+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a5fe0 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.368+0000 7f5079ffb640 1 -- 172.21.15.28:0/914945998 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5080037040 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.368+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841aafb0 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.369+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084113c20 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.369+0000 7f5079ffb640 1 -- 172.21.15.28:0/914945998 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5080044070 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.369+0000 7f5079ffb640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f506803e6c0 0x7f5068040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.369+0000 7f5079ffb640 1 -- 172.21.15.28:0/914945998 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 4260+0+0 (secure 0 0 0) 0x7f50800738e0 con 0x7f5084074250 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.369+0000 7f5088a99640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f506803e6c0 0x7f5068040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:03.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.370+0000 7f5088a99640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f506803e6c0 0x7f5068040b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5074007fd0 tx=0x7f5074007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:03.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.375+0000 7f5079ffb640 1 -- 172.21.15.28:0/914945998 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f508003c790 con 0x7f5084074250 2024-09-19T11:23:03.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:03.562+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) -- 0x7f50841137c0 con 0x7f5084074250 2024-09-19T11:23:04.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.218+0000 7f5079ffb640 1 -- 172.21.15.28:0/914945998 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v21) ==== 169+0+0 (secure 0 0 0) 0x7f5080045be0 con 0x7f5084074250 2024-09-19T11:23:04.220 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'default.rgw.buckets.index' created 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f506803e6c0 msgr2=0x7f5068040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f506803e6c0 0x7f5068040b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5074007fd0 tx=0x7f5074007aa0 comp rx=0 tx=0).stop 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5084074250 msgr2=0x7f50841a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5084074250 0x7f50841a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5080009720 tx=0x7f50800046c0 comp rx=0 tx=0).stop 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 shutdown_connections 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f506803e6c0 0x7f5068040b80 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5074007fd0 tx=0x7f5074007aa0 comp rx=0 tx=0).stop 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 --2- 172.21.15.28:0/914945998 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5084074250 0x7f50841a5780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.222+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 >> 172.21.15.28:0/914945998 conn(0x7f50840702f0 msgr2=0x7f508411a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:04.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.223+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 shutdown_connections 2024-09-19T11:23:04.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.224+0000 7f508b4fe640 1 -- 172.21.15.28:0/914945998 wait complete. 2024-09-19T11:23:04.246 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-09-19T11:23:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.351+0000 7f23a427f640 1 Processor -- start 2024-09-19T11:23:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.351+0000 7f23a427f640 1 -- start start 2024-09-19T11:23:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.351+0000 7f23a427f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f239c110900 0x7f239c0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.351+0000 7f23a427f640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f239c074ca0 con 0x7f239c074810 2024-09-19T11:23:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.351+0000 7f23a427f640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f239c074e70 con 0x7f239c110900 2024-09-19T11:23:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.352+0000 7f23a201b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f239c110900 0x7f239c0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.352+0000 7f23a181a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f239c074810 0x7f239c11b120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:49000/0 (socket says 172.21.15.28:49000) 2024-09-19T11:23:04.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.352+0000 7f23a181a640 1 -- 172.21.15.28:0/4059803328 learned_addr learned my addr 172.21.15.28:0/4059803328 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:04.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.352+0000 7f23a201b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f239c110900 0x7f239c0742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49362/0 (socket says 172.21.15.28:49362) 2024-09-19T11:23:04.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.352+0000 7f23a201b640 1 -- 172.21.15.28:0/4059803328 >> v1:172.21.15.22:6789/0 conn(0x7f239c074810 legacy=0x7f239c11b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:04.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.352+0000 7f23a201b640 1 -- 172.21.15.28:0/4059803328 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c075020 con 0x7f239c110900 2024-09-19T11:23:04.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.353+0000 7f23a201b640 1 --2- 172.21.15.28:0/4059803328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c110900 0x7f239c0742d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2398009720 tx=0x7f239802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33882d5f94cbd02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.353+0000 7f23a1019640 1 -- 172.21.15.28:0/4059803328 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2398037070 con 0x7f239c110900 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.353+0000 7f23a1019640 1 -- 172.21.15.28:0/4059803328 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239802ad90 con 0x7f239c110900 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.353+0000 7f23a1019640 1 -- 172.21.15.28:0/4059803328 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2398032590 con 0x7f239c110900 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.353+0000 7f23a427f640 1 -- 172.21.15.28:0/4059803328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c110900 msgr2=0x7f239c0742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.353+0000 7f23a427f640 1 --2- 172.21.15.28:0/4059803328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c110900 0x7f239c0742d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2398009720 tx=0x7f239802a110 comp rx=0 tx=0).stop 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.354+0000 7f23a427f640 1 -- 172.21.15.28:0/4059803328 shutdown_connections 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.354+0000 7f23a427f640 1 --2- 172.21.15.28:0/4059803328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c110900 0x7f239c0742d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.354+0000 7f23a427f640 1 -- 172.21.15.28:0/4059803328 >> 172.21.15.28:0/4059803328 conn(0x7f239c0702f0 msgr2=0x7f239c072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.354+0000 7f23a427f640 1 -- 172.21.15.28:0/4059803328 shutdown_connections 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.354+0000 7f23a427f640 1 -- 172.21.15.28:0/4059803328 wait complete. 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.354+0000 7f23a427f640 1 Processor -- start 2024-09-19T11:23:04.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.354+0000 7f23a427f640 1 -- start start 2024-09-19T11:23:04.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.355+0000 7f23a427f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c074810 0x7f239c110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:04.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.355+0000 7f23a427f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f239c11ccc0 con 0x7f239c074810 2024-09-19T11:23:04.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.355+0000 7f23a201b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c074810 0x7f239c110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:04.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.355+0000 7f23a201b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c074810 0x7f239c110fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:49368/0 (socket says 172.21.15.28:49368) 2024-09-19T11:23:04.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.355+0000 7f23a201b640 1 -- 172.21.15.28:0/2985538333 learned_addr learned my addr 172.21.15.28:0/2985538333 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:04.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.355+0000 7f23a201b640 1 -- 172.21.15.28:0/2985538333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c112ed0 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.355+0000 7f23a201b640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c074810 0x7f239c110fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2398009720 tx=0x7f23980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.356+0000 7f2392ffd640 1 -- 172.21.15.28:0/2985538333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2398041070 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.356+0000 7f2392ffd640 1 -- 172.21.15.28:0/2985538333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f239803c460 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.356+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c111570 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.356+0000 7f2392ffd640 1 -- 172.21.15.28:0/2985538333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2398037040 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.356+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c111b10 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.357+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239c074350 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.357+0000 7f2392ffd640 1 -- 172.21.15.28:0/2985538333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2398044070 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.357+0000 7f2392ffd640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f238003e740 0x7f2380040c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.357+0000 7f2392ffd640 1 -- 172.21.15.28:0/2985538333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 4624+0+0 (secure 0 0 0) 0x7f2398073cc0 con 0x7f239c074810 2024-09-19T11:23:04.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.357+0000 7f23a181a640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f238003e740 0x7f2380040c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:04.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.358+0000 7f23a181a640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f238003e740 0x7f2380040c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f238c007fd0 tx=0x7f238c007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:04.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.363+0000 7f2392ffd640 1 -- 172.21.15.28:0/2985538333 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f239803d3f0 con 0x7f239c074810 2024-09-19T11:23:04.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:04.550+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) -- 0x7f239c111d90 con 0x7f239c074810 2024-09-19T11:23:05.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.221+0000 7f2392ffd640 1 -- 172.21.15.28:0/2985538333 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v22) ==== 187+0+0 (secure 0 0 0) 0x7f239803ac80 con 0x7f239c074810 2024-09-19T11:23:05.223 INFO:teuthology.orchestra.run.smithi028.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-09-19T11:23:05.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f238003e740 msgr2=0x7f2380040c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:05.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f238003e740 0x7f2380040c00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f238c007fd0 tx=0x7f238c007aa0 comp rx=0 tx=0).stop 2024-09-19T11:23:05.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c074810 msgr2=0x7f239c110fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:05.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c074810 0x7f239c110fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2398009720 tx=0x7f23980046c0 comp rx=0 tx=0).stop 2024-09-19T11:23:05.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 shutdown_connections 2024-09-19T11:23:05.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f238003e740 0x7f2380040c00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:05.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 --2- 172.21.15.28:0/2985538333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f239c074810 0x7f239c110fd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:05.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.225+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 >> 172.21.15.28:0/2985538333 conn(0x7f239c0702f0 msgr2=0x7f239c1187d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:05.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.227+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 shutdown_connections 2024-09-19T11:23:05.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:05.234+0000 7f23a427f640 1 -- 172.21.15.28:0/2985538333 wait complete. 2024-09-19T11:23:05.260 DEBUG:tasks.rgw:Pools created 2024-09-19T11:23:05.261 INFO:tasks.rgw:Starting rgw... 2024-09-19T11:23:05.261 INFO:tasks.rgw:rgw client.0 config is {} 2024-09-19T11:23:05.261 INFO:tasks.rgw:Using beast as radosgw frontend 2024-09-19T11:23:05.261 DEBUG:teuthology.orchestra.run.smithi028:> sudo echo -n http://smithi028.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-09-19T11:23:05.290 INFO:teuthology.orchestra.run.smithi028.stdout:http://smithi028.front.sepia.ceph.com:80 2024-09-19T11:23:05.290 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-09-19T11:23:05.357 INFO:tasks.rgw.client.0:Restarting daemon 2024-09-19T11:23:05.357 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-09-19T11:23:05.399 INFO:tasks.rgw.client.0:Started 2024-09-19T11:23:05.400 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi028.front.sepia.ceph.com:80/ 2024-09-19T11:23:05.400 DEBUG:teuthology.orchestra.run.smithi028:> curl http://smithi028.front.sepia.ceph.com:80/ 2024-09-19T11:23:05.428 INFO:teuthology.orchestra.run.smithi028.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-19T11:23:05.428 INFO:teuthology.orchestra.run.smithi028.stderr: Dload Upload Total Spent Left Speed 2024-09-19T11:23:05.430 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-19T11:23:05.430 INFO:teuthology.orchestra.run.smithi028.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-19T11:23:05.430 INFO:teuthology.orchestra.run.smithi028.stderr:curl: (7) Failed to connect to smithi028.front.sepia.ceph.com port 80: Connection refused 2024-09-19T11:23:06.431 DEBUG:teuthology.orchestra.run.smithi028:> curl http://smithi028.front.sepia.ceph.com:80/ 2024-09-19T11:23:06.457 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-19T11:23:06.457 INFO:teuthology.orchestra.run.smithi028.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-19T11:23:06.457 INFO:teuthology.orchestra.run.smithi028.stderr: Dload Upload Total Spent Left Speed 2024-09-19T11:23:06.458 INFO:teuthology.orchestra.run.smithi028.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-19T11:23:06.458 INFO:teuthology.orchestra.run.smithi028.stderr:curl: (7) Failed to connect to smithi028.front.sepia.ceph.com port 80: Connection refused 2024-09-19T11:23:08.458 DEBUG:teuthology.orchestra.run.smithi028:> curl http://smithi028.front.sepia.ceph.com:80/ 2024-09-19T11:23:08.483 INFO:teuthology.orchestra.run.smithi028.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-19T11:23:08.483 INFO:teuthology.orchestra.run.smithi028.stderr: Dload Upload Total Spent Left Speed 2024-09-19T11:23:08.485 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-19T11:23:08.485 INFO:teuthology.orchestra.run.smithi028.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-19T11:23:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:curl: (7) Failed to connect to smithi028.front.sepia.ceph.com port 80: Connection refused 2024-09-19T11:23:12.487 DEBUG:teuthology.orchestra.run.smithi028:> curl http://smithi028.front.sepia.ceph.com:80/ 2024-09-19T11:23:12.513 INFO:teuthology.orchestra.run.smithi028.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-19T11:23:12.513 INFO:teuthology.orchestra.run.smithi028.stderr: Dload Upload Total Spent Left Speed 2024-09-19T11:23:12.516 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-19T11:23:12.516 INFO:teuthology.orchestra.run.smithi028.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-19T11:23:12.516 INFO:teuthology.orchestra.run.smithi028.stderr:curl: (7) Failed to connect to smithi028.front.sepia.ceph.com port 80: Connection refused 2024-09-19T11:23:20.517 DEBUG:teuthology.orchestra.run.smithi028:> curl http://smithi028.front.sepia.ceph.com:80/ 2024-09-19T11:23:20.542 INFO:teuthology.orchestra.run.smithi028.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-19T11:23:20.543 INFO:teuthology.orchestra.run.smithi028.stderr: Dload Upload Total Spent Left Speed 2024-09-19T11:23:20.545 INFO:teuthology.orchestra.run.smithi028.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 93500 0 --:--:-- --:--:-- --:--:-- 93500 2024-09-19T11:23:20.545 INFO:teuthology.orchestra.run.smithi028.stdout:anonymous 2024-09-19T11:23:20.546 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-09-19T11:23:20.632 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-09-19T11:23:20.632 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-09-19T11:23:20.632 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-09-19T11:23:20.633 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-09-19T11:23:20.684 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:20.684 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.683+0000 7fc839131800 1 Processor -- start 2024-09-19T11:23:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.683+0000 7fc839131800 1 -- start start 2024-09-19T11:23:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.683+0000 7fc839131800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562d42434d60 0x562d4243a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc839131800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562d424340a0 con 0x562d424339f0 2024-09-19T11:23:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc839131800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x562d422bc7d0 con 0x562d42434d60 2024-09-19T11:23:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc834598640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562d42434d60 0x562d4243a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc834598640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562d42434d60 0x562d4243a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58438/0 (socket says 172.21.15.28:58438) 2024-09-19T11:23:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc834598640 1 -- 172.21.15.28:0/1108190981 learned_addr learned my addr 172.21.15.28:0/1108190981 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc834d99640 1 --1- 172.21.15.28:0/1108190981 >> v1:172.21.15.22:6789/0 conn(0x562d424339f0 0x562d42434650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32914/0 (socket says 172.21.15.28:32914) 2024-09-19T11:23:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc834598640 1 -- 172.21.15.28:0/1108190981 >> v1:172.21.15.22:6789/0 conn(0x562d424339f0 legacy=0x562d42434650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc834598640 1 -- 172.21.15.28:0/1108190981 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d42435210 con 0x562d42434d60 2024-09-19T11:23:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.684+0000 7fc833d97640 1 -- 172.21.15.28:0/1108190981 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3457042720 0 0) 0x562d424340a0 con 0x562d424339f0 2024-09-19T11:23:20.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.686+0000 7fc834598640 1 --2- 172.21.15.28:0/1108190981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42434d60 0x562d4243a900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00a520 tx=0x7fc81c045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304367f371ae5bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc833d97640 1 -- 172.21.15.28:0/1108190981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc81c055070 con 0x562d42434d60 2024-09-19T11:23:20.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc833d97640 1 -- 172.21.15.28:0/1108190981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc81c0515b0 con 0x562d42434d60 2024-09-19T11:23:20.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc833d97640 1 -- 172.21.15.28:0/1108190981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc81c0518d0 con 0x562d42434d60 2024-09-19T11:23:20.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc839131800 1 -- 172.21.15.28:0/1108190981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42434d60 msgr2=0x562d4243a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc839131800 1 --2- 172.21.15.28:0/1108190981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42434d60 0x562d4243a900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00a520 tx=0x7fc81c045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc839131800 1 -- 172.21.15.28:0/1108190981 shutdown_connections 2024-09-19T11:23:20.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc839131800 1 --2- 172.21.15.28:0/1108190981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42434d60 0x562d4243a900 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc839131800 1 -- 172.21.15.28:0/1108190981 >> 172.21.15.28:0/1108190981 conn(0x562d423a6d60 msgr2=0x562d423a71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc839131800 1 -- 172.21.15.28:0/1108190981 shutdown_connections 2024-09-19T11:23:20.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.687+0000 7fc839131800 1 -- 172.21.15.28:0/1108190981 wait complete. 2024-09-19T11:23:20.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.705+0000 7fc839131800 1 Processor -- start 2024-09-19T11:23:20.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.705+0000 7fc839131800 1 -- start start 2024-09-19T11:23:20.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.705+0000 7fc839131800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d42523320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.705+0000 7fc839131800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d422c9dd0 con 0x562d424339f0 2024-09-19T11:23:20.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.706+0000 7fc834d99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d42523320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.706+0000 7fc834d99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d42523320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58454/0 (socket says 172.21.15.28:58454) 2024-09-19T11:23:20.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.706+0000 7fc834d99640 1 -- 172.21.15.28:0/4167141342 learned_addr learned my addr 172.21.15.28:0/4167141342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.706+0000 7fc834d99640 1 -- 172.21.15.28:0/4167141342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d42523910 con 0x562d424339f0 2024-09-19T11:23:20.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.706+0000 7fc834d99640 1 --2- 172.21.15.28:0/4167141342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d42523320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc824001ea0 tx=0x7fc82400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a63783dc7604f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc722e0d640 1 -- 172.21.15.28:0/4167141342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc824008070 con 0x562d424339f0 2024-09-19T11:23:20.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc722e0d640 1 -- 172.21.15.28:0/4167141342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82400d500 con 0x562d424339f0 2024-09-19T11:23:20.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc722e0d640 1 -- 172.21.15.28:0/4167141342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82400d880 con 0x562d424339f0 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc839131800 1 -- 172.21.15.28:0/4167141342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 msgr2=0x562d42523320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc839131800 1 --2- 172.21.15.28:0/4167141342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d42523320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc824001ea0 tx=0x7fc82400b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc839131800 1 -- 172.21.15.28:0/4167141342 shutdown_connections 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc839131800 1 --2- 172.21.15.28:0/4167141342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d42523320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.707+0000 7fc839131800 1 -- 172.21.15.28:0/4167141342 >> 172.21.15.28:0/4167141342 conn(0x562d423a6d60 msgr2=0x562d4251eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc839131800 1 -- 172.21.15.28:0/4167141342 shutdown_connections 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc839131800 1 -- 172.21.15.28:0/4167141342 wait complete. 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc839131800 1 Processor -- start 2024-09-19T11:23:20.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc839131800 1 -- start start 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc839131800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d4252ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc839131800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d4243b050 con 0x562d424339f0 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc834d99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d4252ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc834d99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d4252ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58464/0 (socket says 172.21.15.28:58464) 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.708+0000 7fc834d99640 1 -- 172.21.15.28:0/182703062 learned_addr learned my addr 172.21.15.28:0/182703062 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.709+0000 7fc834d99640 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4252b370 con 0x562d424339f0 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.709+0000 7fc834d99640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d4252ae10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc824000f60 tx=0x7fc824000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.709+0000 7fc72160a640 1 -- 172.21.15.28:0/182703062 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82400f8d0 con 0x562d424339f0 2024-09-19T11:23:20.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.709+0000 7fc72160a640 1 -- 172.21.15.28:0/182703062 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82400fa70 con 0x562d424339f0 2024-09-19T11:23:20.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.709+0000 7fc72160a640 1 -- 172.21.15.28:0/182703062 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82400fd50 con 0x562d424339f0 2024-09-19T11:23:20.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.709+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d4252b600 con 0x562d424339f0 2024-09-19T11:23:20.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.709+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4252c000 con 0x562d424339f0 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.710+0000 7fc72160a640 1 -- 172.21.15.28:0/182703062 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc824017040 con 0x562d424339f0 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc72160a640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc71c03d000 0x7fc71c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc72160a640 1 -- 172.21.15.28:0/182703062 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc824015020 con 0x562d424339f0 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc834598640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc71c03d000 0x7fc71c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42434d60 0x562d42532460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d425329a0 con 0x562d42434d60 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc83559a640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42434d60 0x562d42532460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc834598640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc71c03d000 0x7fc71c03f4c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc81c057000 tx=0x7fc81c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.711+0000 7fc83559a640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42434d60 0x562d42532460 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.712+0000 7fc83559a640 1 -- 172.21.15.28:0/182703062 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d42434d60 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.712+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42532d40 con 0x562d42434d60 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.712+0000 7fc83559a640 1 -- 172.21.15.28:0/182703062 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d42434d60 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.712+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d42534840 0x562d42536cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.712+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42537450 con 0x562d42534840 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.712+0000 7fc834d99640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d42534840 0x562d42536cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.712+0000 7fc834d99640 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d42534840 0x562d42536cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.713+0000 7fc834d99640 1 -- 172.21.15.28:0/182703062 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc82402d070 con 0x562d42534840 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.713+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42533590 con 0x562d42434d60 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.713+0000 7fc83559a640 1 -- 172.21.15.28:0/182703062 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d42434d60 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.713+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d425339e0 con 0x562d42534840 2024-09-19T11:23:20.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.713+0000 7fc834d99640 1 -- 172.21.15.28:0/182703062 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc82402d070 con 0x562d42534840 2024-09-19T11:23:20.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.714+0000 7fc839131800 1 Processor -- start 2024-09-19T11:23:20.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.714+0000 7fc839131800 1 -- start start 2024-09-19T11:23:20.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.714+0000 7fc839131800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42545dc0 0x562d425461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.714+0000 7fc839131800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d423284a0 con 0x562d42545dc0 2024-09-19T11:23:20.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.714+0000 7fc83559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42545dc0 0x562d425461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc83559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42545dc0 0x562d425461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58480/0 (socket says 172.21.15.28:58480) 2024-09-19T11:23:20.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc83559a640 1 -- 172.21.15.28:0/4176099979 learned_addr learned my addr 172.21.15.28:0/4176099979 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc83559a640 1 -- 172.21.15.28:0/4176099979 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d42546820 con 0x562d42545dc0 2024-09-19T11:23:20.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc83559a640 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42545dc0 0x562d425461e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562d4252be10 tx=0x7fc82c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc71a7fc640 1 -- 172.21.15.28:0/4176099979 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82c013640 con 0x562d42545dc0 2024-09-19T11:23:20.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc71a7fc640 1 -- 172.21.15.28:0/4176099979 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c0137e0 con 0x562d42545dc0 2024-09-19T11:23:20.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc71a7fc640 1 -- 172.21.15.28:0/4176099979 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82c013ae0 con 0x562d42545dc0 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.715+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d42547050 con 0x562d42545dc0 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.716+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d425475f0 con 0x562d42545dc0 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.716+0000 7fc71a7fc640 1 -- 172.21.15.28:0/4176099979 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc82c011040 con 0x562d42545dc0 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc71a7fc640 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70c03d000 0x7fc70c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc71a7fc640 1 -- 172.21.15.28:0/4176099979 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc82c00e040 con 0x562d42545dc0 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc834d99640 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70c03d000 0x7fc70c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc839131800 1 Processor -- start 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc834d99640 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70c03d000 0x7fc70c03f4c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc82400eab0 tx=0x7fc82401b440 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc839131800 1 -- start start 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc839131800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42552c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.717+0000 7fc839131800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d42523d40 con 0x562d42552860 2024-09-19T11:23:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc83559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42552c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc83559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42552c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58488/0 (socket says 172.21.15.28:58488) 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc83559a640 1 -- 172.21.15.28:0/2279084327 learned_addr learned my addr 172.21.15.28:0/2279084327 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc83559a640 1 -- 172.21.15.28:0/2279084327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d42553230 con 0x562d42552860 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc83559a640 1 --2- 172.21.15.28:0/2279084327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42552c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0547f0 tx=0x7fc82c01fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87d25495e442e2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc70bfff640 1 -- 172.21.15.28:0/2279084327 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82c051070 con 0x562d42552860 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc70bfff640 1 -- 172.21.15.28:0/2279084327 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c053440 con 0x562d42552860 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.718+0000 7fc70bfff640 1 -- 172.21.15.28:0/2279084327 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82c021a90 con 0x562d42552860 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 -- 172.21.15.28:0/2279084327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 msgr2=0x562d42552c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 --2- 172.21.15.28:0/2279084327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42552c40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0547f0 tx=0x7fc82c01fe70 comp rx=0 tx=0).stop 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 -- 172.21.15.28:0/2279084327 shutdown_connections 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 --2- 172.21.15.28:0/2279084327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42552c40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 -- 172.21.15.28:0/2279084327 >> 172.21.15.28:0/2279084327 conn(0x562d4254ec60 msgr2=0x562d4254cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 -- 172.21.15.28:0/2279084327 shutdown_connections 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 -- 172.21.15.28:0/2279084327 wait complete. 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 Processor -- start 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.719+0000 7fc839131800 1 -- start start 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc839131800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42559e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc839131800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d42553870 con 0x562d42552860 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc83559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42559e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc83559a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42559e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58490/0 (socket says 172.21.15.28:58490) 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 learned_addr learned my addr 172.21.15.28:0/4130144084 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d4255a3e0 con 0x562d42552860 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc83559a640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42559e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0547f0 tx=0x7fc82c054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc70a7fc640 1 -- 172.21.15.28:0/4130144084 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82c021440 con 0x562d42552860 2024-09-19T11:23:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc70a7fc640 1 -- 172.21.15.28:0/4130144084 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c04c070 con 0x562d42552860 2024-09-19T11:23:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d4255a670 con 0x562d42552860 2024-09-19T11:23:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.720+0000 7fc70a7fc640 1 -- 172.21.15.28:0/4130144084 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc82c0459c0 con 0x562d42552860 2024-09-19T11:23:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.721+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d4255b200 con 0x562d42552860 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.721+0000 7fc70a7fc640 1 -- 172.21.15.28:0/4130144084 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc82c045b60 con 0x562d42552860 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc70a7fc640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70003d000 0x7fc70003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc70a7fc640 1 -- 172.21.15.28:0/4130144084 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc82c0539d0 con 0x562d42552860 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc834d99640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70003d000 0x7fc70003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42565760 0x562d42567c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42568370 con 0x562d42565760 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc834d99640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70003d000 0x7fc70003f4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc82400f640 tx=0x7fc824046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.722+0000 7fc834598640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42565760 0x562d42567c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc834598640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42565760 0x562d42567c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 20 realm 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d425687a0 con 0x562d42565760 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42569070 con 0x562d42565760 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.723+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42569410 con 0x562d42565760 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.724+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.724+0000 7fc839131800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.724+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.724+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d4256a170 0x562d4256c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.724+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d4256caa0 con 0x562d4256a170 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.724+0000 7fc83559a640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d4256a170 0x562d4256c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.724+0000 7fc83559a640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d4256a170 0x562d4256c560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc839131800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc839131800 20 searching for the correct realm 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc83559a640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc82c04d2a0 0x7fc82c03d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc82c04d2a0 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc834d99640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc82c04d2a0 0x7fc82c03d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.725+0000 7fc834d99640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc82c04d2a0 0x7fc82c03d7f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.726+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.726+0000 7fc834d99640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc824054320 0x7fc824054720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.726+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.726+0000 7fc834598640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc824054320 0x7fc824054720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.726+0000 7fc834598640 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc824054320 0x7fc824054720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.727+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.727+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.727+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.727+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.727+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.727+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.728+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.728+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.728+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.728+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc82c04d2a0 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.728+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.728+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.729+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.729+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.729+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.729+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.729+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.729+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc82c04d2a0 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v24'2 uv2 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.730+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.731+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.731+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.731+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.731+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.731+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.731+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.732+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.732+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.732+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.732+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.732+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.732+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.733+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.733+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc82c04d2a0 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.733+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.733+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.733+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.733+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc82c04d2a0 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.734+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d4256a170 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.735+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.735+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc824054320 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.735+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.735+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x562d42565760 2024-09-19T11:23:20.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.735+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.735+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256e410 con 0x7fc82c04d2a0 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d4e0 con 0x562d42565760 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d4256dd80 con 0x562d42565760 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.736+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d120 con 0x562d4256a170 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 zone default found 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 4 Realm: () 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:20.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 zonegroup default 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d9b0 con 0x7fc824054320 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42570cb0 con 0x562d4256a170 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.737+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d425710a0 con 0x562d4256a170 2024-09-19T11:23:20.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.738+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.738+0000 7fc839131800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.738+0000 7fc839131800 20 started sync module instance, tier type = 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.738+0000 7fc839131800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.738+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42570140 con 0x7fc82c04d2a0 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.739+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.739+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94752386516272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d425719b0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.739+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42572200 con 0x562d42565760 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.740+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94752386516272] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.741+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.741+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94752386520320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42572260 con 0x562d42565760 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.741+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42572d70 con 0x562d4256a170 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.742+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.742+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94752386520320] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.742+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94752386522448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42572d30 con 0x562d4256a170 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.742+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.743+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.743+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94752386528048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d425740b0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.743+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42574bd0 con 0x562d42565760 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.743+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94752386522448] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.744+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.744+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94752386533024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42574b90 con 0x562d42565760 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.744+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42575630 con 0x7fc824054320 2024-09-19T11:23:20.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.744+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94752386528048] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.745+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.746+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94752386535776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42575630 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.746+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42576070 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.746+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94752386533024] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.747+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94752386535776] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.747+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.747+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94752386530304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42575bc0 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.747+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42576490 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94752386538432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42576b40 con 0x7fc824054320 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 20 add_watcher() i=0 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 20 add_watcher() i=1 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 20 add_watcher() i=2 2024-09-19T11:23:20.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 20 add_watcher() i=3 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 20 add_watcher() i=4 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 20 add_watcher() i=5 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94752386530304] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.748+0000 7fc839131800 20 add_watcher() i=6 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.749+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94752386538432] v31'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c057030 con 0x7fc824054320 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.749+0000 7fc839131800 20 add_watcher() i=7 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.749+0000 7fc839131800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.749+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562d42579b50 con 0x562d42565760 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.750+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc81c057030 con 0x562d42565760 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.750+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94752386541168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42576f70 con 0x562d42565760 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94752386541168] v31'24 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc6eafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:20.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc839131800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc839131800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc839131800 5 note: GC not initialized 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc6e9fe3640 20 reqs_thread_entry: start 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc839131800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc839131800 20 Filter name: none 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc65b7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.751+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425abf90 con 0x562d4256a170 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.752+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(62 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.752+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425710a0 con 0x562d4256a170 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.752+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(63 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.752+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425710a0 con 0x562d42565760 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.752+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(64 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.752+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d9b0 con 0x562d42565760 2024-09-19T11:23:20.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d9b0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(67 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d42565760 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.753+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(69 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(71 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d42565760 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(72 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.754+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.755+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.755+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.755+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(74 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.755+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.755+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(75 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.755+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(76 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(77 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(78 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(79 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.756+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.757+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(80 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.757+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d42565760 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.757+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(81 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.757+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d42565760 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.757+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(82 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.757+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.757+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(83 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.758+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.758+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(84 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.758+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.758+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(85 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.758+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d42565760 2024-09-19T11:23:20.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.758+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(86 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.758+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.759+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(87 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.759+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.759+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(88 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.759+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.759+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(89 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.759+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc824054320 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.760+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(90 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.760+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d42565760 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.760+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(91 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.760+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x562d4256a170 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.760+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(92 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.760+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562d425737e0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.760+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(93 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.761+0000 7fc6eafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.761+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94752386541168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d425710a0 con 0x562d42565760 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.762+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94752386541168] v31'25 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.762+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94752386516272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d9b0 con 0x7fc82c04d2a0 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.763+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94752386516272] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.763+0000 7fc839131800 20 remove_watcher() i=0 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.763+0000 7fc839131800 2 removed watcher, disabling cache 2024-09-19T11:23:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.763+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94752386520320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d9b0 con 0x562d42565760 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.765+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94752386520320] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.765+0000 7fc839131800 20 remove_watcher() i=1 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.765+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94752386522448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d4256d9b0 con 0x562d4256a170 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.766+0000 7fc83559a640 1 -- 172.21.15.28:0/4130144084 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94752386522448] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc82c00a4d0 con 0x562d4256a170 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.766+0000 7fc839131800 20 remove_watcher() i=2 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.766+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94752386528048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42571500 con 0x7fc82c04d2a0 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.767+0000 7fc834d99640 1 -- 172.21.15.28:0/4130144084 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94752386528048] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc824039070 con 0x7fc82c04d2a0 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.767+0000 7fc839131800 20 remove_watcher() i=3 2024-09-19T11:23:20.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.767+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94752386533024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42571500 con 0x562d42565760 2024-09-19T11:23:20.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.768+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94752386533024] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c07f040 con 0x562d42565760 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.768+0000 7fc839131800 20 remove_watcher() i=4 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.768+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94752386535776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d425746a0 con 0x7fc824054320 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.769+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94752386535776] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.769+0000 7fc839131800 20 remove_watcher() i=5 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.769+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94752386530304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42571500 con 0x7fc824054320 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.770+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94752386530304] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.770+0000 7fc839131800 20 remove_watcher() i=6 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.770+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94752386538432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562d42575160 con 0x7fc824054320 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.771+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94752386538432] v31'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc81c07f040 con 0x7fc824054320 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.771+0000 7fc839131800 20 remove_watcher() i=7 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc824054320 msgr2=0x7fc824054720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc824054320 0x7fc824054720 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42565760 msgr2=0x562d42567c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42565760 0x562d42567c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc82c04d2a0 msgr2=0x7fc82c03d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc82c04d2a0 0x7fc82c03d7f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d4256a170 msgr2=0x562d4256c560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d4256a170 0x562d4256c560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70003d000 msgr2=0x7fc70003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70003d000 0x7fc70003f4c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc82400f640 tx=0x7fc824046000 comp rx=0 tx=0).stop 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc834598640 1 -- 172.21.15.28:0/4130144084 reap_dead start 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 msgr2=0x562d42559e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42559e80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0547f0 tx=0x7fc82c054780 comp rx=0 tx=0).stop 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 shutdown_connections 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4130144084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42552860 0x562d42559e80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 >> 172.21.15.28:0/4130144084 conn(0x562d4254ec60 msgr2=0x562d4254ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 shutdown_connections 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4130144084 wait complete. 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70c03d000 msgr2=0x7fc70c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70c03d000 0x7fc70c03f4c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc82400eab0 tx=0x7fc82401b440 comp rx=0 tx=0).stop 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42545dc0 msgr2=0x562d425461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42545dc0 0x562d425461e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x562d4252be10 tx=0x7fc82c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 shutdown_connections 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc70c03d000 0x7fc70c03f4c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 --2- 172.21.15.28:0/4176099979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d42545dc0 0x562d425461e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 >> 172.21.15.28:0/4176099979 conn(0x562d4253a170 msgr2=0x562d4253a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 shutdown_connections 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.772+0000 7fc839131800 1 -- 172.21.15.28:0/4176099979 wait complete. 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42434d60 msgr2=0x562d42532460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42434d60 0x562d42532460 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d42534840 msgr2=0x562d42536cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d42534840 0x562d42536cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc71c03d000 msgr2=0x7fc71c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc71c03d000 0x7fc71c03f4c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc81c057000 tx=0x7fc81c052000 comp rx=0 tx=0).stop 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 msgr2=0x562d4252ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d4252ae10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc824000f60 tx=0x7fc824000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 shutdown_connections 2024-09-19T11:23:20.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d42534840 0x562d42536cc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc71c03d000 0x7fc71c03f4c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d42434d60 0x562d42532460 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 --2- 172.21.15.28:0/182703062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d424339f0 0x562d4252ae10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 >> 172.21.15.28:0/182703062 conn(0x562d423a6d60 msgr2=0x562d42524810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 shutdown_connections 2024-09-19T11:23:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.773+0000 7fc839131800 1 -- 172.21.15.28:0/182703062 wait complete. 2024-09-19T11:23:20.784 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T11:23:20.785 DEBUG:tasks.util.rgw: json result: [] 2024-09-19T11:23:20.785 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-09-19T11:23:20.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-09-19T11:23:20.785 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-09-19T11:23:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe91c687800 1 Processor -- start 2024-09-19T11:23:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe91c687800 1 -- start start 2024-09-19T11:23:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe91c687800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e931788aa0 0x55e93178e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe91c687800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e931787d40 con 0x55e931787690 2024-09-19T11:23:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe91c687800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55e9316137d0 con 0x55e931788aa0 2024-09-19T11:23:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe913fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e931788aa0 0x55e93178e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe91a171640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55e931787690 0x55e931788390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32924/0 (socket says 172.21.15.28:32924) 2024-09-19T11:23:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.831+0000 7fe91a171640 1 -- 172.21.15.28:0/3955449593 learned_addr learned my addr 172.21.15.28:0/3955449593 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.832+0000 7fe913fff640 1 -- 172.21.15.28:0/3955449593 >> v1:172.21.15.22:6789/0 conn(0x55e931787690 legacy=0x55e931788390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.832+0000 7fe919970640 1 -- 172.21.15.28:0/3955449593 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857901359 0 0) 0x55e931787d40 con 0x55e931787690 2024-09-19T11:23:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.832+0000 7fe913fff640 1 -- 172.21.15.28:0/3955449593 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e931788f50 con 0x55e931788aa0 2024-09-19T11:23:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.833+0000 7fe913fff640 1 --2- 172.21.15.28:0/3955449593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931788aa0 0x55e93178e640 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe90400a470 tx=0x7fe904045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7661b6a738b84498 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe919970640 1 -- 172.21.15.28:0/3955449593 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe904053570 con 0x55e931788aa0 2024-09-19T11:23:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe919970640 1 -- 172.21.15.28:0/3955449593 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe904058070 con 0x55e931788aa0 2024-09-19T11:23:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe919970640 1 -- 172.21.15.28:0/3955449593 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe904053880 con 0x55e931788aa0 2024-09-19T11:23:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe91c687800 1 -- 172.21.15.28:0/3955449593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931788aa0 msgr2=0x55e93178e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe91c687800 1 --2- 172.21.15.28:0/3955449593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931788aa0 0x55e93178e640 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe90400a470 tx=0x7fe904045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe91c687800 1 -- 172.21.15.28:0/3955449593 shutdown_connections 2024-09-19T11:23:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe91c687800 1 --2- 172.21.15.28:0/3955449593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931788aa0 0x55e93178e640 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe91c687800 1 -- 172.21.15.28:0/3955449593 >> 172.21.15.28:0/3955449593 conn(0x55e931782a60 msgr2=0x55e931784ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe91c687800 1 -- 172.21.15.28:0/3955449593 shutdown_connections 2024-09-19T11:23:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.834+0000 7fe91c687800 1 -- 172.21.15.28:0/3955449593 wait complete. 2024-09-19T11:23:20.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.849+0000 7fe91c687800 1 Processor -- start 2024-09-19T11:23:20.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.849+0000 7fe91c687800 1 -- start start 2024-09-19T11:23:20.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.849+0000 7fe91c687800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e9316fb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.849+0000 7fe91c687800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e931620dd0 con 0x55e931787690 2024-09-19T11:23:20.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.850+0000 7fe91a171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e9316fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.850+0000 7fe91a171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e9316fb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58510/0 (socket says 172.21.15.28:58510) 2024-09-19T11:23:20.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.850+0000 7fe91a171640 1 -- 172.21.15.28:0/1690923065 learned_addr learned my addr 172.21.15.28:0/1690923065 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.850+0000 7fe91a171640 1 -- 172.21.15.28:0/1690923065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93187ad20 con 0x55e931787690 2024-09-19T11:23:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.850+0000 7fe91a171640 1 --2- 172.21.15.28:0/1690923065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e9316fb860 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe90c001ea0 tx=0x7fe90c00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf5a6ea096fd239 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe808608640 1 -- 172.21.15.28:0/1690923065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c008070 con 0x55e931787690 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe808608640 1 -- 172.21.15.28:0/1690923065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c00d500 con 0x55e931787690 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe808608640 1 -- 172.21.15.28:0/1690923065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c00d880 con 0x55e931787690 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe91c687800 1 -- 172.21.15.28:0/1690923065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 msgr2=0x55e9316fb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe91c687800 1 --2- 172.21.15.28:0/1690923065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e9316fb860 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe90c001ea0 tx=0x7fe90c00b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe91c687800 1 -- 172.21.15.28:0/1690923065 shutdown_connections 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe91c687800 1 --2- 172.21.15.28:0/1690923065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e9316fb860 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe91c687800 1 -- 172.21.15.28:0/1690923065 >> 172.21.15.28:0/1690923065 conn(0x55e931782a60 msgr2=0x55e9318729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe91c687800 1 -- 172.21.15.28:0/1690923065 shutdown_connections 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.851+0000 7fe91c687800 1 -- 172.21.15.28:0/1690923065 wait complete. 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91c687800 1 Processor -- start 2024-09-19T11:23:20.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91c687800 1 -- start start 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91c687800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e931878430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91c687800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e93178ed90 con 0x55e931787690 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91a171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e931878430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91a171640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e931878430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58522/0 (socket says 172.21.15.28:58522) 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91a171640 1 -- 172.21.15.28:0/4849627 learned_addr learned my addr 172.21.15.28:0/4849627 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91a171640 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e931878990 con 0x55e931787690 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.852+0000 7fe91a171640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e931878430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0012f0 tx=0x7fe90c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.853+0000 7fe806e05640 1 -- 172.21.15.28:0/4849627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c00f9d0 con 0x55e931787690 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.853+0000 7fe806e05640 1 -- 172.21.15.28:0/4849627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c00fb70 con 0x55e931787690 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.853+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e931876d40 con 0x55e931787690 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.853+0000 7fe806e05640 1 -- 172.21.15.28:0/4849627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe90c00fe50 con 0x55e931787690 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.853+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e931877300 con 0x55e931787690 2024-09-19T11:23:20.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe806e05640 1 -- 172.21.15.28:0/4849627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe90c017040 con 0x55e931787690 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe806e05640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe80003d150 0x7fe80003f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe806e05640 1 -- 172.21.15.28:0/4849627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fe90c015020 con 0x55e931787690 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe913fff640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe80003d150 0x7fe80003f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e931788aa0 0x55e9318891c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e931889740 con 0x55e931788aa0 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe91a972640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e931788aa0 0x55e9318891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe913fff640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe80003d150 0x7fe80003f610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe904057000 tx=0x7fe904052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.854+0000 7fe91a972640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e931788aa0 0x55e9318891c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.855+0000 7fe91a972640 1 -- 172.21.15.28:0/4849627 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe91400a4d0 con 0x55e931788aa0 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.855+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e931877800 con 0x55e931788aa0 2024-09-19T11:23:20.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.855+0000 7fe91a972640 1 -- 172.21.15.28:0/4849627 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe91400a4d0 con 0x55e931788aa0 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.855+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e93188ba50 0x55e93188de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.855+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e93188e510 con 0x55e93188ba50 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.855+0000 7fe91a171640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e93188ba50 0x55e93188de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.855+0000 7fe91a171640 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e93188ba50 0x55e93188de40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.856+0000 7fe91a171640 1 -- 172.21.15.28:0/4849627 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe90c02d070 con 0x55e93188ba50 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.856+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e93188a8f0 con 0x55e931788aa0 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.856+0000 7fe91a972640 1 -- 172.21.15.28:0/4849627 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe91400a4d0 con 0x55e931788aa0 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.856+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e93188ad10 con 0x55e93188ba50 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91a171640 1 -- 172.21.15.28:0/4849627 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe90c02d070 con 0x55e93188ba50 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91c687800 1 Processor -- start 2024-09-19T11:23:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91c687800 1 -- start start 2024-09-19T11:23:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91c687800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e93189ce80 0x55e93189d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91c687800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e93167f4a0 con 0x55e93189ce80 2024-09-19T11:23:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91a972640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e93189ce80 0x55e93189d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91a972640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e93189ce80 0x55e93189d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58524/0 (socket says 172.21.15.28:58524) 2024-09-19T11:23:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.857+0000 7fe91a972640 1 -- 172.21.15.28:0/368106261 learned_addr learned my addr 172.21.15.28:0/368106261 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.858+0000 7fe91a972640 1 -- 172.21.15.28:0/368106261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93189d8e0 con 0x55e93189ce80 2024-09-19T11:23:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.858+0000 7fe91a972640 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e93189ce80 0x55e93189d2a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55e931877fd0 tx=0x7fe9140167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.858+0000 7fe7fffff640 1 -- 172.21.15.28:0/368106261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe914013640 con 0x55e93189ce80 2024-09-19T11:23:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.858+0000 7fe7fffff640 1 -- 172.21.15.28:0/368106261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9140137e0 con 0x55e93189ce80 2024-09-19T11:23:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.858+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e93189e110 con 0x55e93189ce80 2024-09-19T11:23:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.858+0000 7fe7fffff640 1 -- 172.21.15.28:0/368106261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe914013ae0 con 0x55e93189ce80 2024-09-19T11:23:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.858+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93189e6b0 con 0x55e93189ce80 2024-09-19T11:23:20.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.859+0000 7fe7fffff640 1 -- 172.21.15.28:0/368106261 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe914011040 con 0x55e93189ce80 2024-09-19T11:23:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.859+0000 7fe7fffff640 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7f403d000 0x7fe7f403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.859+0000 7fe7fffff640 1 -- 172.21.15.28:0/368106261 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fe91400e040 con 0x55e93189ce80 2024-09-19T11:23:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.859+0000 7fe91a171640 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7f403d000 0x7fe7f403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.859+0000 7fe91c687800 1 Processor -- start 2024-09-19T11:23:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91a171640 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7f403d000 0x7fe7f403f4c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55e931889bd0 tx=0x7fe90c053170 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91c687800 1 -- start start 2024-09-19T11:23:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91c687800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91c687800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e93187b000 con 0x55e9318a9900 2024-09-19T11:23:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91a972640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91a972640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58536/0 (socket says 172.21.15.28:58536) 2024-09-19T11:23:20.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91a972640 1 -- 172.21.15.28:0/883221510 learned_addr learned my addr 172.21.15.28:0/883221510 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91a972640 1 -- 172.21.15.28:0/883221510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9318aa2f0 con 0x55e9318a9900 2024-09-19T11:23:20.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe91a972640 1 --2- 172.21.15.28:0/883221510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318a9d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe914016cd0 tx=0x7fe914014480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99479e66f4c4e78b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe7fdffb640 1 -- 172.21.15.28:0/883221510 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe914051070 con 0x55e9318a9900 2024-09-19T11:23:20.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe7fdffb640 1 -- 172.21.15.28:0/883221510 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe914053440 con 0x55e9318a9900 2024-09-19T11:23:20.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.860+0000 7fe7fdffb640 1 -- 172.21.15.28:0/883221510 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe914021a60 con 0x55e9318a9900 2024-09-19T11:23:20.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 -- 172.21.15.28:0/883221510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 msgr2=0x55e9318a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 --2- 172.21.15.28:0/883221510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318a9d00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe914016cd0 tx=0x7fe914014480 comp rx=0 tx=0).stop 2024-09-19T11:23:20.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 -- 172.21.15.28:0/883221510 shutdown_connections 2024-09-19T11:23:20.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 --2- 172.21.15.28:0/883221510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318a9d00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 -- 172.21.15.28:0/883221510 >> 172.21.15.28:0/883221510 conn(0x55e9318a5d00 msgr2=0x55e9318a3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 -- 172.21.15.28:0/883221510 shutdown_connections 2024-09-19T11:23:20.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 -- 172.21.15.28:0/883221510 wait complete. 2024-09-19T11:23:20.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 Processor -- start 2024-09-19T11:23:20.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 -- start start 2024-09-19T11:23:20.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.861+0000 7fe91c687800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e9318aa910 con 0x55e9318a9900 2024-09-19T11:23:20.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe91a972640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe91a972640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58544/0 (socket says 172.21.15.28:58544) 2024-09-19T11:23:20.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 learned_addr learned my addr 172.21.15.28:0/165529200 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:20.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9318b1470 con 0x55e9318a9900 2024-09-19T11:23:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe91a972640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318b0f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe914054740 tx=0x7fe914054770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe7ebfff640 1 -- 172.21.15.28:0/165529200 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe914045e00 con 0x55e9318a9900 2024-09-19T11:23:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe7ebfff640 1 -- 172.21.15.28:0/165529200 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe914013c80 con 0x55e9318a9900 2024-09-19T11:23:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe7ebfff640 1 -- 172.21.15.28:0/165529200 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe914013e30 con 0x55e9318a9900 2024-09-19T11:23:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e9318b1700 con 0x55e9318a9900 2024-09-19T11:23:20.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.862+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9318b1700 con 0x55e9318a9900 2024-09-19T11:23:20.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.863+0000 7fe7ebfff640 1 -- 172.21.15.28:0/165529200 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe914045130 con 0x55e9318a9900 2024-09-19T11:23:20.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.863+0000 7fe7ebfff640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7e403d000 0x7fe7e403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe7ebfff640 1 -- 172.21.15.28:0/165529200 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fe914036c90 con 0x55e9318a9900 2024-09-19T11:23:20.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe91a171640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7e403d000 0x7fe7e403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe91a171640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7e403d000 0x7fe7e403f4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00f640 tx=0x7fe90c048000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e9318bc5e0 0x55e9318beaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318bf1f0 con 0x55e9318bc5e0 2024-09-19T11:23:20.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe913fff640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e9318bc5e0 0x55e9318beaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.864+0000 7fe913fff640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e9318bc5e0 0x55e9318beaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 realm 2024-09-19T11:23:20.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318bf620 con 0x55e9318bc5e0 2024-09-19T11:23:20.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:20.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318bfef0 con 0x55e9318bc5e0 2024-09-19T11:23:20.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c0290 con 0x55e9318bc5e0 2024-09-19T11:23:20.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:20.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e9318c0ff0 0x55e9318c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c39f0 con 0x55e9318c0ff0 2024-09-19T11:23:20.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.865+0000 7fe91a972640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e9318c0ff0 0x55e9318c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91a972640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e9318c0ff0 0x55e9318c33e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91c687800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:20.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91c687800 20 searching for the correct realm 2024-09-19T11:23:20.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91a972640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe914035070 0x7fe914055020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe914035070 2024-09-19T11:23:20.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.866+0000 7fe91a171640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe914035070 0x7fe914055020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.867+0000 7fe91a171640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe914035070 0x7fe914055020 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.867+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe90c02d070 con 0x7fe914035070 2024-09-19T11:23:20.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.867+0000 7fe91a171640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe90c0535b0 0x7fe90c0539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:20.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.867+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.867+0000 7fe913fff640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe90c0535b0 0x7fe90c0539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:20.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.868+0000 7fe913fff640 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe90c0535b0 0x7fe90c0539d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:20.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.868+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.868+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.868+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.868+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.868+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.868+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe914035070 2024-09-19T11:23:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe90c02d070 con 0x7fe914035070 2024-09-19T11:23:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.869+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe914035070 2024-09-19T11:23:20.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe90c02d070 con 0x7fe914035070 2024-09-19T11:23:20.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.870+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v24'2 uv2 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.871+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.872+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe914035070 2024-09-19T11:23:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.873+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe90c02d070 con 0x7fe914035070 2024-09-19T11:23:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.873+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.873+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.873+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.873+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.873+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe914035070 2024-09-19T11:23:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe90c02d070 con 0x7fe914035070 2024-09-19T11:23:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318c0ff0 2024-09-19T11:23:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.874+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe90c0535b0 2024-09-19T11:23:20.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x55e9318bc5e0 2024-09-19T11:23:20.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5320 con 0x7fe914035070 2024-09-19T11:23:20.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:20.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:20.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:20.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:20.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:20.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:20.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.875+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c43b0 con 0x55e9318bc5e0 2024-09-19T11:23:20.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c4c00 con 0x55e9318bc5e0 2024-09-19T11:23:20.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:20.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c57a0 con 0x55e9318c0ff0 2024-09-19T11:23:20.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:20.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 zone default found 2024-09-19T11:23:20.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 4 Realm: () 2024-09-19T11:23:20.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:20.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:20.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:20.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 zonegroup default 2024-09-19T11:23:20.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.876+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c5b80 con 0x7fe90c0535b0 2024-09-19T11:23:20.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c79b0 con 0x55e9318c0ff0 2024-09-19T11:23:20.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c7e40 con 0x55e9318c0ff0 2024-09-19T11:23:20.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 20 started sync module instance, tier type = 2024-09-19T11:23:20.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:20.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.877+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c6e50 con 0x7fe914035070 2024-09-19T11:23:20.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.878+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.878+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94460047028800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c6ab0 con 0x7fe914035070 2024-09-19T11:23:20.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.878+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c8d70 con 0x55e9318bc5e0 2024-09-19T11:23:20.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.879+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94460047028800] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.879+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.879+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94460047034560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c9190 con 0x55e9318bc5e0 2024-09-19T11:23:20.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.880+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c9b50 con 0x55e9318c0ff0 2024-09-19T11:23:20.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.881+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.881+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94460047038112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c9fb0 con 0x55e9318c0ff0 2024-09-19T11:23:20.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.881+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318caad0 con 0x7fe914035070 2024-09-19T11:23:20.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.881+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94460047034560] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.882+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94460047038112] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.882+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.882+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94460047041952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c9ac0 con 0x7fe914035070 2024-09-19T11:23:20.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.882+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cb4e0 con 0x55e9318bc5e0 2024-09-19T11:23:20.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.883+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94460047041952] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.883+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.883+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94460047044656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cb900 con 0x55e9318bc5e0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.883+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cc2c0 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.884+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94460047044656] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.884+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.884+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94460047046192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cc2c0 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.884+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318ccc80 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.885+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94460047046192] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.885+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.886+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94460047048688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318ccc80 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.886+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cd640 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.886+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94460047048688] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94460047050704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cd640 con 0x7fe90c0535b0 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 20 add_watcher() i=0 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 20 add_watcher() i=1 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 20 add_watcher() i=2 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 20 add_watcher() i=3 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 20 add_watcher() i=4 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 20 add_watcher() i=5 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.887+0000 7fe91c687800 20 add_watcher() i=6 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.888+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94460047050704] v31'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe904051070 con 0x7fe90c0535b0 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.888+0000 7fe91c687800 20 add_watcher() i=7 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.888+0000 7fe91c687800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.888+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d0aa0 con 0x55e9318bc5e0 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.888+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.888+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94460047053664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cdfc0 con 0x55e9318bc5e0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.889+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94460047053664] v31'26 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe904051070 con 0x55e9318bc5e0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.889+0000 7fe7d0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.889+0000 7fe91c687800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.889+0000 7fe91c687800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.889+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d9280 con 0x55e9318bc5e0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.891+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.891+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d9910 con 0x7fe90c0535b0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.892+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'10 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.892+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d9e80 con 0x55e9318c0ff0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.893+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.893+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d9e80 con 0x55e9318bc5e0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.894+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'27 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.894+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318da330 con 0x7fe914035070 2024-09-19T11:23:20.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.895+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'20 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.895+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d7220 con 0x55e9318c0ff0 2024-09-19T11:23:20.948 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.896+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'13 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.896+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d7220 con 0x55e9318bc5e0 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.897+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.897+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d7220 con 0x7fe90c0535b0 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.898+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'13 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.898+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d72f0 con 0x7fe914035070 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.899+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.899+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318dae90 con 0x55e9318c0ff0 2024-09-19T11:23:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.900+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'16 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.900+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318dae90 con 0x7fe914035070 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.901+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'21 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.901+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318daf20 con 0x7fe90c0535b0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.903+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'12 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.903+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318daf20 con 0x7fe914035070 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.904+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'10 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.904+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db3d0 con 0x7fe90c0535b0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.905+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'19 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.905+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db3d0 con 0x55e9318bc5e0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.906+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'28 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.906+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db3d0 con 0x55e9318c0ff0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.907+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'14 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.907+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db3d0 con 0x7fe914035070 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.908+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'22 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.908+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.909+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'10 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.909+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318bc5e0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.910+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.910+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318bc5e0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.912+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'29 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.912+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.913+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.913+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x7fe914035070 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.914+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.914+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.915+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.915+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.916+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.916+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.917+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'6 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.917+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.918+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'14 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.918+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x7fe90c0535b0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.919+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.919+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.920+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.920+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x7fe90c0535b0 2024-09-19T11:23:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.921+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'14 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.921+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x55e9318c0ff0 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.922+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'16 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.922+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x7fe90c0535b0 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.923+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'20 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.923+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318db340 con 0x7fe914035070 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.924+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe74b7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 20 Filter name: none 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe7497fa640 20 reqs_thread_entry: start 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 20 rados->read ofs=0 len=0 2024-09-19T11:23:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e931905d10 con 0x7fe914035070 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(94 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.925+0000 7fe91c687800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:could not remove user: unable to remove user, user does not exist 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.926+0000 7fe7d0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.926+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94460047053664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318d7c80 con 0x55e9318bc5e0 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.927+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94460047053664] v31'30 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.927+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94460047028800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cd330 con 0x7fe914035070 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.928+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94460047028800] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.928+0000 7fe91c687800 20 remove_watcher() i=0 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.928+0000 7fe91c687800 2 removed watcher, disabling cache 2024-09-19T11:23:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.928+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94460047034560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c7240 con 0x55e9318bc5e0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.929+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94460047034560] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.930+0000 7fe91c687800 20 remove_watcher() i=1 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.930+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94460047038112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c7240 con 0x55e9318c0ff0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.931+0000 7fe91a972640 1 -- 172.21.15.28:0/165529200 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94460047038112] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe914032070 con 0x55e9318c0ff0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.931+0000 7fe91c687800 20 remove_watcher() i=2 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.931+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94460047041952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c96a0 con 0x7fe914035070 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.932+0000 7fe91a171640 1 -- 172.21.15.28:0/165529200 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94460047041952] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90c061040 con 0x7fe914035070 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.932+0000 7fe91c687800 20 remove_watcher() i=3 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.932+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94460047044656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c7240 con 0x55e9318bc5e0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.933+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94460047044656] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x55e9318bc5e0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.933+0000 7fe91c687800 20 remove_watcher() i=4 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.933+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94460047046192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c96a0 con 0x7fe90c0535b0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.934+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94460047046192] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.934+0000 7fe91c687800 20 remove_watcher() i=5 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.934+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94460047048688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318cb030 con 0x7fe90c0535b0 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.935+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94460047048688] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.935+0000 7fe91c687800 20 remove_watcher() i=6 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.935+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94460047050704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e9318c7240 con 0x7fe90c0535b0 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94460047050704] v31'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe90407f040 con 0x7fe90c0535b0 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 20 remove_watcher() i=7 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe90c0535b0 msgr2=0x7fe90c0539d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe90c0535b0 0x7fe90c0539d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e9318bc5e0 msgr2=0x55e9318beaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e9318bc5e0 0x55e9318beaa0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe914035070 msgr2=0x7fe914055020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe914035070 0x7fe914055020 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e9318c0ff0 msgr2=0x55e9318c33e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e9318c0ff0 0x55e9318c33e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7e403d000 msgr2=0x7fe7e403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7e403d000 0x7fe7e403f4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00f640 tx=0x7fe90c048000 comp rx=0 tx=0).stop 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 msgr2=0x55e9318b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe913fff640 1 -- 172.21.15.28:0/165529200 reap_dead start 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318b0f10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe914054740 tx=0x7fe914054770 comp rx=0 tx=0).stop 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 shutdown_connections 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 --2- 172.21.15.28:0/165529200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e9318a9900 0x55e9318b0f10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 >> 172.21.15.28:0/165529200 conn(0x55e9318a5d00 msgr2=0x55e9318a3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 shutdown_connections 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.936+0000 7fe91c687800 1 -- 172.21.15.28:0/165529200 wait complete. 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7f403d000 msgr2=0x7fe7f403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7f403d000 0x7fe7f403f4c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55e931889bd0 tx=0x7fe90c053170 comp rx=0 tx=0).stop 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e93189ce80 msgr2=0x55e93189d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e93189ce80 0x55e93189d2a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x55e931877fd0 tx=0x7fe9140167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 shutdown_connections 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe7f403d000 0x7fe7f403f4c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/368106261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e93189ce80 0x55e93189d2a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 >> 172.21.15.28:0/368106261 conn(0x55e931891230 msgr2=0x55e931891610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 shutdown_connections 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/368106261 wait complete. 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e931788aa0 msgr2=0x55e9318891c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e931788aa0 0x55e9318891c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e93188ba50 msgr2=0x55e93188de40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e93188ba50 0x55e93188de40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe80003d150 msgr2=0x7fe80003f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe80003d150 0x7fe80003f610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe904057000 tx=0x7fe904052000 comp rx=0 tx=0).stop 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 msgr2=0x55e931878430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e931878430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0012f0 tx=0x7fe90c000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 shutdown_connections 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e93188ba50 0x55e93188de40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe80003d150 0x7fe80003f610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e931788aa0 0x55e9318891c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 --2- 172.21.15.28:0/4849627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e931787690 0x55e931878430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 >> 172.21.15.28:0/4849627 conn(0x55e931782a60 msgr2=0x55e9318711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 shutdown_connections 2024-09-19T11:23:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:20.937+0000 7fe91c687800 1 -- 172.21.15.28:0/4849627 wait complete. 2024-09-19T11:23:20.960 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-09-19T11:23:20.960 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-09-19T11:23:20.960 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-09-19T11:23:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effe1e2f800 1 Processor -- start 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effe1e2f800 1 -- start start 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effe1e2f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d2d4806aa0 0x55d2d480c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effe1e2f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d2d4805d40 con 0x55d2d4805690 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effe1e2f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55d2d46917d0 con 0x55d2d4806aa0 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effdf118640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d2d4806aa0 0x55d2d480c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effdf118640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d2d4806aa0 0x55d2d480c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58552/0 (socket says 172.21.15.28:58552) 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effdf118640 1 -- 172.21.15.28:0/660533523 learned_addr learned my addr 172.21.15.28:0/660533523 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.005+0000 7effdf919640 1 --1- 172.21.15.28:0/660533523 >> v1:172.21.15.22:6789/0 conn(0x55d2d4805690 0x55d2d4806390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32928/0 (socket says 172.21.15.28:32928) 2024-09-19T11:23:21.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.006+0000 7effdf118640 1 -- 172.21.15.28:0/660533523 >> v1:172.21.15.22:6789/0 conn(0x55d2d4805690 legacy=0x55d2d4806390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.006+0000 7effdf118640 1 -- 172.21.15.28:0/660533523 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d4806f50 con 0x55d2d4806aa0 2024-09-19T11:23:21.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.007+0000 7effdf118640 1 --2- 172.21.15.28:0/660533523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4806aa0 0x55d2d480c640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effd400a520 tx=0x7effd4045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d047465aaf18cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effde917640 1 -- 172.21.15.28:0/660533523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd4055070 con 0x55d2d4806aa0 2024-09-19T11:23:21.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effde917640 1 -- 172.21.15.28:0/660533523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd40515b0 con 0x55d2d4806aa0 2024-09-19T11:23:21.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effde917640 1 -- 172.21.15.28:0/660533523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd40518d0 con 0x55d2d4806aa0 2024-09-19T11:23:21.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effe1e2f800 1 -- 172.21.15.28:0/660533523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4806aa0 msgr2=0x55d2d480c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effe1e2f800 1 --2- 172.21.15.28:0/660533523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4806aa0 0x55d2d480c640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effd400a520 tx=0x7effd4045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effe1e2f800 1 -- 172.21.15.28:0/660533523 shutdown_connections 2024-09-19T11:23:21.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effe1e2f800 1 --2- 172.21.15.28:0/660533523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4806aa0 0x55d2d480c640 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effe1e2f800 1 -- 172.21.15.28:0/660533523 >> 172.21.15.28:0/660533523 conn(0x55d2d4800a60 msgr2=0x55d2d4802ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effe1e2f800 1 -- 172.21.15.28:0/660533523 shutdown_connections 2024-09-19T11:23:21.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.008+0000 7effe1e2f800 1 -- 172.21.15.28:0/660533523 wait complete. 2024-09-19T11:23:21.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.023+0000 7effe1e2f800 1 Processor -- start 2024-09-19T11:23:21.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.023+0000 7effe1e2f800 1 -- start start 2024-09-19T11:23:21.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.023+0000 7effe1e2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d48f6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.023+0000 7effe1e2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2d469edd0 con 0x55d2d4805690 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7effdf919640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d48f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7effdf919640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d48f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58562/0 (socket says 172.21.15.28:58562) 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7effdf919640 1 -- 172.21.15.28:0/1383336399 learned_addr learned my addr 172.21.15.28:0/1383336399 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7effdf919640 1 -- 172.21.15.28:0/1383336399 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d48f6910 con 0x55d2d4805690 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7effdf919640 1 --2- 172.21.15.28:0/1383336399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d48f6320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7effd000b260 tx=0x7effd000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f1e3556b438f0c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7efecde0b640 1 -- 172.21.15.28:0/1383336399 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd000be60 con 0x55d2d4805690 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7efecde0b640 1 -- 172.21.15.28:0/1383336399 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd0006070 con 0x55d2d4805690 2024-09-19T11:23:21.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.024+0000 7efecde0b640 1 -- 172.21.15.28:0/1383336399 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd000e710 con 0x55d2d4805690 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 -- 172.21.15.28:0/1383336399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 msgr2=0x55d2d48f6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 --2- 172.21.15.28:0/1383336399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d48f6320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7effd000b260 tx=0x7effd000b660 comp rx=0 tx=0).stop 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 -- 172.21.15.28:0/1383336399 shutdown_connections 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 --2- 172.21.15.28:0/1383336399 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d48f6320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 -- 172.21.15.28:0/1383336399 >> 172.21.15.28:0/1383336399 conn(0x55d2d4800a60 msgr2=0x55d2d48f1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 -- 172.21.15.28:0/1383336399 shutdown_connections 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 -- 172.21.15.28:0/1383336399 wait complete. 2024-09-19T11:23:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.025+0000 7effe1e2f800 1 Processor -- start 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effe1e2f800 1 -- start start 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effe1e2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d480f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effe1e2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2d480cdc0 con 0x55d2d4805690 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effdf919640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d480f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effdf919640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d480f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58572/0 (socket says 172.21.15.28:58572) 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effdf919640 1 -- 172.21.15.28:0/4063119670 learned_addr learned my addr 172.21.15.28:0/4063119670 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effdf919640 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d480fa20 con 0x55d2d4805690 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.026+0000 7effdf919640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d480f4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55d2d48f72b0 tx=0x7effd0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.027+0000 7efec7fff640 1 -- 172.21.15.28:0/4063119670 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd000ee10 con 0x55d2d4805690 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.027+0000 7efec7fff640 1 -- 172.21.15.28:0/4063119670 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd000fa30 con 0x55d2d4805690 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.027+0000 7efec7fff640 1 -- 172.21.15.28:0/4063119670 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd000fd30 con 0x55d2d4805690 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.027+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2d480ddd0 con 0x55d2d4805690 2024-09-19T11:23:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.027+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d480e390 con 0x55d2d4805690 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7efec7fff640 1 -- 172.21.15.28:0/4063119670 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7effd0017070 con 0x55d2d4805690 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7efec7fff640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efec803d000 0x7efec803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7efec7fff640 1 -- 172.21.15.28:0/4063119670 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7effd0015020 con 0x55d2d4805690 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7effdf118640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efec803d000 0x7efec803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d4806aa0 0x55d2d4907390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49078d0 con 0x55d2d4806aa0 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7effe011a640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d4806aa0 0x55d2d4907390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7effdf118640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efec803d000 0x7efec803f4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7effd4057000 tx=0x7effd4052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.028+0000 7effe011a640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d4806aa0 0x55d2d4907390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.029+0000 7effe011a640 1 -- 172.21.15.28:0/4063119670 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d4806aa0 2024-09-19T11:23:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.029+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d480e890 con 0x55d2d4806aa0 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.029+0000 7effe011a640 1 -- 172.21.15.28:0/4063119670 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d4806aa0 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.029+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d4909c20 0x55d2d490c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.029+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d490c6e0 con 0x55d2d4909c20 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.029+0000 7effdf919640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d4909c20 0x55d2d490c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.030+0000 7effdf919640 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d4909c20 0x55d2d490c010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.030+0000 7effdf919640 1 -- 172.21.15.28:0/4063119670 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7effd002e070 con 0x55d2d4909c20 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.030+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4908ac0 con 0x55d2d4806aa0 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.030+0000 7effe011a640 1 -- 172.21.15.28:0/4063119670 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d4806aa0 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.030+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4908f20 con 0x55d2d4909c20 2024-09-19T11:23:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.031+0000 7effdf919640 1 -- 172.21.15.28:0/4063119670 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7effd002e070 con 0x55d2d4909c20 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.031+0000 7effe1e2f800 1 Processor -- start 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.031+0000 7effe1e2f800 1 -- start start 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.031+0000 7effe1e2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d491b030 0x55d2d491b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.031+0000 7effe1e2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2d46fd4a0 con 0x55d2d491b030 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7effe011a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d491b030 0x55d2d491b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7effe011a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d491b030 0x55d2d491b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58582/0 (socket says 172.21.15.28:58582) 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7effe011a640 1 -- 172.21.15.28:0/596231503 learned_addr learned my addr 172.21.15.28:0/596231503 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7effe011a640 1 -- 172.21.15.28:0/596231503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d491ba90 con 0x55d2d491b030 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7effe011a640 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d491b030 0x55d2d491b450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55d2d480f060 tx=0x7effd80167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7efec57fa640 1 -- 172.21.15.28:0/596231503 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd8021570 con 0x55d2d491b030 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7efec57fa640 1 -- 172.21.15.28:0/596231503 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd8021710 con 0x55d2d491b030 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2d491c2c0 con 0x55d2d491b030 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.032+0000 7efec57fa640 1 -- 172.21.15.28:0/596231503 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd80219f0 con 0x55d2d491b030 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.033+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d491c860 con 0x55d2d491b030 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.033+0000 7efec57fa640 1 -- 172.21.15.28:0/596231503 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7effd8011040 con 0x55d2d491b030 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7efec57fa640 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efeb803d000 0x7efeb803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7efec57fa640 1 -- 172.21.15.28:0/596231503 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7effd800e040 con 0x55d2d491b030 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effdf919640 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efeb803d000 0x7efeb803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effe1e2f800 1 Processor -- start 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effdf919640 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efeb803d000 0x7efeb803f4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effd0001530 tx=0x7effd001c790 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effe1e2f800 1 -- start start 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effe1e2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d4927ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effe1e2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2d48f6d40 con 0x55d2d4927aa0 2024-09-19T11:23:21.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effe011a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d4927ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effe011a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d4927ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58590/0 (socket says 172.21.15.28:58590) 2024-09-19T11:23:21.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.034+0000 7effe011a640 1 -- 172.21.15.28:0/2058755678 learned_addr learned my addr 172.21.15.28:0/2058755678 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7effe011a640 1 -- 172.21.15.28:0/2058755678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d4928490 con 0x55d2d4927aa0 2024-09-19T11:23:21.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7effe011a640 1 --2- 172.21.15.28:0/2058755678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d4927ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7effd8054840 tx=0x7effd801f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc8d00caa85eaa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7efebeffd640 1 -- 172.21.15.28:0/2058755678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd8044070 con 0x55d2d4927aa0 2024-09-19T11:23:21.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7efebeffd640 1 -- 172.21.15.28:0/2058755678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd8056200 con 0x55d2d4927aa0 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7efebeffd640 1 -- 172.21.15.28:0/2058755678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd80294f0 con 0x55d2d4927aa0 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7effe1e2f800 1 -- 172.21.15.28:0/2058755678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 msgr2=0x55d2d4927ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7effe1e2f800 1 --2- 172.21.15.28:0/2058755678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d4927ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7effd8054840 tx=0x7effd801f980 comp rx=0 tx=0).stop 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7effe1e2f800 1 -- 172.21.15.28:0/2058755678 shutdown_connections 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7effe1e2f800 1 --2- 172.21.15.28:0/2058755678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d4927ea0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.035+0000 7effe1e2f800 1 -- 172.21.15.28:0/2058755678 >> 172.21.15.28:0/2058755678 conn(0x55d2d4923ec0 msgr2=0x55d2d4922100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe1e2f800 1 -- 172.21.15.28:0/2058755678 shutdown_connections 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe1e2f800 1 -- 172.21.15.28:0/2058755678 wait complete. 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe1e2f800 1 Processor -- start 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe1e2f800 1 -- start start 2024-09-19T11:23:21.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe1e2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d492f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe1e2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2d4928ab0 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe011a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d492f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe011a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d492f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58592/0 (socket says 172.21.15.28:58592) 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.036+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 learned_addr learned my addr 172.21.15.28:0/181216571 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.037+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d492f5f0 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.037+0000 7effe011a640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d492f090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effd8054840 tx=0x7effd80547d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.037+0000 7efebd7fa640 1 -- 172.21.15.28:0/181216571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd802a5e0 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.037+0000 7efebd7fa640 1 -- 172.21.15.28:0/181216571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd8045a10 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.037+0000 7efebd7fa640 1 -- 172.21.15.28:0/181216571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd8045d30 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.037+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2d492f880 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.037+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d492f880 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.038+0000 7efebd7fa640 1 -- 172.21.15.28:0/181216571 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7effd8054980 con 0x55d2d4927aa0 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.038+0000 7efebd7fa640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efea803d000 0x7efea803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.038+0000 7efebd7fa640 1 -- 172.21.15.28:0/181216571 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7effd8017450 con 0x55d2d4927aa0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.038+0000 7effdf919640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efea803d000 0x7efea803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.038+0000 7effdf919640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efea803d000 0x7efea803f4c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7effd000bfb0 tx=0x7effd0051000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d493a7f0 0x55d2d493ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d493d400 con 0x55d2d493a7f0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effdf118640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d493a7f0 0x55d2d493ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effdf118640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d493a7f0 0x55d2d493ccb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effe1e2f800 20 realm 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.039+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d493d830 con 0x55d2d493a7f0 2024-09-19T11:23:21.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d493e100 con 0x55d2d493a7f0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d493e4a0 con 0x55d2d493a7f0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d493f200 0x55d2d49415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4941c00 con 0x55d2d493f200 2024-09-19T11:23:21.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.040+0000 7effe011a640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d493f200 0x55d2d49415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe011a640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d493f200 0x55d2d49415f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe1e2f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe1e2f800 20 searching for the correct realm 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe011a640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7effd8007050 0x7effd8029730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd8007050 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.041+0000 7effdf919640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7effd8007050 0x7effd8029730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.042+0000 7effdf919640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7effd8007050 0x7effd8029730 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.042+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.042+0000 7effdf919640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7effd000f070 0x7effd004d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.042+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.042+0000 7effdf118640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7effd000f070 0x7effd004d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.043+0000 7effdf118640 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7effd000f070 0x7effd004d7f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.043+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.043+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.043+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.043+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.043+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.043+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.044+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.044+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.044+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.044+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd8007050 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.044+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.044+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.044+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd8007050 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.045+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.046+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.046+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.046+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v24'2 uv2 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.046+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.046+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.046+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.047+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd8007050 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.048+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.049+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.049+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.049+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.049+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.049+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.049+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd8007050 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.050+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.050+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493f200 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.050+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.050+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd000f070 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.050+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.050+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x55d2d493a7f0 2024-09-19T11:23:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943550 con 0x7effd8007050 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49425f0 con 0x55d2d493a7f0 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4942e40 con 0x55d2d493a7f0 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.051+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49439d0 con 0x55d2d493f200 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 zone default found 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 4 Realm: () 2024-09-19T11:23:21.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 zonegroup default 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4943dd0 con 0x7effd000f070 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49460b0 con 0x55d2d493f200 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.052+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4946450 con 0x55d2d493f200 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.053+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.053+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.053+0000 7effe1e2f800 20 started sync module instance, tier type = 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.053+0000 7effe1e2f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.053+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4945540 con 0x7effd8007050 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.054+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.054+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94363997985120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49451a0 con 0x7effd8007050 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.054+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4947350 con 0x55d2d493a7f0 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.055+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94363997985120] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.055+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.055+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94363997990560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4947770 con 0x55d2d493a7f0 2024-09-19T11:23:21.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.055+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4948130 con 0x55d2d493f200 2024-09-19T11:23:21.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.056+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94363997990560] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.056+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.056+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94363997992096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4948130 con 0x55d2d493f200 2024-09-19T11:23:21.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.056+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4948b50 con 0x7effd8007050 2024-09-19T11:23:21.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.057+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.057+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94363997996608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4949420 con 0x7effd8007050 2024-09-19T11:23:21.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.057+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94363997992096] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.057+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4949ee0 con 0x55d2d493a7f0 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.058+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.058+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94363998001680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494a300 con 0x55d2d493a7f0 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.058+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494ad40 con 0x7effd000f070 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.058+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94363997996608] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.059+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.059+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94363998005264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494ad00 con 0x7effd000f070 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.059+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494b790 con 0x7effd000f070 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.060+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94363998001680] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.060+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94363998005264] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.061+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.061+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94363997998960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494b2a0 con 0x7effd000f070 2024-09-19T11:23:21.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.061+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494bc60 con 0x7effd000f070 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94363997998960] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94363997994112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494bc60 con 0x7effd000f070 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 20 add_watcher() i=0 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 20 add_watcher() i=1 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 20 add_watcher() i=2 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 20 add_watcher() i=3 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 20 add_watcher() i=4 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 20 add_watcher() i=5 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.062+0000 7effe1e2f800 20 add_watcher() i=6 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.063+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94363997994112] v31'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.063+0000 7effe1e2f800 20 add_watcher() i=7 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.063+0000 7effe1e2f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:21.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.063+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494f080 con 0x55d2d493a7f0 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.063+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.063+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94363998009776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d494c580 con 0x55d2d493a7f0 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.064+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94363998009776] v31'31 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.064+0000 7efe95fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.064+0000 7effe1e2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.064+0000 7effe1e2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.064+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4957860 con 0x55d2d493a7f0 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.065+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.066+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4957ef0 con 0x7effd000f070 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.067+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'11 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.067+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958460 con 0x55d2d493f200 2024-09-19T11:23:21.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.068+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.068+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958460 con 0x55d2d493a7f0 2024-09-19T11:23:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.069+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'32 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.069+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958910 con 0x7effd8007050 2024-09-19T11:23:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.070+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'23 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.070+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958910 con 0x55d2d493f200 2024-09-19T11:23:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.071+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'15 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.071+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958a10 con 0x55d2d493a7f0 2024-09-19T11:23:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.072+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.072+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958e10 con 0x7effd000f070 2024-09-19T11:23:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.074+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'15 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.074+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958df0 con 0x7effd8007050 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.075+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.075+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958d20 con 0x55d2d493f200 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.076+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'17 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.076+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4958d20 con 0x7effd8007050 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.077+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'24 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.077+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959310 con 0x7effd000f070 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.078+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'13 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.078+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49591d0 con 0x7effd8007050 2024-09-19T11:23:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.079+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'11 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.079+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x7effd000f070 2024-09-19T11:23:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.080+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'21 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.080+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493a7f0 2024-09-19T11:23:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.081+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'33 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.081+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.082+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'17 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.082+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x7effd8007050 2024-09-19T11:23:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.083+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'25 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.083+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.084+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'11 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.084+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493a7f0 2024-09-19T11:23:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.085+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.085+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493a7f0 2024-09-19T11:23:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.086+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'34 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.086+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.088+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.088+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x7effd8007050 2024-09-19T11:23:21.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.089+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.089+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.089+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.089+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.090+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.090+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.091+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'7 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.091+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.092+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'16 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.092+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x7effd000f070 2024-09-19T11:23:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.094+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.094+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.095+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.095+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x7effd000f070 2024-09-19T11:23:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.096+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'16 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.096+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x55d2d493f200 2024-09-19T11:23:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.097+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'19 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.097+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x7effd000f070 2024-09-19T11:23:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.098+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'22 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.098+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4959140 con 0x7effd8007050 2024-09-19T11:23:21.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.099+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7efe94fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 20 Filter name: none 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7efe0e7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:94 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4983bc0 con 0x7effd000f070 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(94 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.100+0000 7effe1e2f800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-09-19T11:23:21.102 INFO:teuthology.orchestra.run.smithi028.stderr:could not remove user: unable to remove user, user does not exist 2024-09-19T11:23:21.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.101+0000 7efe95fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.101+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94363998009776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4955090 con 0x55d2d493a7f0 2024-09-19T11:23:21.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.102+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94363998009776] v31'35 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.102+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94363997985120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4955090 con 0x7effd8007050 2024-09-19T11:23:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.103+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94363997985120] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.103+0000 7effe1e2f800 20 remove_watcher() i=0 2024-09-19T11:23:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.103+0000 7effe1e2f800 2 removed watcher, disabling cache 2024-09-19T11:23:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.103+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94363997990560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4955090 con 0x55d2d493a7f0 2024-09-19T11:23:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.104+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94363997990560] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.105+0000 7effe1e2f800 20 remove_watcher() i=1 2024-09-19T11:23:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.105+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94363997992096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4955090 con 0x55d2d493f200 2024-09-19T11:23:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.105+0000 7effe011a640 1 -- 172.21.15.28:0/181216571 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94363997992096] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd800a4d0 con 0x55d2d493f200 2024-09-19T11:23:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.106+0000 7effe1e2f800 20 remove_watcher() i=2 2024-09-19T11:23:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.106+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94363997996608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49474a0 con 0x7effd8007050 2024-09-19T11:23:21.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.107+0000 7effdf919640 1 -- 172.21.15.28:0/181216571 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94363997996608] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd002e070 con 0x7effd8007050 2024-09-19T11:23:21.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.107+0000 7effe1e2f800 20 remove_watcher() i=3 2024-09-19T11:23:21.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.107+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94363998001680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4955090 con 0x55d2d493a7f0 2024-09-19T11:23:21.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.108+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94363998001680] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x55d2d493a7f0 2024-09-19T11:23:21.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.108+0000 7effe1e2f800 20 remove_watcher() i=4 2024-09-19T11:23:21.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.108+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94363998005264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d49474a0 con 0x7effd000f070 2024-09-19T11:23:21.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.109+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94363998005264] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.109+0000 7effe1e2f800 20 remove_watcher() i=5 2024-09-19T11:23:21.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.109+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94363997998960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4949a10 con 0x7effd000f070 2024-09-19T11:23:21.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.110+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94363997998960] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.110+0000 7effe1e2f800 20 remove_watcher() i=6 2024-09-19T11:23:21.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.110+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94363997994112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2d4955090 con 0x7effd000f070 2024-09-19T11:23:21.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94363997994112] v31'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7effd4057030 con 0x7effd000f070 2024-09-19T11:23:21.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 20 remove_watcher() i=7 2024-09-19T11:23:21.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7effd000f070 msgr2=0x7effd004d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7effd000f070 0x7effd004d7f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d493a7f0 msgr2=0x55d2d493ccb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d493a7f0 0x55d2d493ccb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7effd8007050 msgr2=0x7effd8029730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7effd8007050 0x7effd8029730 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d493f200 msgr2=0x55d2d49415f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d493f200 0x55d2d49415f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efea803d000 msgr2=0x7efea803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efea803d000 0x7efea803f4c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7effd000bfb0 tx=0x7effd0051000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effdf118640 1 -- 172.21.15.28:0/181216571 reap_dead start 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 msgr2=0x55d2d492f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d492f090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7effd8054840 tx=0x7effd80547d0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 shutdown_connections 2024-09-19T11:23:21.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 --2- 172.21.15.28:0/181216571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4927aa0 0x55d2d492f090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 >> 172.21.15.28:0/181216571 conn(0x55d2d4923ec0 msgr2=0x55d2d49220d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 shutdown_connections 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.111+0000 7effe1e2f800 1 -- 172.21.15.28:0/181216571 wait complete. 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efeb803d000 msgr2=0x7efeb803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efeb803d000 0x7efeb803f4c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7effd0001530 tx=0x7effd001c790 comp rx=0 tx=0).stop 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d491b030 msgr2=0x55d2d491b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d491b030 0x55d2d491b450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55d2d480f060 tx=0x7effd80167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 shutdown_connections 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efeb803d000 0x7efeb803f4c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/596231503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d491b030 0x55d2d491b450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 >> 172.21.15.28:0/596231503 conn(0x55d2d490f3e0 msgr2=0x55d2d490f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 shutdown_connections 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/596231503 wait complete. 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d4806aa0 msgr2=0x55d2d4907390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d4806aa0 0x55d2d4907390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d4909c20 msgr2=0x55d2d490c010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d4909c20 0x55d2d490c010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efec803d000 msgr2=0x7efec803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efec803d000 0x7efec803f4c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7effd4057000 tx=0x7effd4052000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 msgr2=0x55d2d480f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d480f4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x55d2d48f72b0 tx=0x7effd0000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 shutdown_connections 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2d4909c20 0x55d2d490c010 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efec803d000 0x7efec803f4c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2d4806aa0 0x55d2d4907390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 --2- 172.21.15.28:0/4063119670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2d4805690 0x55d2d480f4c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 >> 172.21.15.28:0/4063119670 conn(0x55d2d4800a60 msgr2=0x55d2d48f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 shutdown_connections 2024-09-19T11:23:21.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.112+0000 7effe1e2f800 1 -- 172.21.15.28:0/4063119670 wait complete. 2024-09-19T11:23:21.122 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T11:23:21.123 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-09-19T11:23:21.123 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-09-19T11:23:21.123 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-09-19T11:23:21.169 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:21.169 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:21.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.168+0000 7f7469d2a800 1 Processor -- start 2024-09-19T11:23:21.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.168+0000 7f7469d2a800 1 -- start start 2024-09-19T11:23:21.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.168+0000 7f7469d2a800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55bc43c81f20 0x55bc43c84310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.168+0000 7f7469d2a800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55bc43c885d0 con 0x55bc43c848b0 2024-09-19T11:23:21.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.168+0000 7f7469d2a800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55bc43b0c7d0 con 0x55bc43c81f20 2024-09-19T11:23:21.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.169+0000 7f7465aad640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55bc43c81f20 0x55bc43c84310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.169+0000 7f74652ac640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55bc43c848b0 0x55bc43c86d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32934/0 (socket says 172.21.15.28:32934) 2024-09-19T11:23:21.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.169+0000 7f74652ac640 1 -- 172.21.15.28:0/1324090722 learned_addr learned my addr 172.21.15.28:0/1324090722 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:21.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.169+0000 7f7465aad640 1 -- 172.21.15.28:0/1324090722 >> v1:172.21.15.22:6789/0 conn(0x55bc43c848b0 legacy=0x55bc43c86d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.169+0000 7f7465aad640 1 -- 172.21.15.28:0/1324090722 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43c89800 con 0x55bc43c81f20 2024-09-19T11:23:21.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7465aad640 1 --2- 172.21.15.28:0/1324090722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c84310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7458005fd0 tx=0x7f7458045c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5ae0778727286 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7464aab640 1 -- 172.21.15.28:0/1324090722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458058530 con 0x55bc43c81f20 2024-09-19T11:23:21.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7464aab640 1 -- 172.21.15.28:0/1324090722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74580586d0 con 0x55bc43c81f20 2024-09-19T11:23:21.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7464aab640 1 -- 172.21.15.28:0/1324090722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74580589d0 con 0x55bc43c81f20 2024-09-19T11:23:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7469d2a800 1 -- 172.21.15.28:0/1324090722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 msgr2=0x55bc43c84310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1324090722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c84310 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7458005fd0 tx=0x7f7458045c30 comp rx=0 tx=0).stop 2024-09-19T11:23:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7469d2a800 1 -- 172.21.15.28:0/1324090722 shutdown_connections 2024-09-19T11:23:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1324090722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c84310 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7469d2a800 1 -- 172.21.15.28:0/1324090722 >> 172.21.15.28:0/1324090722 conn(0x55bc43c7ba60 msgr2=0x55bc43c7dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7469d2a800 1 -- 172.21.15.28:0/1324090722 shutdown_connections 2024-09-19T11:23:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.171+0000 7f7469d2a800 1 -- 172.21.15.28:0/1324090722 wait complete. 2024-09-19T11:23:21.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.186+0000 7f7469d2a800 1 Processor -- start 2024-09-19T11:23:21.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.187+0000 7f7469d2a800 1 -- start start 2024-09-19T11:23:21.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.187+0000 7f7469d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43d71330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.187+0000 7f7469d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc43b19dd0 con 0x55bc43c81f20 2024-09-19T11:23:21.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.187+0000 7f7465aad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43d71330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.187+0000 7f7465aad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43d71330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58614/0 (socket says 172.21.15.28:58614) 2024-09-19T11:23:21.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.187+0000 7f7465aad640 1 -- 172.21.15.28:0/2744879965 learned_addr learned my addr 172.21.15.28:0/2744879965 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.187+0000 7f7465aad640 1 -- 172.21.15.28:0/2744879965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43d71920 con 0x55bc43c81f20 2024-09-19T11:23:21.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7465aad640 1 --2- 172.21.15.28:0/2744879965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43d71330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f745800a3f0 tx=0x7f7458051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c8eb44a87a414 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7353e07640 1 -- 172.21.15.28:0/2744879965 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458058a30 con 0x55bc43c81f20 2024-09-19T11:23:21.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7353e07640 1 -- 172.21.15.28:0/2744879965 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7458068070 con 0x55bc43c81f20 2024-09-19T11:23:21.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7353e07640 1 -- 172.21.15.28:0/2744879965 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458058d60 con 0x55bc43c81f20 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7469d2a800 1 -- 172.21.15.28:0/2744879965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 msgr2=0x55bc43d71330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7469d2a800 1 --2- 172.21.15.28:0/2744879965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43d71330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f745800a3f0 tx=0x7f7458051000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7469d2a800 1 -- 172.21.15.28:0/2744879965 shutdown_connections 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7469d2a800 1 --2- 172.21.15.28:0/2744879965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43d71330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.188+0000 7f7469d2a800 1 -- 172.21.15.28:0/2744879965 >> 172.21.15.28:0/2744879965 conn(0x55bc43c7ba60 msgr2=0x55bc43d6ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7469d2a800 1 -- 172.21.15.28:0/2744879965 shutdown_connections 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7469d2a800 1 -- 172.21.15.28:0/2744879965 wait complete. 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7469d2a800 1 Processor -- start 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7469d2a800 1 -- start start 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7469d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c81620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7469d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc43c88aa0 con 0x55bc43c81f20 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7465aad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c81620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7465aad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c81620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58628/0 (socket says 172.21.15.28:58628) 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.189+0000 7f7465aad640 1 -- 172.21.15.28:0/1780920825 learned_addr learned my addr 172.21.15.28:0/1780920825 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.190+0000 7f7465aad640 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43c81b80 con 0x55bc43c81f20 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.190+0000 7f7465aad640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c81620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7458005bf0 tx=0x7f7458003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.190+0000 7f7352604640 1 -- 172.21.15.28:0/1780920825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458063680 con 0x55bc43c81f20 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.190+0000 7f7352604640 1 -- 172.21.15.28:0/1780920825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7458063820 con 0x55bc43c81f20 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.190+0000 7f7352604640 1 -- 172.21.15.28:0/1780920825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7458063b40 con 0x55bc43c81f20 2024-09-19T11:23:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.190+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc43c7ff30 con 0x55bc43c81f20 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.190+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc43c80490 con 0x55bc43c81f20 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.191+0000 7f7352604640 1 -- 172.21.15.28:0/1780920825 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7458063ce0 con 0x55bc43c81f20 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.191+0000 7f7352604640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734c03d050 0x7f734c03f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.191+0000 7f7352604640 1 -- 172.21.15.28:0/1780920825 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f74580988d0 con 0x55bc43c81f20 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.191+0000 7f74652ac640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734c03d050 0x7f734c03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.191+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43c848b0 0x55bc43d82170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.191+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43d82790 con 0x55bc43c848b0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f74662ae640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43c848b0 0x55bc43d82170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f74652ac640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734c03d050 0x7f734c03f510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f745c001d70 tx=0x7f745c0012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f74662ae640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43c848b0 0x55bc43d82170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f74662ae640 1 -- 172.21.15.28:0/1780920825 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f74600083e0 con 0x55bc43c848b0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43d83050 con 0x55bc43c848b0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f74662ae640 1 -- 172.21.15.28:0/1780920825 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f74600083e0 con 0x55bc43c848b0 2024-09-19T11:23:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43d84a80 0x55bc43d86ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.192+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43d87680 con 0x55bc43d84a80 2024-09-19T11:23:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.193+0000 7f7465aad640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43d84a80 0x55bc43d86ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.193+0000 7f7465aad640 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43d84a80 0x55bc43d86ef0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.193+0000 7f7465aad640 1 -- 172.21.15.28:0/1780920825 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7458092040 con 0x55bc43d84a80 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.193+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43d838a0 con 0x55bc43c848b0 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.193+0000 7f74662ae640 1 -- 172.21.15.28:0/1780920825 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f74600083e0 con 0x55bc43c848b0 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.194+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43d83c40 con 0x55bc43d84a80 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.194+0000 7f7465aad640 1 -- 172.21.15.28:0/1780920825 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7458092040 con 0x55bc43d84a80 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.194+0000 7f7469d2a800 1 Processor -- start 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.194+0000 7f7469d2a800 1 -- start start 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.194+0000 7f7469d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43d95ef0 0x55bc43d96310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.194+0000 7f7469d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc43b784a0 con 0x55bc43d95ef0 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.194+0000 7f74662ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43d95ef0 0x55bc43d96310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f74662ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43d95ef0 0x55bc43d96310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58632/0 (socket says 172.21.15.28:58632) 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f74662ae640 1 -- 172.21.15.28:0/572860046 learned_addr learned my addr 172.21.15.28:0/572860046 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f74662ae640 1 -- 172.21.15.28:0/572860046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43d96950 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f74662ae640 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43d95ef0 0x55bc43d96310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55bc43c81050 tx=0x7f746000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f734b7fe640 1 -- 172.21.15.28:0/572860046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f746001c020 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f734b7fe640 1 -- 172.21.15.28:0/572860046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7460007c90 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f734b7fe640 1 -- 172.21.15.28:0/572860046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460018440 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc43d97180 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.195+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc43d97720 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.196+0000 7f734b7fe640 1 -- 172.21.15.28:0/572860046 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f746001f020 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.196+0000 7f734b7fe640 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734003d000 0x7f734003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.196+0000 7f734b7fe640 1 -- 172.21.15.28:0/572860046 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f7460055a00 con 0x55bc43d95ef0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.196+0000 7f7465aad640 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734003d000 0x7f734003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f7469d2a800 1 Processor -- start 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f7469d2a800 1 -- start start 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f7465aad640 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734003d000 0x7f734003f4c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55bc43c80bb0 tx=0x7f7458098040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f7469d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f7469d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc43d71c90 con 0x55bc43da2a10 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f74662ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f74662ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58644/0 (socket says 172.21.15.28:58644) 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f74662ae640 1 -- 172.21.15.28:0/3292980651 learned_addr learned my addr 172.21.15.28:0/3292980651 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f74662ae640 1 -- 172.21.15.28:0/3292980651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43da3400 con 0x55bc43da2a10 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.197+0000 7f74662ae640 1 --2- 172.21.15.28:0/3292980651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da2e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7460053d50 tx=0x7f7460053f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52957ef85f3138a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f73497fa640 1 -- 172.21.15.28:0/3292980651 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460052030 con 0x55bc43da2a10 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f73497fa640 1 -- 172.21.15.28:0/3292980651 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7460015d00 con 0x55bc43da2a10 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f73497fa640 1 -- 172.21.15.28:0/3292980651 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460016c50 con 0x55bc43da2a10 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 -- 172.21.15.28:0/3292980651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 msgr2=0x55bc43da2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3292980651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da2e10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7460053d50 tx=0x7f7460053f40 comp rx=0 tx=0).stop 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 -- 172.21.15.28:0/3292980651 shutdown_connections 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3292980651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da2e10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 -- 172.21.15.28:0/3292980651 >> 172.21.15.28:0/3292980651 conn(0x55bc43d9ee10 msgr2=0x55bc43d9d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 -- 172.21.15.28:0/3292980651 shutdown_connections 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 -- 172.21.15.28:0/3292980651 wait complete. 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 Processor -- start 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.198+0000 7f7469d2a800 1 -- start start 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f7469d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f7469d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc43da3a20 con 0x55bc43da2a10 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f74662ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f74662ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58648/0 (socket says 172.21.15.28:58648) 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 learned_addr learned my addr 172.21.15.28:0/3106948067 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc43daa530 con 0x55bc43da2a10 2024-09-19T11:23:21.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f74662ae640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da9fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7460017f90 tx=0x7f7460053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f73377fe640 1 -- 172.21.15.28:0/3106948067 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7460052050 con 0x55bc43da2a10 2024-09-19T11:23:21.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f73377fe640 1 -- 172.21.15.28:0/3106948067 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f746002e030 con 0x55bc43da2a10 2024-09-19T11:23:21.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc43daa7c0 con 0x55bc43da2a10 2024-09-19T11:23:21.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.199+0000 7f73377fe640 1 -- 172.21.15.28:0/3106948067 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74600139e0 con 0x55bc43da2a10 2024-09-19T11:23:21.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.200+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc43dab350 con 0x55bc43da2a10 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.200+0000 7f73377fe640 1 -- 172.21.15.28:0/3106948067 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7460016040 con 0x55bc43da2a10 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.200+0000 7f73377fe640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f733003d000 0x7f733003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.201+0000 7f73377fe640 1 -- 172.21.15.28:0/3106948067 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f746004a030 con 0x55bc43da2a10 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.201+0000 7f7465aad640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f733003d000 0x7f733003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.201+0000 7f7465aad640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f733003d000 0x7f733003f4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7458059fb0 tx=0x7f745808a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.201+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.201+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43db5960 0x55bc43db7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.201+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43db8570 con 0x55bc43db5960 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.201+0000 7f74652ac640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43db5960 0x55bc43db7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f74652ac640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43db5960 0x55bc43db7e20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f745c009050 con 0x55bc43db5960 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 20 realm 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43db4cb0 con 0x55bc43db5960 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f745c009050 con 0x55bc43db5960 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.202+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43db9060 con 0x55bc43db5960 2024-09-19T11:23:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f745c009050 con 0x55bc43db5960 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43db9930 con 0x55bc43db5960 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f745c009050 con 0x55bc43db5960 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f7469d2a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43dba350 0x55bc43dbc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbce80 con 0x55bc43dba350 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f74662ae640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43dba350 0x55bc43dbc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f74662ae640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43dba350 0x55bc43dbc740 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.203+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f7469d2a800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:21.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f7469d2a800 20 searching for the correct realm 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f74662ae640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7460021020 0x7f746001f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7460021020 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f7465aad640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7460021020 0x7f746001f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.204+0000 7f7465aad640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7460021020 0x7f746001f310 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.205+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.205+0000 7f7465aad640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7458095040 0x7f745804e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.205+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.205+0000 7f74652ac640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7458095040 0x7f745804e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.205+0000 7f74652ac640 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7458095040 0x7f745804e350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.205+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c009050 con 0x7f7458095040 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.205+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c009050 con 0x55bc43db5960 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.206+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7460021020 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.207+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7460021020 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v24'2 uv2 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.208+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.209+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7460021020 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.210+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.211+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7460021020 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43dba350 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7458095040 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x55bc43db5960 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.212+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe7b0 con 0x7f7460021020 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbd840 con 0x55bc43db5960 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbe090 con 0x55bc43db5960 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbec30 con 0x55bc43dba350 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.213+0000 7f7469d2a800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 zone default found 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 4 Realm: () 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 zonegroup default 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dbefd0 con 0x7f7458095040 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc12b0 con 0x55bc43dba350 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc1680 con 0x55bc43dba350 2024-09-19T11:23:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 started sync module instance, tier type = 2024-09-19T11:23:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.214+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc06f0 con 0x7f7460021020 2024-09-19T11:23:21.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.215+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.215+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94267080704576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc1ae0 con 0x7f7460021020 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.215+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc2330 con 0x55bc43db5960 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.216+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.217+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94267080712080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc29e0 con 0x55bc43db5960 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.217+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc3500 con 0x55bc43dba350 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.217+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94267080704576] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.218+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.218+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94267080716240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc29e0 con 0x55bc43dba350 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.218+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc3fc0 con 0x7f7460021020 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.218+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94267080712080] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.219+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94267080716240] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.219+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.219+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94267080706832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc2430 con 0x7f7460021020 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.219+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc4860 con 0x55bc43db5960 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.220+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.220+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94267080721840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc5130 con 0x55bc43db5960 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.220+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc5bd0 con 0x7f7458095040 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.220+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94267080706832] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.221+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.221+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94267080726208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc5b90 con 0x7f7458095040 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.221+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc6610 con 0x7f7458095040 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.221+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94267080721840] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.222+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.222+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94267080723584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc6610 con 0x7f7458095040 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.222+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc7010 con 0x7f7458095040 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.222+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94267080726208] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94267080723584] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94267080718992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc6610 con 0x7f7458095040 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 20 add_watcher() i=0 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 20 add_watcher() i=1 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 20 add_watcher() i=2 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 20 add_watcher() i=3 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 20 add_watcher() i=4 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 20 add_watcher() i=5 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.223+0000 7f7469d2a800 20 add_watcher() i=6 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.224+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94267080718992] v31'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.224+0000 7f7469d2a800 20 add_watcher() i=7 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.224+0000 7f7469d2a800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.224+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc9ea0 con 0x55bc43db5960 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.224+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.225+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94267080729392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc73f0 con 0x55bc43db5960 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.225+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94267080729392] v31'36 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.226+0000 7f73177fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.226+0000 7f7469d2a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.226+0000 7f7469d2a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.226+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd2800 con 0x55bc43db5960 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.227+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.227+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd2800 con 0x7f7458095040 2024-09-19T11:23:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.228+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.228+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3220 con 0x55bc43dba350 2024-09-19T11:23:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.229+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.229+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd32b0 con 0x55bc43db5960 2024-09-19T11:23:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.230+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'37 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.230+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd36f0 con 0x7f7460021020 2024-09-19T11:23:21.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.231+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'26 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.231+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3a60 con 0x55bc43dba350 2024-09-19T11:23:21.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.232+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'17 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.232+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3a60 con 0x55bc43db5960 2024-09-19T11:23:21.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.233+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.233+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3a60 con 0x7f7458095040 2024-09-19T11:23:21.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.234+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'17 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.234+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd4070 con 0x7f7460021020 2024-09-19T11:23:21.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.235+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.235+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd4050 con 0x55bc43dba350 2024-09-19T11:23:21.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.236+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'18 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.236+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd4050 con 0x7f7460021020 2024-09-19T11:23:21.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.237+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'27 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.237+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x7f7458095040 2024-09-19T11:23:21.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.239+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.239+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x7f7460021020 2024-09-19T11:23:21.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.240+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.240+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x7f7458095040 2024-09-19T11:23:21.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.241+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'23 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.241+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x55bc43db5960 2024-09-19T11:23:21.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.242+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'38 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.242+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x55bc43dba350 2024-09-19T11:23:21.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.243+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'20 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.243+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x7f7460021020 2024-09-19T11:23:21.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.245+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'28 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.245+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x55bc43dba350 2024-09-19T11:23:21.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.246+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'12 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.246+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x55bc43db5960 2024-09-19T11:23:21.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.247+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.247+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3eb0 con 0x55bc43db5960 2024-09-19T11:23:21.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.248+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'39 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.248+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3eb0 con 0x55bc43dba350 2024-09-19T11:23:21.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.249+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.249+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3e80 con 0x7f7460021020 2024-09-19T11:23:21.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.250+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.250+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3ea0 con 0x55bc43dba350 2024-09-19T11:23:21.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.251+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.251+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3ef0 con 0x55bc43dba350 2024-09-19T11:23:21.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.252+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.252+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3f30 con 0x55bc43dba350 2024-09-19T11:23:21.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.254+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'8 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.254+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3f30 con 0x55bc43dba350 2024-09-19T11:23:21.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.255+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'18 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.255+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3f80 con 0x7f7458095040 2024-09-19T11:23:21.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.256+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.256+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3fc0 con 0x55bc43dba350 2024-09-19T11:23:21.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.257+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.257+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3fc0 con 0x7f7458095040 2024-09-19T11:23:21.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.258+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'18 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.258+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3fc0 con 0x55bc43dba350 2024-09-19T11:23:21.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.259+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'22 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.259+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3fc0 con 0x7f7458095040 2024-09-19T11:23:21.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.261+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'24 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.261+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd3fc0 con 0x7f7460021020 2024-09-19T11:23:21.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f73167fc640 20 reqs_thread_entry: start 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f7469d2a800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f7469d2a800 20 Filter name: none 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f7287fff640 20 reqs_thread_entry: start 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f7469d2a800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f7469d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.262+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dfeb80 con 0x7f7458095040 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.263+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(94 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.263+0000 7f7469d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.263+0000 7f7469d2a800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:could not remove user: unable to remove user, user does not exist2024-09-19T11:23:21.263+0000 7f7469d2a800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-19T11:23:21.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.263+0000 7f73177fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.263+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94267080729392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dd1810 con 0x55bc43db5960 2024-09-19T11:23:21.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.265+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94267080729392] v31'40 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.265+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94267080704576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc6300 con 0x7f7460021020 2024-09-19T11:23:21.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.266+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94267080704576] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.266+0000 7f7469d2a800 20 remove_watcher() i=0 2024-09-19T11:23:21.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.266+0000 7f7469d2a800 2 removed watcher, disabling cache 2024-09-19T11:23:21.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.266+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94267080712080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc0240 con 0x55bc43db5960 2024-09-19T11:23:21.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.267+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94267080712080] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.267+0000 7f7469d2a800 20 remove_watcher() i=1 2024-09-19T11:23:21.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.267+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94267080716240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc1f90 con 0x55bc43dba350 2024-09-19T11:23:21.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.268+0000 7f74662ae640 1 -- 172.21.15.28:0/3106948067 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94267080716240] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f746003a070 con 0x55bc43dba350 2024-09-19T11:23:21.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.268+0000 7f7469d2a800 20 remove_watcher() i=2 2024-09-19T11:23:21.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.268+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94267080706832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc2fd0 con 0x7f7460021020 2024-09-19T11:23:21.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.269+0000 7f7465aad640 1 -- 172.21.15.28:0/3106948067 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94267080706832] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7458092040 con 0x7f7460021020 2024-09-19T11:23:21.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.269+0000 7f7469d2a800 20 remove_watcher() i=3 2024-09-19T11:23:21.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.269+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94267080721840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc2fd0 con 0x55bc43db5960 2024-09-19T11:23:21.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.270+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94267080721840] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x55bc43db5960 2024-09-19T11:23:21.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.270+0000 7f7469d2a800 20 remove_watcher() i=4 2024-09-19T11:23:21.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.270+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94267080726208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc2fd0 con 0x7f7458095040 2024-09-19T11:23:21.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.271+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94267080726208] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.271+0000 7f7469d2a800 20 remove_watcher() i=5 2024-09-19T11:23:21.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.271+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94267080723584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc56c0 con 0x7f7458095040 2024-09-19T11:23:21.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.272+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94267080723584] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.272+0000 7f7469d2a800 20 remove_watcher() i=6 2024-09-19T11:23:21.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.272+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94267080718992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bc43dc4c80 con 0x7f7458095040 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94267080718992] v31'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f745c029040 con 0x7f7458095040 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 20 remove_watcher() i=7 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7458095040 msgr2=0x7f745804e350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7458095040 0x7f745804e350 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43db5960 msgr2=0x55bc43db7e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43db5960 0x55bc43db7e20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7460021020 msgr2=0x7f746001f310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7460021020 0x7f746001f310 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43dba350 msgr2=0x55bc43dbc740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43dba350 0x55bc43dbc740 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f733003d000 msgr2=0x7f733003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f733003d000 0x7f733003f4c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7458059fb0 tx=0x7f745808a000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f74652ac640 1 -- 172.21.15.28:0/3106948067 reap_dead start 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 msgr2=0x55bc43da9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da9fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7460017f90 tx=0x7f7460053ce0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 shutdown_connections 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 --2- 172.21.15.28:0/3106948067 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43da2a10 0x55bc43da9fd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.273+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 >> 172.21.15.28:0/3106948067 conn(0x55bc43d9ee10 msgr2=0x55bc43d9d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 shutdown_connections 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/3106948067 wait complete. 2024-09-19T11:23:21.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734003d000 msgr2=0x7f734003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734003d000 0x7f734003f4c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55bc43c80bb0 tx=0x7f7458098040 comp rx=0 tx=0).stop 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43d95ef0 msgr2=0x55bc43d96310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43d95ef0 0x55bc43d96310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55bc43c81050 tx=0x7f746000bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 shutdown_connections 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734003d000 0x7f734003f4c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 --2- 172.21.15.28:0/572860046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43d95ef0 0x55bc43d96310 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 >> 172.21.15.28:0/572860046 conn(0x55bc43d8a2a0 msgr2=0x55bc43d8a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 shutdown_connections 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/572860046 wait complete. 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43c848b0 msgr2=0x55bc43d82170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43c848b0 0x55bc43d82170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43d84a80 msgr2=0x55bc43d86ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43d84a80 0x55bc43d86ef0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734c03d050 msgr2=0x7f734c03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.274+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734c03d050 0x7f734c03f510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f745c001d70 tx=0x7f745c0012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 msgr2=0x55bc43c81620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c81620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7458005bf0 tx=0x7f7458003330 comp rx=0 tx=0).stop 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 shutdown_connections 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bc43d84a80 0x55bc43d86ef0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f734c03d050 0x7f734c03f510 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bc43c848b0 0x55bc43d82170 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 --2- 172.21.15.28:0/1780920825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc43c81f20 0x55bc43c81620 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 >> 172.21.15.28:0/1780920825 conn(0x55bc43c7ba60 msgr2=0x55bc43d72720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 shutdown_connections 2024-09-19T11:23:21.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.275+0000 7f7469d2a800 1 -- 172.21.15.28:0/1780920825 wait complete. 2024-09-19T11:23:21.285 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T11:23:21.285 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-09-19T11:23:21.286 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-09-19T11:23:21.286 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-09-19T11:23:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.330+0000 7f6699346800 1 Processor -- start 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f6699346800 1 -- start start 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f6699346800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561e3a4a5880 0x561e3a4ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f6699346800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561e3a4ae8e0 con 0x561e3a4a3d60 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f6699346800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x561e3a3307d0 con 0x561e3a4a5880 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f668ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561e3a4a5880 0x561e3a4ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f668ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561e3a4a5880 0x561e3a4ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58664/0 (socket says 172.21.15.28:58664) 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f668ffff640 1 -- 172.21.15.28:0/2725380044 learned_addr learned my addr 172.21.15.28:0/2725380044 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f6696e30640 1 --1- 172.21.15.28:0/2725380044 >> v1:172.21.15.22:6789/0 conn(0x561e3a4a3d60 0x561e3a4a4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32946/0 (socket says 172.21.15.28:32946) 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f668ffff640 1 -- 172.21.15.28:0/2725380044 >> v1:172.21.15.22:6789/0 conn(0x561e3a4a3d60 legacy=0x561e3a4a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f669662f640 1 -- 172.21.15.28:0/2725380044 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708281072 0 0) 0x561e3a4ae8e0 con 0x561e3a4a3d60 2024-09-19T11:23:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.331+0000 7f668ffff640 1 -- 172.21.15.28:0/2725380044 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3a4afac0 con 0x561e3a4a5880 2024-09-19T11:23:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.333+0000 7f668ffff640 1 --2- 172.21.15.28:0/2725380044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a5880 0x561e3a4ae3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f668000a520 tx=0x7f6680045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6482eab00c8611 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.333+0000 7f669662f640 1 -- 172.21.15.28:0/2725380044 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6680055070 con 0x561e3a4a5880 2024-09-19T11:23:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.333+0000 7f669662f640 1 -- 172.21.15.28:0/2725380044 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66800515b0 con 0x561e3a4a5880 2024-09-19T11:23:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.333+0000 7f669662f640 1 -- 172.21.15.28:0/2725380044 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66800518d0 con 0x561e3a4a5880 2024-09-19T11:23:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.334+0000 7f6699346800 1 -- 172.21.15.28:0/2725380044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a5880 msgr2=0x561e3a4ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.334+0000 7f6699346800 1 --2- 172.21.15.28:0/2725380044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a5880 0x561e3a4ae3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f668000a520 tx=0x7f6680045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.334+0000 7f6699346800 1 -- 172.21.15.28:0/2725380044 shutdown_connections 2024-09-19T11:23:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.334+0000 7f6699346800 1 --2- 172.21.15.28:0/2725380044 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a5880 0x561e3a4ae3a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.334+0000 7f6699346800 1 -- 172.21.15.28:0/2725380044 >> 172.21.15.28:0/2725380044 conn(0x561e3a49fa50 msgr2=0x561e3a4a1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.334+0000 7f6699346800 1 -- 172.21.15.28:0/2725380044 shutdown_connections 2024-09-19T11:23:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.334+0000 7f6699346800 1 -- 172.21.15.28:0/2725380044 wait complete. 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.349+0000 7f6699346800 1 Processor -- start 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.349+0000 7f6699346800 1 -- start start 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.349+0000 7f6699346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a5972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.349+0000 7f6699346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561e3a33ddd0 con 0x561e3a4a3d60 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6696e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a5972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6696e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a5972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58670/0 (socket says 172.21.15.28:58670) 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6696e30640 1 -- 172.21.15.28:0/2016219345 learned_addr learned my addr 172.21.15.28:0/2016219345 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6696e30640 1 -- 172.21.15.28:0/2016219345 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3a5978b0 con 0x561e3a4a3d60 2024-09-19T11:23:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6696e30640 1 --2- 172.21.15.28:0/2016219345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a5972c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6688001ea0 tx=0x7f668800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20daca5674c923db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6584e09640 1 -- 172.21.15.28:0/2016219345 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6688008070 con 0x561e3a4a3d60 2024-09-19T11:23:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6584e09640 1 -- 172.21.15.28:0/2016219345 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668800d500 con 0x561e3a4a3d60 2024-09-19T11:23:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.350+0000 7f6584e09640 1 -- 172.21.15.28:0/2016219345 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f668800d880 con 0x561e3a4a3d60 2024-09-19T11:23:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 -- 172.21.15.28:0/2016219345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 msgr2=0x561e3a5972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 --2- 172.21.15.28:0/2016219345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a5972c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6688001ea0 tx=0x7f668800b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 -- 172.21.15.28:0/2016219345 shutdown_connections 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 --2- 172.21.15.28:0/2016219345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a5972c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 -- 172.21.15.28:0/2016219345 >> 172.21.15.28:0/2016219345 conn(0x561e3a49fa50 msgr2=0x561e3a592c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 -- 172.21.15.28:0/2016219345 shutdown_connections 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 -- 172.21.15.28:0/2016219345 wait complete. 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 Processor -- start 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 -- start start 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a59edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.351+0000 7f6699346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561e3a4aedb0 con 0x561e3a4a3d60 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6696e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a59edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6696e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a59edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58684/0 (socket says 172.21.15.28:58684) 2024-09-19T11:23:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6696e30640 1 -- 172.21.15.28:0/3137720055 learned_addr learned my addr 172.21.15.28:0/3137720055 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6696e30640 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3a59f330 con 0x561e3a4a3d60 2024-09-19T11:23:21.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6696e30640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a59edd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f66880012f0 tx=0x7f6688000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6583606640 1 -- 172.21.15.28:0/3137720055 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f668800f9d0 con 0x561e3a4a3d60 2024-09-19T11:23:21.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6583606640 1 -- 172.21.15.28:0/3137720055 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668800fb70 con 0x561e3a4a3d60 2024-09-19T11:23:21.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561e3a59f5c0 con 0x561e3a4a3d60 2024-09-19T11:23:21.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6583606640 1 -- 172.21.15.28:0/3137720055 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f668800fe50 con 0x561e3a4a3d60 2024-09-19T11:23:21.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.352+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e3a59ffc0 con 0x561e3a4a3d60 2024-09-19T11:23:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.353+0000 7f6583606640 1 -- 172.21.15.28:0/3137720055 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6688017040 con 0x561e3a4a3d60 2024-09-19T11:23:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.353+0000 7f6583606640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657c03d1f0 0x7f657c03f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.353+0000 7f6583606640 1 -- 172.21.15.28:0/3137720055 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f6688015020 con 0x561e3a4a3d60 2024-09-19T11:23:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f668ffff640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657c03d1f0 0x7f657c03f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a4a5880 0x561e3a5a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5a6960 con 0x561e3a4a5880 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f6697631640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a4a5880 0x561e3a5a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f668ffff640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657c03d1f0 0x7f657c03f6b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6680057000 tx=0x7f6680052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f6697631640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a4a5880 0x561e3a5a6420 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f6697631640 1 -- 172.21.15.28:0/3137720055 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f669000a4d0 con 0x561e3a4a5880 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.354+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5a6d00 con 0x561e3a4a5880 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.355+0000 7f6697631640 1 -- 172.21.15.28:0/3137720055 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f669000a4d0 con 0x561e3a4a5880 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.355+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5a8800 0x561e3a5aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.355+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5ab410 con 0x561e3a5a8800 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.355+0000 7f6696e30640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5a8800 0x561e3a5aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.355+0000 7f6696e30640 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5a8800 0x561e3a5aac80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.355+0000 7f6696e30640 1 -- 172.21.15.28:0/3137720055 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6688054040 con 0x561e3a5a8800 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.355+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5a7550 con 0x561e3a4a5880 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.356+0000 7f6697631640 1 -- 172.21.15.28:0/3137720055 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f669000a4d0 con 0x561e3a4a5880 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.356+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5a79a0 con 0x561e3a5a8800 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.356+0000 7f6696e30640 1 -- 172.21.15.28:0/3137720055 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6688054040 con 0x561e3a5a8800 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.356+0000 7f6699346800 1 Processor -- start 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.356+0000 7f6699346800 1 -- start start 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.356+0000 7f6699346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5b9da0 0x561e3a5ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6699346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561e3a39c4a0 con 0x561e3a5b9da0 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6697631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5b9da0 0x561e3a5ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6697631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5b9da0 0x561e3a5ba1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58694/0 (socket says 172.21.15.28:58694) 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6697631640 1 -- 172.21.15.28:0/3446457283 learned_addr learned my addr 172.21.15.28:0/3446457283 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6697631640 1 -- 172.21.15.28:0/3446457283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3a5ba800 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6697631640 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5b9da0 0x561e3a5ba1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x561e3a59fdd0 tx=0x7f66900167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6580e01640 1 -- 172.21.15.28:0/3446457283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690013640 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6580e01640 1 -- 172.21.15.28:0/3446457283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66900137e0 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561e3a5bb030 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6580e01640 1 -- 172.21.15.28:0/3446457283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690013ac0 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.357+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e3a5bb5d0 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.358+0000 7f6580e01640 1 -- 172.21.15.28:0/3446457283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6690011040 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.358+0000 7f6580e01640 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657003d000 0x7f657003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6580e01640 1 -- 172.21.15.28:0/3446457283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f669000e040 con 0x561e3a5b9da0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6696e30640 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657003d000 0x7f657003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6699346800 1 Processor -- start 2024-09-19T11:23:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6696e30640 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657003d000 0x7f657003f4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f668800ecb0 tx=0x7f6688001460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6699346800 1 -- start start 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6699346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6699346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561e3a597ce0 con 0x561e3a5c6820 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6697631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6697631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58698/0 (socket says 172.21.15.28:58698) 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6697631640 1 -- 172.21.15.28:0/3715603108 learned_addr learned my addr 172.21.15.28:0/3715603108 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.359+0000 7f6697631640 1 -- 172.21.15.28:0/3715603108 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3a5c7210 con 0x561e3a5c6820 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6697631640 1 --2- 172.21.15.28:0/3715603108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5c6c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f669000afe0 tx=0x7f66900204b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376548d2d78bc21d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f657a7fc640 1 -- 172.21.15.28:0/3715603108 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690044030 con 0x561e3a5c6820 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f657a7fc640 1 -- 172.21.15.28:0/3715603108 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690021a90 con 0x561e3a5c6820 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f657a7fc640 1 -- 172.21.15.28:0/3715603108 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690021570 con 0x561e3a5c6820 2024-09-19T11:23:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6699346800 1 -- 172.21.15.28:0/3715603108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 msgr2=0x561e3a5c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6699346800 1 --2- 172.21.15.28:0/3715603108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5c6c20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f669000afe0 tx=0x7f66900204b0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6699346800 1 -- 172.21.15.28:0/3715603108 shutdown_connections 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6699346800 1 --2- 172.21.15.28:0/3715603108 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5c6c20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6699346800 1 -- 172.21.15.28:0/3715603108 >> 172.21.15.28:0/3715603108 conn(0x561e3a5c2c20 msgr2=0x561e3a5c0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6699346800 1 -- 172.21.15.28:0/3715603108 shutdown_connections 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.360+0000 7f6699346800 1 -- 172.21.15.28:0/3715603108 wait complete. 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6699346800 1 Processor -- start 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6699346800 1 -- start start 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6699346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6699346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561e3a5c7830 con 0x561e3a5c6820 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6697631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6697631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58712/0 (socket says 172.21.15.28:58712) 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 learned_addr learned my addr 172.21.15.28:0/3273547331 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.361+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e3a5ce360 con 0x561e3a5c6820 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.362+0000 7f6697631640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5cde00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6690020b60 tx=0x7f66900547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.362+0000 7f6578ff9640 1 -- 172.21.15.28:0/3273547331 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690020bd0 con 0x561e3a5c6820 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.362+0000 7f6578ff9640 1 -- 172.21.15.28:0/3273547331 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690020d70 con 0x561e3a5c6820 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.362+0000 7f6578ff9640 1 -- 172.21.15.28:0/3273547331 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669001fa90 con 0x561e3a5c6820 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.362+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561e3a5ce5f0 con 0x561e3a5c6820 2024-09-19T11:23:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.362+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e3a5cf180 con 0x561e3a5c6820 2024-09-19T11:23:21.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.362+0000 7f6578ff9640 1 -- 172.21.15.28:0/3273547331 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f669001fc30 con 0x561e3a5c6820 2024-09-19T11:23:21.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.363+0000 7f6578ff9640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f656003d000 0x7f656003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.363+0000 7f6578ff9640 1 -- 172.21.15.28:0/3273547331 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f669003d060 con 0x561e3a5c6820 2024-09-19T11:23:21.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.363+0000 7f6696e30640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f656003d000 0x7f656003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.363+0000 7f6696e30640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f656003d000 0x7f656003f4c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f668800bbf0 tx=0x7f668804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.363+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.363+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a5d9750 0x561e3a5dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.363+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5dc360 con 0x561e3a5d9750 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f668ffff640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a5d9750 0x561e3a5dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f668ffff640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a5d9750 0x561e3a5dbc10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 20 realm 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5dc790 con 0x561e3a5d9750 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:21.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.364+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5dd060 con 0x561e3a5d9750 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5dd400 con 0x561e3a5d9750 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6699346800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5de160 0x561e3a5e0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e0b60 con 0x561e3a5de160 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6697631640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5de160 0x561e3a5e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.365+0000 7f6697631640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5de160 0x561e3a5e0550 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6699346800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6699346800 20 searching for the correct realm 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x561e3a5de160 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6697631640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f669003c3f0 0x7f6690008970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x7f669003c3f0 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6696e30640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f669003c3f0 0x7f6690008970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.366+0000 7f6696e30640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f669003c3f0 0x7f6690008970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.367+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.367+0000 7f6696e30640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f668803c070 0x7f66880542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.367+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x7f668803c070 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.367+0000 7f668ffff640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f668803c070 0x7f66880542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.367+0000 7f668ffff640 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f668803c070 0x7f66880542d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.367+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.367+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x561e3a5d9750 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x561e3a5d9750 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x7f668803c070 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x7f668803c070 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.368+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x7f669003c3f0 2024-09-19T11:23:21.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x561e3a5de160 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x561e3a5d9750 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x7f668803c070 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.369+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x561e3a5d9750 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2490 con 0x7f669003c3f0 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5de160 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5de160 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v24'2 uv2 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.370+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f668803c070 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5de160 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5de160 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5d9750 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.371+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5d9750 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f668803c070 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5de160 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f669003c3f0 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.372+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5d9750 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f668803c070 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5de160 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f668803c070 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.373+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f669003c3f0 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5de160 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f668803c070 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x561e3a5d9750 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.374+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f6688061060 con 0x7f669003c3f0 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:21.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e1520 con 0x561e3a5d9750 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e1d70 con 0x561e3a5d9750 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2910 con 0x561e3a5de160 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 20 zone default found 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 4 Realm: () 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.375+0000 7f6699346800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 20 zonegroup default 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e2cf0 con 0x7f668803c070 2024-09-19T11:23:21.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e4fd0 con 0x561e3a5de160 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.376+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e5370 con 0x561e3a5de160 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.377+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.377+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.377+0000 7f6699346800 20 started sync module instance, tier type = 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.377+0000 7f6699346800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.377+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e4460 con 0x7f669003c3f0 2024-09-19T11:23:21.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.378+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.378+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94687828265040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e40c0 con 0x7f669003c3f0 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.378+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e6270 con 0x561e3a5d9750 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.379+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94687828265040] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.379+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.379+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94687828270528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e6690 con 0x561e3a5d9750 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.379+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e7050 con 0x561e3a5de160 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.380+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.380+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94687828274080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e74b0 con 0x561e3a5de160 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.380+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e7fb0 con 0x7f669003c3f0 2024-09-19T11:23:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.380+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94687828270528] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.381+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94687828274080] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.381+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.381+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94687828277920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e6fc0 con 0x7f669003c3f0 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.381+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e89c0 con 0x561e3a5d9750 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.382+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.382+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94687828280592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e8de0 con 0x561e3a5d9750 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.382+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e9830 con 0x7f668803c070 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.382+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94687828277920] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.383+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.383+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94687828280592] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.383+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94687828284192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e97f0 con 0x7f668803c070 2024-09-19T11:23:21.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.383+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e9d90 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.384+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.384+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94687828281648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5ea240 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.384+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5eac60 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94687828284192] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94687828281648] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94687828289360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e9dd0 con 0x7f668803c070 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 20 add_watcher() i=0 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 20 add_watcher() i=1 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 20 add_watcher() i=2 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 20 add_watcher() i=3 2024-09-19T11:23:21.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 20 add_watcher() i=4 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 20 add_watcher() i=5 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.385+0000 7f6699346800 20 add_watcher() i=6 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.387+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94687828289360] v31'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6680057030 con 0x7f668803c070 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.387+0000 7f6699346800 20 add_watcher() i=7 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.387+0000 7f6699346800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.387+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5edd60 con 0x561e3a5d9750 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.387+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6680057030 con 0x561e3a5d9750 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.387+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94687828271584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5eb1e0 con 0x561e3a5d9750 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.388+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94687828271584] v31'41 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.388+0000 7f654d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.388+0000 7f6699346800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:21.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.388+0000 7f6699346800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.388+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f6460 con 0x561e3a5d9750 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.389+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.389+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f6460 con 0x7f668803c070 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.391+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.391+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f6e80 con 0x561e3a5de160 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.391+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.392+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f6e80 con 0x561e3a5d9750 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.392+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'42 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.393+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7260 con 0x7f669003c3f0 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.394+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'29 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.394+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f77d0 con 0x561e3a5de160 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.395+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'19 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.395+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7c80 con 0x561e3a5d9750 2024-09-19T11:23:21.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.396+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.396+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f6460 con 0x7f668803c070 2024-09-19T11:23:21.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.397+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'19 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.397+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7c80 con 0x7f669003c3f0 2024-09-19T11:23:21.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.398+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.398+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7b40 con 0x561e3a5de160 2024-09-19T11:23:21.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.399+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'19 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.399+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7b40 con 0x7f669003c3f0 2024-09-19T11:23:21.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.400+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'30 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.400+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f668803c070 2024-09-19T11:23:21.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.401+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'15 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.401+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f669003c3f0 2024-09-19T11:23:21.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.402+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.402+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f668803c070 2024-09-19T11:23:21.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.403+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'25 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.403+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5d9750 2024-09-19T11:23:21.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.404+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'43 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.404+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.405+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'23 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.405+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f669003c3f0 2024-09-19T11:23:21.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.406+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'31 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.406+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.407+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'13 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.407+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5d9750 2024-09-19T11:23:21.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.408+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.408+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5d9750 2024-09-19T11:23:21.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.409+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'44 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.409+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.411+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.411+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f669003c3f0 2024-09-19T11:23:21.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.412+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.412+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.412+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.413+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.413+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.414+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.414+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'9 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.415+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.415+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'20 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.416+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f668803c070 2024-09-19T11:23:21.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.417+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.417+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.418+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.418+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f668803c070 2024-09-19T11:23:21.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.419+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'20 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.419+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x561e3a5de160 2024-09-19T11:23:21.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.420+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'25 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.420+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f668803c070 2024-09-19T11:23:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.421+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'26 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.421+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f7ff0 con 0x7f669003c3f0 2024-09-19T11:23:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.422+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.422+0000 7f64c7fff640 20 reqs_thread_entry: start 2024-09-19T11:23:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.422+0000 7f6699346800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.422+0000 7f6699346800 20 Filter name: none 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.422+0000 7f64c5ffb640 20 reqs_thread_entry: start 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.423+0000 7f6699346800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.423+0000 7f6699346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.423+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561e3a622ac0 con 0x561e3a5d9750 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.423+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(94 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.423+0000 7f6699346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.423+0000 7f6699346800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.423+0000 7f6699346800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:could not remove user: unable to remove user, user does not exist 2024-09-19T11:23:21.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.424+0000 7f654d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.424+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94687828271584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f5470 con 0x561e3a5d9750 2024-09-19T11:23:21.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.425+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94687828271584] v31'45 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.425+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94687828265040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e61e0 con 0x7f669003c3f0 2024-09-19T11:23:21.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.426+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94687828265040] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.426+0000 7f6699346800 20 remove_watcher() i=0 2024-09-19T11:23:21.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.426+0000 7f6699346800 2 removed watcher, disabling cache 2024-09-19T11:23:21.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.426+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94687828270528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e61e0 con 0x561e3a5d9750 2024-09-19T11:23:21.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.427+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94687828270528] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.427+0000 7f6699346800 20 remove_watcher() i=1 2024-09-19T11:23:21.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.427+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94687828274080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5f5470 con 0x561e3a5de160 2024-09-19T11:23:21.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.428+0000 7f6697631640 1 -- 172.21.15.28:0/3273547331 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94687828274080] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6690053020 con 0x561e3a5de160 2024-09-19T11:23:21.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.428+0000 7f6699346800 20 remove_watcher() i=2 2024-09-19T11:23:21.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.428+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94687828277920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e6ba0 con 0x7f669003c3f0 2024-09-19T11:23:21.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.429+0000 7f6696e30640 1 -- 172.21.15.28:0/3273547331 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94687828277920] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6688054040 con 0x7f669003c3f0 2024-09-19T11:23:21.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.429+0000 7f6699346800 20 remove_watcher() i=3 2024-09-19T11:23:21.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.429+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94687828280592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e6ba0 con 0x561e3a5d9750 2024-09-19T11:23:21.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.430+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94687828280592] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f668007f040 con 0x561e3a5d9750 2024-09-19T11:23:21.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.430+0000 7f6699346800 20 remove_watcher() i=4 2024-09-19T11:23:21.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.430+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94687828284192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e8510 con 0x7f668803c070 2024-09-19T11:23:21.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.431+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94687828284192] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.431+0000 7f6699346800 20 remove_watcher() i=5 2024-09-19T11:23:21.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.431+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94687828281648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e6ba0 con 0x7f668803c070 2024-09-19T11:23:21.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.432+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94687828281648] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.432+0000 7f6699346800 20 remove_watcher() i=6 2024-09-19T11:23:21.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.432+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94687828289360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561e3a5e6ba0 con 0x7f668803c070 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94687828289360] v31'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f668007f040 con 0x7f668803c070 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 20 remove_watcher() i=7 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f668803c070 msgr2=0x7f66880542d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f668803c070 0x7f66880542d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a5d9750 msgr2=0x561e3a5dbc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a5d9750 0x561e3a5dbc10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f669003c3f0 msgr2=0x7f6690008970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f669003c3f0 0x7f6690008970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5de160 msgr2=0x561e3a5e0550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5de160 0x561e3a5e0550 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f656003d000 msgr2=0x7f656003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f656003d000 0x7f656003f4c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f668800bbf0 tx=0x7f668804f000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 msgr2=0x561e3a5cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f668ffff640 1 -- 172.21.15.28:0/3273547331 reap_dead start 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5cde00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6690020b60 tx=0x7f66900547f0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 shutdown_connections 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3273547331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5c6820 0x561e3a5cde00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 >> 172.21.15.28:0/3273547331 conn(0x561e3a5c2c20 msgr2=0x561e3a5c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 shutdown_connections 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3273547331 wait complete. 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657003d000 msgr2=0x7f657003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657003d000 0x7f657003f4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f668800ecb0 tx=0x7f6688001460 comp rx=0 tx=0).stop 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5b9da0 msgr2=0x561e3a5ba1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.433+0000 7f6699346800 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5b9da0 0x561e3a5ba1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x561e3a59fdd0 tx=0x7f66900167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 shutdown_connections 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657003d000 0x7f657003f4c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3446457283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a5b9da0 0x561e3a5ba1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 >> 172.21.15.28:0/3446457283 conn(0x561e3a5ae150 msgr2=0x561e3a5ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 shutdown_connections 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3446457283 wait complete. 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a4a5880 msgr2=0x561e3a5a6420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a4a5880 0x561e3a5a6420 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5a8800 msgr2=0x561e3a5aac80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5a8800 0x561e3a5aac80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657c03d1f0 msgr2=0x7f657c03f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657c03d1f0 0x7f657c03f6b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6680057000 tx=0x7f6680052000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 msgr2=0x561e3a59edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a59edd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f66880012f0 tx=0x7f6688000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 shutdown_connections 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561e3a5a8800 0x561e3a5aac80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f657c03d1f0 0x7f657c03f6b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561e3a4a5880 0x561e3a5a6420 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 --2- 172.21.15.28:0/3137720055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561e3a4a3d60 0x561e3a59edd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 >> 172.21.15.28:0/3137720055 conn(0x561e3a49fa50 msgr2=0x561e3a5987b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 shutdown_connections 2024-09-19T11:23:21.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.434+0000 7f6699346800 1 -- 172.21.15.28:0/3137720055 wait complete. 2024-09-19T11:23:21.444 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T11:23:21.445 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-19T11:23:21.445 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-19T11:23:21.445 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-09-19T11:23:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb7212800 1 Processor -- start 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb7212800 1 -- start start 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb7212800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5623d724a950 0x5623d7253470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb7212800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5623d72539b0 con 0x5623d7248e30 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb7212800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5623d70d57d0 con 0x5623d724a950 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fababfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5623d724a950 0x5623d7253470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb28cb640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5623d7248e30 0x5623d7249230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32962/0 (socket says 172.21.15.28:32962) 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb28cb640 1 -- 172.21.15.28:0/3642047786 learned_addr learned my addr 172.21.15.28:0/3642047786 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fababfff640 1 -- 172.21.15.28:0/3642047786 >> v1:172.21.15.22:6789/0 conn(0x5623d7248e30 legacy=0x5623d7249230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.493+0000 7fabb20ca640 1 -- 172.21.15.28:0/3642047786 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837548044 0 0) 0x5623d72539b0 con 0x5623d7248e30 2024-09-19T11:23:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.494+0000 7fababfff640 1 -- 172.21.15.28:0/3642047786 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d7254b90 con 0x5623d724a950 2024-09-19T11:23:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.495+0000 7fababfff640 1 --2- 172.21.15.28:0/3642047786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d724a950 0x5623d7253470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a470 tx=0x7fab9c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57203ee8af84ca52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.495+0000 7fabb20ca640 1 -- 172.21.15.28:0/3642047786 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab9c053570 con 0x5623d724a950 2024-09-19T11:23:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.495+0000 7fabb20ca640 1 -- 172.21.15.28:0/3642047786 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c058070 con 0x5623d724a950 2024-09-19T11:23:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.495+0000 7fabb20ca640 1 -- 172.21.15.28:0/3642047786 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab9c053880 con 0x5623d724a950 2024-09-19T11:23:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.496+0000 7fabb7212800 1 -- 172.21.15.28:0/3642047786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d724a950 msgr2=0x5623d7253470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.496+0000 7fabb7212800 1 --2- 172.21.15.28:0/3642047786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d724a950 0x5623d7253470 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00a470 tx=0x7fab9c045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.496+0000 7fabb7212800 1 -- 172.21.15.28:0/3642047786 shutdown_connections 2024-09-19T11:23:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.496+0000 7fabb7212800 1 --2- 172.21.15.28:0/3642047786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d724a950 0x5623d7253470 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.496+0000 7fabb7212800 1 -- 172.21.15.28:0/3642047786 >> 172.21.15.28:0/3642047786 conn(0x5623d7244b60 msgr2=0x5623d7246fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.496+0000 7fabb7212800 1 -- 172.21.15.28:0/3642047786 shutdown_connections 2024-09-19T11:23:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.496+0000 7fabb7212800 1 -- 172.21.15.28:0/3642047786 wait complete. 2024-09-19T11:23:21.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.511+0000 7fabb7212800 1 Processor -- start 2024-09-19T11:23:21.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.511+0000 7fabb7212800 1 -- start start 2024-09-19T11:23:21.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.511+0000 7fabb7212800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d733c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.511+0000 7fabb7212800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623d70e2dd0 con 0x5623d7248e30 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7fabb28cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d733c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7fabb28cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d733c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58744/0 (socket says 172.21.15.28:58744) 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7fabb28cb640 1 -- 172.21.15.28:0/3127074624 learned_addr learned my addr 172.21.15.28:0/3127074624 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7fabb28cb640 1 -- 172.21.15.28:0/3127074624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d733ca50 con 0x5623d7248e30 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7fabb28cb640 1 --2- 172.21.15.28:0/3127074624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d733c460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faba4001ea0 tx=0x7faba400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8ef6dec9a99389 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7faaa0e09640 1 -- 172.21.15.28:0/3127074624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faba4008070 con 0x5623d7248e30 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7faaa0e09640 1 -- 172.21.15.28:0/3127074624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba400d500 con 0x5623d7248e30 2024-09-19T11:23:21.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.512+0000 7faaa0e09640 1 -- 172.21.15.28:0/3127074624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faba400d880 con 0x5623d7248e30 2024-09-19T11:23:21.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 -- 172.21.15.28:0/3127074624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 msgr2=0x5623d733c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 --2- 172.21.15.28:0/3127074624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d733c460 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faba4001ea0 tx=0x7faba400b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 -- 172.21.15.28:0/3127074624 shutdown_connections 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 --2- 172.21.15.28:0/3127074624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d733c460 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 -- 172.21.15.28:0/3127074624 >> 172.21.15.28:0/3127074624 conn(0x5623d7244b60 msgr2=0x5623d7337de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 -- 172.21.15.28:0/3127074624 shutdown_connections 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 -- 172.21.15.28:0/3127074624 wait complete. 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 Processor -- start 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 -- start start 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d7343f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.513+0000 7fabb7212800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623d7253e80 con 0x5623d7248e30 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7fabb28cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d7343f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7fabb28cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d7343f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58748/0 (socket says 172.21.15.28:58748) 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7fabb28cb640 1 -- 172.21.15.28:0/1087045770 learned_addr learned my addr 172.21.15.28:0/1087045770 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7fabb28cb640 1 -- 172.21.15.28:0/1087045770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d73444d0 con 0x5623d7248e30 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7fabb28cb640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d7343f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faba40012f0 tx=0x7faba4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7faa9f606640 1 -- 172.21.15.28:0/1087045770 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faba400f9d0 con 0x5623d7248e30 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7faa9f606640 1 -- 172.21.15.28:0/1087045770 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba400fb70 con 0x5623d7248e30 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623d7344760 con 0x5623d7248e30 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7faa9f606640 1 -- 172.21.15.28:0/1087045770 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faba400fe50 con 0x5623d7248e30 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.514+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d7345160 con 0x5623d7248e30 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7fabb7212800 1 Processor -- start 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7fabb7212800 1 -- start start 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7fabb7212800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7353950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7fabb7212800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623d71414a0 con 0x5623d7351560 2024-09-19T11:23:21.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7faa9f606640 1 -- 172.21.15.28:0/1087045770 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7faba4017040 con 0x5623d7248e30 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7fabb30cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7353950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7fabb30cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7353950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58760/0 (socket says 172.21.15.28:58760) 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7fabb30cc640 1 -- 172.21.15.28:0/3705579167 learned_addr learned my addr 172.21.15.28:0/3705579167 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.515+0000 7faa9f606640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa9803d1a0 0x7faa9803f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7faa9f606640 1 -- 172.21.15.28:0/1087045770 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7faba4015020 con 0x5623d7248e30 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb28cb640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa9803d1a0 0x7faa9803f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb30cc640 1 -- 172.21.15.28:0/3705579167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d7353f20 con 0x5623d7351560 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb30cc640 1 --2- 172.21.15.28:0/3705579167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7353950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabac00a3e0 tx=0x7fabac00a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77282ea9817ffbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb28cb640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa9803d1a0 0x7faa9803f660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faba400c010 tx=0x7faba40011b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7faa9d602640 1 -- 172.21.15.28:0/3705579167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac015070 con 0x5623d7351560 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7faa9d602640 1 -- 172.21.15.28:0/3705579167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabac001670 con 0x5623d7351560 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7faa9d602640 1 -- 172.21.15.28:0/3705579167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac001990 con 0x5623d7351560 2024-09-19T11:23:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb7212800 1 -- 172.21.15.28:0/3705579167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 msgr2=0x5623d7353950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb7212800 1 --2- 172.21.15.28:0/3705579167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7353950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabac00a3e0 tx=0x7fabac00a8a0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb7212800 1 -- 172.21.15.28:0/3705579167 shutdown_connections 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb7212800 1 --2- 172.21.15.28:0/3705579167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7353950 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.516+0000 7fabb7212800 1 -- 172.21.15.28:0/3705579167 >> 172.21.15.28:0/3705579167 conn(0x5623d724a950 msgr2=0x5623d734be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb7212800 1 -- 172.21.15.28:0/3705579167 shutdown_connections 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb7212800 1 -- 172.21.15.28:0/3705579167 wait complete. 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb7212800 1 Processor -- start 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb7212800 1 -- start start 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb7212800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7358570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb7212800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623d7354270 con 0x5623d7351560 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb30cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7358570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb30cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7358570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58774/0 (socket says 172.21.15.28:58774) 2024-09-19T11:23:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.517+0000 7fabb30cc640 1 -- 172.21.15.28:0/2385855597 learned_addr learned my addr 172.21.15.28:0/2385855597 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.518+0000 7fabb30cc640 1 -- 172.21.15.28:0/2385855597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d7358ad0 con 0x5623d7351560 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.518+0000 7fabb30cc640 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7358570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fabac013040 tx=0x7fabac003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.518+0000 7faa977fe640 1 -- 172.21.15.28:0/2385855597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac020020 con 0x5623d7351560 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.518+0000 7faa977fe640 1 -- 172.21.15.28:0/2385855597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabac010780 con 0x5623d7351560 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.518+0000 7faa977fe640 1 -- 172.21.15.28:0/2385855597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabac015040 con 0x5623d7351560 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.518+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623d7358d60 con 0x5623d7351560 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.518+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d73596e0 con 0x5623d7351560 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.519+0000 7faa977fe640 1 -- 172.21.15.28:0/2385855597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fabac023020 con 0x5623d7351560 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.519+0000 7faa977fe640 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa8c03cfb0 0x7faa8c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.519+0000 7faa977fe640 1 -- 172.21.15.28:0/2385855597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fabac053dc0 con 0x5623d7351560 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.519+0000 7fababfff640 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa8c03cfb0 0x7faa8c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb7212800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb7212800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fababfff640 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa8c03cfb0 0x7faa8c03f470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fab9c057000 tx=0x7fab9c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb7212800 1 Processor -- start 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb7212800 1 -- start start 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb7212800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7372380 0x5623d73727a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb7212800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5623d733ce80 con 0x5623d7372380 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb28cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7372380 0x5623d73727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb28cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7372380 0x5623d73727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58790/0 (socket says 172.21.15.28:58790) 2024-09-19T11:23:21.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.520+0000 7fabb28cb640 1 -- 172.21.15.28:0/1606164267 learned_addr learned my addr 172.21.15.28:0/1606164267 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.521+0000 7fabb28cb640 1 -- 172.21.15.28:0/1606164267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d7372de0 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.521+0000 7fabb28cb640 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7372380 0x5623d73727a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faba400ead0 tx=0x7faba401aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.521+0000 7faa777e6640 1 -- 172.21.15.28:0/1606164267 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faba4047070 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.521+0000 7faa777e6640 1 -- 172.21.15.28:0/1606164267 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faba401acd0 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.521+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623d7373610 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.521+0000 7faa777e6640 1 -- 172.21.15.28:0/1606164267 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faba401a3c0 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.521+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d7373bb0 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7faa777e6640 1 -- 172.21.15.28:0/1606164267 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7faba40546c0 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7faa777e6640 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa9f403d000 0x7fa9f403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7faa777e6640 1 -- 172.21.15.28:0/1606164267 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7faba4037070 con 0x5623d7372380 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7fababfff640 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa9f403d000 0x7fa9f403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7faa757e2640 20 reqs_thread_entry: start 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7fababfff640 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa9f403d000 0x7fa9f403f4c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5623d71bd860 tx=0x7fab9c005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5623d737fe30 0x5623d73822f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.522+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5623d73829c0 con 0x5623d737fe30 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.523+0000 7fabb30cc640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5623d737fe30 0x5623d73822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.523+0000 7fabb30cc640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5623d737fe30 0x5623d73822f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.523+0000 7fabb30cc640 1 -- 172.21.15.28:0/1087045770 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fabac00e070 con 0x5623d737fe30 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.523+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5623d7384790 0x5623d7386b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.523+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:2 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5623d73872c0 con 0x5623d7384790 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.523+0000 7fabb28cb640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5623d7384790 0x5623d7386b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.524+0000 7fabb28cb640 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5623d7384790 0x5623d7386b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.524+0000 7fabb28cb640 1 -- 172.21.15.28:0/1087045770 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(2 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7faba402d070 con 0x5623d7384790 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.524+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5623d738c140 con 0x5623d7384790 2024-09-19T11:23:21.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.525+0000 7fabb28cb640 1 -- 172.21.15.28:0/1087045770 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [stat,call,call,writefull 0~993] v31'3 uv3 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7faba402d070 con 0x5623d7384790 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa8c03cfb0 msgr2=0x7faa8c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa8c03cfb0 0x7faa8c03f470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fab9c057000 tx=0x7fab9c052000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 msgr2=0x5623d7358570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7358570 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fabac013040 tx=0x7fabac003370 comp rx=0 tx=0).stop 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 shutdown_connections 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa8c03cfb0 0x7faa8c03f470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 --2- 172.21.15.28:0/2385855597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7351560 0x5623d7358570 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 >> 172.21.15.28:0/2385855597 conn(0x5623d724a950 msgr2=0x5623d734be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 shutdown_connections 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.526+0000 7fabb7212800 1 -- 172.21.15.28:0/2385855597 wait complete. 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa9f403d000 msgr2=0x7fa9f403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa9f403d000 0x7fa9f403f4c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x5623d71bd860 tx=0x7fab9c005a00 comp rx=0 tx=0).stop 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7372380 msgr2=0x5623d73727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7372380 0x5623d73727a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faba400ead0 tx=0x7faba401aba0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 shutdown_connections 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa9f403d000 0x7fa9f403f4c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1606164267 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7372380 0x5623d73727a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 >> 172.21.15.28:0/1606164267 conn(0x5623d73678e0 msgr2=0x5623d7367cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 shutdown_connections 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1606164267 wait complete. 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5623d737fe30 msgr2=0x5623d73822f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5623d737fe30 0x5623d73822f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5623d7384790 msgr2=0x5623d7386b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5623d7384790 0x5623d7386b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa9803d1a0 msgr2=0x7faa9803f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa9803d1a0 0x7faa9803f660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faba400c010 tx=0x7faba40011b0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 msgr2=0x5623d7343f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d7343f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faba40012f0 tx=0x7faba4000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 shutdown_connections 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5623d7384790 0x5623d7386b80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7faa9803d1a0 0x7faa9803f660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5623d737fe30 0x5623d73822f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 --2- 172.21.15.28:0/1087045770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5623d7248e30 0x5623d7343f70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 >> 172.21.15.28:0/1087045770 conn(0x5623d7244b60 msgr2=0x5623d733d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 shutdown_connections 2024-09-19T11:23:21.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.527+0000 7fabb7212800 1 -- 172.21.15.28:0/1087045770 wait complete. 2024-09-19T11:23:21.538 INFO:teuthology.orchestra.run.smithi028.stdout:{"id":"293899f1-2738-4b64-a8e1-a18ff039d626","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-09-19T11:23:21.538 DEBUG:tasks.util.rgw: json result: {'id': '293899f1-2738-4b64-a8e1-a18ff039d626', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-09-19T11:23:21.539 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-09-19T11:23:21.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-09-19T11:23:21.540 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-09-19T11:23:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:21.583 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:21.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.582+0000 7f26ddb39800 1 Processor -- start 2024-09-19T11:23:21.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26ddb39800 1 -- start start 2024-09-19T11:23:21.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26ddb39800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5587de32a680 0x5587de32b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26ddb39800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5587de32ad30 con 0x5587de32b8c0 2024-09-19T11:23:21.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26ddb39800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5587de1b67d0 con 0x5587de32a680 2024-09-19T11:23:21.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26d97c7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5587de32a680 0x5587de32b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26d97c7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5587de32a680 0x5587de32b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58796/0 (socket says 172.21.15.28:58796) 2024-09-19T11:23:21.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26d8fc6640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5587de32b8c0 0x5587de330620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32966/0 (socket says 172.21.15.28:32966) 2024-09-19T11:23:21.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26d97c7640 1 -- 172.21.15.28:0/2686368420 learned_addr learned my addr 172.21.15.28:0/2686368420 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26d97c7640 1 -- 172.21.15.28:0/2686368420 >> v1:172.21.15.22:6789/0 conn(0x5587de32b8c0 legacy=0x5587de330620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.583+0000 7f26d97c7640 1 -- 172.21.15.28:0/2686368420 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de32bd70 con 0x5587de32a680 2024-09-19T11:23:21.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.585+0000 7f26d97c7640 1 --2- 172.21.15.28:0/2686368420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de32b380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00a520 tx=0x7f26cc045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abca1f7c968c8ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.585+0000 7f26cbfff640 1 -- 172.21.15.28:0/2686368420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26cc055070 con 0x5587de32a680 2024-09-19T11:23:21.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.585+0000 7f26cbfff640 1 -- 172.21.15.28:0/2686368420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26cc0515b0 con 0x5587de32a680 2024-09-19T11:23:21.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.585+0000 7f26cbfff640 1 -- 172.21.15.28:0/2686368420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26cc0518d0 con 0x5587de32a680 2024-09-19T11:23:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.586+0000 7f26ddb39800 1 -- 172.21.15.28:0/2686368420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 msgr2=0x5587de32b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.586+0000 7f26ddb39800 1 --2- 172.21.15.28:0/2686368420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de32b380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00a520 tx=0x7f26cc045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.586+0000 7f26ddb39800 1 -- 172.21.15.28:0/2686368420 shutdown_connections 2024-09-19T11:23:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.586+0000 7f26ddb39800 1 --2- 172.21.15.28:0/2686368420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de32b380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.586+0000 7f26ddb39800 1 -- 172.21.15.28:0/2686368420 >> 172.21.15.28:0/2686368420 conn(0x5587de325a50 msgr2=0x5587de327eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.586+0000 7f26ddb39800 1 -- 172.21.15.28:0/2686368420 shutdown_connections 2024-09-19T11:23:21.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.586+0000 7f26ddb39800 1 -- 172.21.15.28:0/2686368420 wait complete. 2024-09-19T11:23:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.601+0000 7f26ddb39800 1 Processor -- start 2024-09-19T11:23:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.601+0000 7f26ddb39800 1 -- start start 2024-09-19T11:23:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.602+0000 7f26ddb39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de29e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.602+0000 7f26ddb39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587de1c3dd0 con 0x5587de32a680 2024-09-19T11:23:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.602+0000 7f26d97c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de29e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.602+0000 7f26d97c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de29e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58802/0 (socket says 172.21.15.28:58802) 2024-09-19T11:23:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.602+0000 7f26d97c7640 1 -- 172.21.15.28:0/3234628047 learned_addr learned my addr 172.21.15.28:0/3234628047 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.602+0000 7f26d97c7640 1 -- 172.21.15.28:0/3234628047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de29edc0 con 0x5587de32a680 2024-09-19T11:23:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.602+0000 7f26d97c7640 1 --2- 172.21.15.28:0/3234628047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de29e7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26cc025da0 tx=0x7f26cc051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ead85ac29915114 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f25c7606640 1 -- 172.21.15.28:0/3234628047 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26cc051950 con 0x5587de32a680 2024-09-19T11:23:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f25c7606640 1 -- 172.21.15.28:0/3234628047 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26cc051af0 con 0x5587de32a680 2024-09-19T11:23:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f25c7606640 1 -- 172.21.15.28:0/3234628047 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26cc051dd0 con 0x5587de32a680 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f26ddb39800 1 -- 172.21.15.28:0/3234628047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 msgr2=0x5587de29e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f26ddb39800 1 --2- 172.21.15.28:0/3234628047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de29e7d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26cc025da0 tx=0x7f26cc051000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f26ddb39800 1 -- 172.21.15.28:0/3234628047 shutdown_connections 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f26ddb39800 1 --2- 172.21.15.28:0/3234628047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de29e7d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f26ddb39800 1 -- 172.21.15.28:0/3234628047 >> 172.21.15.28:0/3234628047 conn(0x5587de325a50 msgr2=0x5587de415b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f26ddb39800 1 -- 172.21.15.28:0/3234628047 shutdown_connections 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.603+0000 7f26ddb39800 1 -- 172.21.15.28:0/3234628047 wait complete. 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26ddb39800 1 Processor -- start 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26ddb39800 1 -- start start 2024-09-19T11:23:21.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26ddb39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de422d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26ddb39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587de331d40 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26d97c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de422d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26d97c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de422d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58818/0 (socket says 172.21.15.28:58818) 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26d97c7640 1 -- 172.21.15.28:0/1170036981 learned_addr learned my addr 172.21.15.28:0/1170036981 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.604+0000 7f26d97c7640 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de423260 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.605+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de422d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f26cc067040 tx=0x7f26cc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.605+0000 7f25c5e03640 1 -- 172.21.15.28:0/1170036981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26cc073070 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.605+0000 7f25c5e03640 1 -- 172.21.15.28:0/1170036981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26cc05a450 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.605+0000 7f25c5e03640 1 -- 172.21.15.28:0/1170036981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26cc05a730 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.605+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5587de4234f0 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.605+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587de424100 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f25c5e03640 1 -- 172.21.15.28:0/1170036981 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f26cc076070 con 0x5587de32a680 2024-09-19T11:23:21.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f25c5e03640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25c003d000 0x7f25c003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f25c5e03640 1 -- 172.21.15.28:0/1170036981 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f26cc062070 con 0x5587de32a680 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f26d8fc6640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25c003d000 0x7f25c003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de32b8c0 0x5587de42a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de42a890 con 0x5587de32b8c0 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de32b8c0 0x5587de42a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.606+0000 7f26d8fc6640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25c003d000 0x7f25c003f4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26d0001e30 tx=0x7f26d0001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.607+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de32b8c0 0x5587de42a350 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.607+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1170036981 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26d400a4d0 con 0x5587de32b8c0 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.607+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de42ac30 con 0x5587de32b8c0 2024-09-19T11:23:21.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.607+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1170036981 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f26d400a4d0 con 0x5587de32b8c0 2024-09-19T11:23:21.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.607+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de42c6f0 0x5587de42eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.607+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de42f320 con 0x5587de42c6f0 2024-09-19T11:23:21.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.607+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de42c6f0 0x5587de42eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.608+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de42c6f0 0x5587de42eb90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.608+0000 7f26d97c7640 1 -- 172.21.15.28:0/1170036981 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f26cc075030 con 0x5587de42c6f0 2024-09-19T11:23:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.608+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de42b480 con 0x5587de32b8c0 2024-09-19T11:23:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.608+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1170036981 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f26d400a4d0 con 0x5587de32b8c0 2024-09-19T11:23:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.608+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de42b820 con 0x5587de42c6f0 2024-09-19T11:23:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.608+0000 7f26d97c7640 1 -- 172.21.15.28:0/1170036981 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f26cc075030 con 0x5587de42c6f0 2024-09-19T11:23:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.609+0000 7f26ddb39800 1 Processor -- start 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.609+0000 7f26ddb39800 1 -- start start 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.609+0000 7f26ddb39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de41b6e0 0x5587de419f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.609+0000 7f26ddb39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587de2224a0 con 0x5587de41b6e0 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.609+0000 7f26d9fc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de41b6e0 0x5587de419f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.609+0000 7f26d9fc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de41b6e0 0x5587de419f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58830/0 (socket says 172.21.15.28:58830) 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.609+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1260072863 learned_addr learned my addr 172.21.15.28:0/1260072863 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.610+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1260072863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de41bc00 con 0x5587de41b6e0 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.610+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de41b6e0 0x5587de419f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x5587de423b90 tx=0x7f26d40167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.610+0000 7f25beffd640 1 -- 172.21.15.28:0/1260072863 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d4013640 con 0x5587de41b6e0 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.610+0000 7f25beffd640 1 -- 172.21.15.28:0/1260072863 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d40137e0 con 0x5587de41b6e0 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.610+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5587de41a490 con 0x5587de41b6e0 2024-09-19T11:23:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.610+0000 7f25beffd640 1 -- 172.21.15.28:0/1260072863 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d4013ae0 con 0x5587de41b6e0 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.610+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587de41a9e0 con 0x5587de41b6e0 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f25beffd640 1 -- 172.21.15.28:0/1260072863 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f26d4011040 con 0x5587de41b6e0 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f25beffd640 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25b403d000 0x7f25b403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f25beffd640 1 -- 172.21.15.28:0/1260072863 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f26d400e040 con 0x5587de41b6e0 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25b403d000 0x7f25b403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f26ddb39800 1 Processor -- start 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25b403d000 0x7f25b403f4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5587de424e00 tx=0x7f26cc096fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f26ddb39800 1 -- start start 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f26ddb39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de44cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.611+0000 7f26ddb39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587de41e0f0 con 0x5587de44c940 2024-09-19T11:23:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f26d9fc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de44cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f26d9fc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de44cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58846/0 (socket says 172.21.15.28:58846) 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f26d9fc8640 1 -- 172.21.15.28:0/697233011 learned_addr learned my addr 172.21.15.28:0/697233011 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f26d9fc8640 1 -- 172.21.15.28:0/697233011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de44d330 con 0x5587de44c940 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/697233011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de44cd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26d4054860 tx=0x7f26d4020ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61dcfe66603d420 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f25bcff9640 1 -- 172.21.15.28:0/697233011 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d4044030 con 0x5587de44c940 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f25bcff9640 1 -- 172.21.15.28:0/697233011 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d4021960 con 0x5587de44c940 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.612+0000 7f25bcff9640 1 -- 172.21.15.28:0/697233011 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d404de60 con 0x5587de44c940 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 -- 172.21.15.28:0/697233011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 msgr2=0x5587de44cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 --2- 172.21.15.28:0/697233011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de44cd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26d4054860 tx=0x7f26d4020ad0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 -- 172.21.15.28:0/697233011 shutdown_connections 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 --2- 172.21.15.28:0/697233011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de44cd40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 -- 172.21.15.28:0/697233011 >> 172.21.15.28:0/697233011 conn(0x5587de448d40 msgr2=0x5587de446f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 -- 172.21.15.28:0/697233011 shutdown_connections 2024-09-19T11:23:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 -- 172.21.15.28:0/697233011 wait complete. 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 Processor -- start 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 -- start start 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de453fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26ddb39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5587de44d950 con 0x5587de44c940 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26d9fc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de453fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26d9fc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de453fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58854/0 (socket says 172.21.15.28:58854) 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.613+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 learned_addr learned my addr 172.21.15.28:0/1946554705 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.614+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587de454530 con 0x5587de44c940 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.614+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de453fd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26d4020510 tx=0x7f26d40547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.614+0000 7f25aaffd640 1 -- 172.21.15.28:0/1946554705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d4054a30 con 0x5587de44c940 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.614+0000 7f25aaffd640 1 -- 172.21.15.28:0/1946554705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d40143c0 con 0x5587de44c940 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.614+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5587de4547c0 con 0x5587de44c940 2024-09-19T11:23:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.614+0000 7f25aaffd640 1 -- 172.21.15.28:0/1946554705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26d4020bd0 con 0x5587de44c940 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.614+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587de455350 con 0x5587de44c940 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.615+0000 7f25aaffd640 1 -- 172.21.15.28:0/1946554705 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f26d4020d70 con 0x5587de44c940 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.615+0000 7f25aaffd640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25a403d000 0x7f25a403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.615+0000 7f25aaffd640 1 -- 172.21.15.28:0/1946554705 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f26d404b070 con 0x5587de44c940 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.615+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25a403d000 0x7f25a403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.615+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.615+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de45f840 0x5587de461d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.615+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de462450 con 0x5587de45f840 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26d8fc6640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de45f840 0x5587de461d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25a403d000 0x7f25a403f4c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0873f0 tx=0x7f26cc093000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26d8fc6640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de45f840 0x5587de461d00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26d000f610 con 0x5587de45f840 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 20 realm 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de45eb90 con 0x5587de45f840 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26d000f610 con 0x5587de45f840 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.616+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de462f30 con 0x5587de45f840 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26d000f610 con 0x5587de45f840 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26ddb39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de463800 con 0x5587de45f840 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f26d000f610 con 0x5587de45f840 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26ddb39800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de464140 0x5587de466530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de466c70 con 0x5587de464140 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de464140 0x5587de466530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.617+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de464140 0x5587de466530 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26ddb39800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26ddb39800 20 searching for the correct realm 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26d9fc8640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26d40123d0 0x7f26d403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26d40123d0 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26d40123d0 0x7f26d403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.618+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26d40123d0 0x7f26d403b350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.619+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.619+0000 7f26d97c7640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26cc0a6040 0x7f26cc050610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.619+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.619+0000 7f26d8fc6640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26cc0a6040 0x7f26cc050610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.619+0000 7f26d8fc6640 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26cc0a6040 0x7f26cc050610 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.619+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d000f610 con 0x7f26cc0a6040 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.619+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d000f610 con 0x5587de45f840 2024-09-19T11:23:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x5587de45f840 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.620+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26d40123d0 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x5587de45f840 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.621+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x5587de45f840 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26d40123d0 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.622+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x5587de45f840 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f26d002b040 con 0x5587de45f840 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.623+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26d40123d0 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d002b040 con 0x5587de45f840 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.624+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26d40123d0 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.625+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de464140 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26cc0a6040 2024-09-19T11:23:21.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x5587de45f840 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5587de468620 con 0x7f26d40123d0 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26ddb39800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26ddb39800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26ddb39800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26ddb39800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26ddb39800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26ddb39800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.626+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4676f0 con 0x5587de45f840 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de467330 con 0x5587de45f840 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de468050 con 0x5587de464140 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 zone default found 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 4 Realm: () 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 zonegroup default 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.627+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de467bc0 con 0x7f26cc0a6040 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de46aa60 con 0x5587de464140 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de46aec0 con 0x5587de464140 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 20 started sync module instance, tier type = 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.628+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de469eb0 con 0x7f26d40123d0 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.629+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.629+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94042038117024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46b7d0 con 0x7f26d40123d0 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.629+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46c020 con 0x5587de45f840 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.630+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94042038117024] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.630+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.630+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94042038121248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46c0a0 con 0x5587de45f840 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.630+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46cbc0 con 0x5587de464140 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.631+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.631+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94042038126224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46cb80 con 0x5587de464140 2024-09-19T11:23:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.632+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46dba0 con 0x7f26d40123d0 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.632+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94042038121248] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.632+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94042038126224] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.633+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.633+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94042038128560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46bb70 con 0x7f26d40123d0 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.633+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46e480 con 0x5587de45f840 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.634+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94042038128560] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.634+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.634+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94042038130352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46e440 con 0x5587de45f840 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.634+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46eef0 con 0x7f26cc0a6040 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.635+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94042038130352] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.635+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.635+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94042038135248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46e440 con 0x7f26cc0a6040 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.635+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46f900 con 0x7f26cc0a6040 2024-09-19T11:23:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.636+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94042038135248] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.636+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.636+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94042038132624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46fae0 con 0x7f26cc0a6040 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.636+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de470330 con 0x7f26cc0a6040 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94042038140096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de470710 con 0x7f26cc0a6040 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 20 add_watcher() i=0 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 20 add_watcher() i=1 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 20 add_watcher() i=2 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 20 add_watcher() i=3 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 20 add_watcher() i=4 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 20 add_watcher() i=5 2024-09-19T11:23:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94042038132624] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.637+0000 7f26ddb39800 20 add_watcher() i=6 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.638+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94042038140096] v31'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.638+0000 7f26ddb39800 20 add_watcher() i=7 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.638+0000 7f26ddb39800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.638+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de46f630 con 0x5587de45f840 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.638+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.638+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94042038141344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de46f630 con 0x5587de45f840 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.639+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94042038141344] v31'46 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.639+0000 7f258ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26ddb39800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26ddb39800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26ddb39800 5 note: GC not initialized 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f258efe5640 20 reqs_thread_entry: start 2024-09-19T11:23:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26ddb39800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26ddb39800 20 Filter name: none 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f258cfe1640 20 reqs_thread_entry: start 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6de0 con 0x5587de45f840 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.640+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6de0 con 0x7f26d40123d0 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.641+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.642+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.643+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.643+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.643+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.643+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.643+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.643+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26d40123d0 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.644+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.644+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.644+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.644+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.644+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.644+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.645+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.646+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.646+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26d40123d0 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.646+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.646+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.646+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.646+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.647+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.648+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.648+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.648+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.648+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.648+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.648+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.648+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.649+0000 7f258ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.649+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94042038141344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.650+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94042038141344] v31'47 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.650+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94042038117024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26d40123d0 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.651+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94042038117024] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.651+0000 7f26ddb39800 20 remove_watcher() i=0 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.651+0000 7f26ddb39800 2 removed watcher, disabling cache 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.652+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94042038121248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.653+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94042038121248] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.653+0000 7f26ddb39800 20 remove_watcher() i=1 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.653+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94042038126224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de464140 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.654+0000 7f26d9fc8640 1 -- 172.21.15.28:0/1946554705 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94042038126224] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d403a070 con 0x5587de464140 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.654+0000 7f26ddb39800 20 remove_watcher() i=2 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.654+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94042038128560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26d40123d0 2024-09-19T11:23:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.655+0000 7f26d97c7640 1 -- 172.21.15.28:0/1946554705 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94042038128560] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc075030 con 0x7f26d40123d0 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.655+0000 7f26ddb39800 20 remove_watcher() i=3 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.655+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94042038130352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x5587de45f840 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.656+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94042038130352] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x5587de45f840 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.656+0000 7f26ddb39800 20 remove_watcher() i=4 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.656+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94042038135248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.657+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94042038135248] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.657+0000 7f26ddb39800 20 remove_watcher() i=5 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.657+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94042038132624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.658+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94042038132624] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.658+0000 7f26ddb39800 20 remove_watcher() i=6 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.658+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94042038140096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5587de4a6da0 con 0x7f26cc0a6040 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94042038140096] v31'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26d003b040 con 0x7f26cc0a6040 2024-09-19T11:23:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 20 remove_watcher() i=7 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26cc0a6040 msgr2=0x7f26cc050610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26cc0a6040 0x7f26cc050610 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de45f840 msgr2=0x5587de461d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de45f840 0x5587de461d00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26d40123d0 msgr2=0x7f26d403b350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26d40123d0 0x7f26d403b350 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de464140 msgr2=0x5587de466530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de464140 0x5587de466530 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25a403d000 msgr2=0x7f25a403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25a403d000 0x7f25a403f4c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0873f0 tx=0x7f26cc093000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 msgr2=0x5587de453fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26d8fc6640 1 -- 172.21.15.28:0/1946554705 reap_dead start 2024-09-19T11:23:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de453fd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f26d4020510 tx=0x7f26d40547c0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 shutdown_connections 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1946554705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de44c940 0x5587de453fd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 >> 172.21.15.28:0/1946554705 conn(0x5587de448d40 msgr2=0x5587de446f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 shutdown_connections 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.659+0000 7f26ddb39800 1 -- 172.21.15.28:0/1946554705 wait complete. 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25b403d000 msgr2=0x7f25b403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25b403d000 0x7f25b403f4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5587de424e00 tx=0x7f26cc096fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de41b6e0 msgr2=0x5587de419f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de41b6e0 0x5587de419f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x5587de423b90 tx=0x7f26d40167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 shutdown_connections 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25b403d000 0x7f25b403f4c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1260072863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de41b6e0 0x5587de419f50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 >> 172.21.15.28:0/1260072863 conn(0x5587de432030 msgr2=0x5587de432410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 shutdown_connections 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1260072863 wait complete. 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de32b8c0 msgr2=0x5587de42a350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de32b8c0 0x5587de42a350 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de42c6f0 msgr2=0x5587de42eb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de42c6f0 0x5587de42eb90 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25c003d000 msgr2=0x7f25c003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25c003d000 0x7f25c003f4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f26d0001e30 tx=0x7f26d0001390 comp rx=0 tx=0).stop 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 msgr2=0x5587de422d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de422d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f26cc067040 tx=0x7f26cc002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 shutdown_connections 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5587de42c6f0 0x5587de42eb90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f25c003d000 0x7f25c003f4c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5587de32b8c0 0x5587de42a350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 --2- 172.21.15.28:0/1170036981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5587de32a680 0x5587de422d00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 >> 172.21.15.28:0/1170036981 conn(0x5587de325a50 msgr2=0x5587de414380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 shutdown_connections 2024-09-19T11:23:21.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.660+0000 7f26ddb39800 1 -- 172.21.15.28:0/1170036981 wait complete. 2024-09-19T11:23:21.678 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[],"summary":[]} 2024-09-19T11:23:21.678 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-09-19T11:23:21.678 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:21.678 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:21.679 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.724+0000 7ff963c0e800 1 Processor -- start 2024-09-19T11:23:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.724+0000 7ff963c0e800 1 -- start start 2024-09-19T11:23:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.724+0000 7ff963c0e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x557e54bfaea0 0x557e54bfd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.724+0000 7ff963c0e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557e54c015e0 con 0x557e54bfd8c0 2024-09-19T11:23:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.724+0000 7ff963c0e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x557e54a857d0 con 0x557e54bfaea0 2024-09-19T11:23:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.725+0000 7ff95f41a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x557e54bfaea0 0x557e54bfd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.725+0000 7ff95ec19640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x557e54bfd8c0 0x557e54bffd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32976/0 (socket says 172.21.15.28:32976) 2024-09-19T11:23:21.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.725+0000 7ff95ec19640 1 -- 172.21.15.28:0/1490145559 learned_addr learned my addr 172.21.15.28:0/1490145559 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:21.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.725+0000 7ff95f41a640 1 -- 172.21.15.28:0/1490145559 >> v1:172.21.15.22:6789/0 conn(0x557e54bfd8c0 legacy=0x557e54bffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.725+0000 7ff95f41a640 1 -- 172.21.15.28:0/1490145559 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e54c02810 con 0x557e54bfaea0 2024-09-19T11:23:21.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff95f41a640 1 --2- 172.21.15.28:0/1490145559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54bfd290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff95000a470 tx=0x7ff950045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe07b66d60b2c30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff95e418640 1 -- 172.21.15.28:0/1490145559 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff950053570 con 0x557e54bfaea0 2024-09-19T11:23:21.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff95e418640 1 -- 172.21.15.28:0/1490145559 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff950058070 con 0x557e54bfaea0 2024-09-19T11:23:21.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff95e418640 1 -- 172.21.15.28:0/1490145559 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff950053880 con 0x557e54bfaea0 2024-09-19T11:23:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff963c0e800 1 -- 172.21.15.28:0/1490145559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 msgr2=0x557e54bfd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff963c0e800 1 --2- 172.21.15.28:0/1490145559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54bfd290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff95000a470 tx=0x7ff950045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff963c0e800 1 -- 172.21.15.28:0/1490145559 shutdown_connections 2024-09-19T11:23:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff963c0e800 1 --2- 172.21.15.28:0/1490145559 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54bfd290 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.727+0000 7ff963c0e800 1 -- 172.21.15.28:0/1490145559 >> 172.21.15.28:0/1490145559 conn(0x557e54bf4a60 msgr2=0x557e54bf6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.728+0000 7ff963c0e800 1 -- 172.21.15.28:0/1490145559 shutdown_connections 2024-09-19T11:23:21.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.728+0000 7ff963c0e800 1 -- 172.21.15.28:0/1490145559 wait complete. 2024-09-19T11:23:21.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff963c0e800 1 Processor -- start 2024-09-19T11:23:21.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff963c0e800 1 -- start start 2024-09-19T11:23:21.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff963c0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cea350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff963c0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e54a92dd0 con 0x557e54bfaea0 2024-09-19T11:23:21.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff95f41a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff95f41a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cea350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58864/0 (socket says 172.21.15.28:58864) 2024-09-19T11:23:21.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff95f41a640 1 -- 172.21.15.28:0/4206656182 learned_addr learned my addr 172.21.15.28:0/4206656182 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.743+0000 7ff95f41a640 1 -- 172.21.15.28:0/4206656182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e54cea940 con 0x557e54bfaea0 2024-09-19T11:23:21.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.744+0000 7ff95f41a640 1 --2- 172.21.15.28:0/4206656182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cea350 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff950025c20 tx=0x7ff950051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af13580381b9c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.744+0000 7ff84d60a640 1 -- 172.21.15.28:0/4206656182 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95005c580 con 0x557e54bfaea0 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.744+0000 7ff84d60a640 1 -- 172.21.15.28:0/4206656182 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff950063030 con 0x557e54bfaea0 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.744+0000 7ff84d60a640 1 -- 172.21.15.28:0/4206656182 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95005c850 con 0x557e54bfaea0 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.744+0000 7ff963c0e800 1 -- 172.21.15.28:0/4206656182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 msgr2=0x557e54cea350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.744+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4206656182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cea350 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff950025c20 tx=0x7ff950051000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 -- 172.21.15.28:0/4206656182 shutdown_connections 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4206656182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cea350 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 -- 172.21.15.28:0/4206656182 >> 172.21.15.28:0/4206656182 conn(0x557e54bf4a60 msgr2=0x557e54ce5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 -- 172.21.15.28:0/4206656182 shutdown_connections 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 -- 172.21.15.28:0/4206656182 wait complete. 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 Processor -- start 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 -- start start 2024-09-19T11:23:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cf1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff963c0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e54c01ab0 con 0x557e54bfaea0 2024-09-19T11:23:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff95f41a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cf1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff95f41a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cf1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58880/0 (socket says 172.21.15.28:58880) 2024-09-19T11:23:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.745+0000 7ff95f41a640 1 -- 172.21.15.28:0/4223077550 learned_addr learned my addr 172.21.15.28:0/4223077550 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.746+0000 7ff95f41a640 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e54cf22f0 con 0x557e54bfaea0 2024-09-19T11:23:21.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.746+0000 7ff95f41a640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cf1d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff950045f50 tx=0x7ff950002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.746+0000 7ff84be07640 1 -- 172.21.15.28:0/4223077550 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff950054ad0 con 0x557e54bfaea0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.746+0000 7ff84be07640 1 -- 172.21.15.28:0/4223077550 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff950054c70 con 0x557e54bfaea0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.746+0000 7ff84be07640 1 -- 172.21.15.28:0/4223077550 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff950068420 con 0x557e54bfaea0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.746+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557e54cf2580 con 0x557e54bfaea0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.746+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e54cf2f80 con 0x557e54bfaea0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.747+0000 7ff84be07640 1 -- 172.21.15.28:0/4223077550 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff9500685c0 con 0x557e54bfaea0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.747+0000 7ff84be07640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff84403d000 0x7ff84403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.747+0000 7ff84be07640 1 -- 172.21.15.28:0/4223077550 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7ff950070030 con 0x557e54bfaea0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.747+0000 7ff95ec19640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff84403d000 0x7ff84403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.747+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54bfd8c0 0x557e54cf93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.747+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54cf9920 con 0x557e54bfd8c0 2024-09-19T11:23:21.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54bfd8c0 0x557e54cf93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff95ec19640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff84403d000 0x7ff84403f4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff954001d70 tx=0x7ff9540012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54bfd8c0 0x557e54cf93e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff95fc1b640 1 -- 172.21.15.28:0/4223077550 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff95800a4d0 con 0x557e54bfd8c0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54cf9cc0 con 0x557e54bfd8c0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff95fc1b640 1 -- 172.21.15.28:0/4223077550 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff95800a4d0 con 0x557e54bfd8c0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54cfb780 0x557e54cfdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.748+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54cfe3b0 con 0x557e54cfb780 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.749+0000 7ff95f41a640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54cfb780 0x557e54cfdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.749+0000 7ff95f41a640 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54cfb780 0x557e54cfdc20 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.749+0000 7ff95f41a640 1 -- 172.21.15.28:0/4223077550 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff950051070 con 0x557e54cfb780 2024-09-19T11:23:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.749+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54cfa510 con 0x557e54bfd8c0 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.750+0000 7ff95fc1b640 1 -- 172.21.15.28:0/4223077550 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff95800a4d0 con 0x557e54bfd8c0 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.750+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54cfa8b0 con 0x557e54cfb780 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.750+0000 7ff95f41a640 1 -- 172.21.15.28:0/4223077550 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff950051070 con 0x557e54cfb780 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.750+0000 7ff963c0e800 1 Processor -- start 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.750+0000 7ff963c0e800 1 -- start start 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.750+0000 7ff963c0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54d0cd10 0x557e54d0d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.750+0000 7ff963c0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e54af14a0 con 0x557e54d0cd10 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff95fc1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54d0cd10 0x557e54d0d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff95fc1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54d0cd10 0x557e54d0d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58888/0 (socket says 172.21.15.28:58888) 2024-09-19T11:23:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff95fc1b640 1 -- 172.21.15.28:0/3196060645 learned_addr learned my addr 172.21.15.28:0/3196060645 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff95fc1b640 1 -- 172.21.15.28:0/3196060645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e54d0d770 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54d0cd10 0x557e54d0d130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x557e54cf3150 tx=0x7ff9580167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff849602640 1 -- 172.21.15.28:0/3196060645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff958013640 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff849602640 1 -- 172.21.15.28:0/3196060645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9580137e0 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557e54d0dfa0 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff849602640 1 -- 172.21.15.28:0/3196060645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff958013ae0 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.751+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e54d0e540 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.752+0000 7ff849602640 1 -- 172.21.15.28:0/3196060645 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff958011040 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.752+0000 7ff849602640 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83803d000 0x7ff83803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.752+0000 7ff849602640 1 -- 172.21.15.28:0/3196060645 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7ff95800e040 con 0x557e54d0cd10 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.752+0000 7ff95f41a640 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83803d000 0x7ff83803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff963c0e800 1 Processor -- start 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff95f41a640 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83803d000 0x7ff83803f4c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557e54cf3e90 tx=0x7ff950068910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff963c0e800 1 -- start start 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff963c0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54bf9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff963c0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e54cead10 con 0x557e54bf8ea0 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff95fc1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54bf9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff95fc1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54bf9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58900/0 (socket says 172.21.15.28:58900) 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff95fc1b640 1 -- 172.21.15.28:0/4002990039 learned_addr learned my addr 172.21.15.28:0/4002990039 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff95fc1b640 1 -- 172.21.15.28:0/4002990039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e54bf9870 con 0x557e54bf8ea0 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/4002990039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54bf9280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff958054860 tx=0x7ff958020ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df6751587289b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.753+0000 7ff842ffd640 1 -- 172.21.15.28:0/4002990039 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff958044030 con 0x557e54bf8ea0 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff842ffd640 1 -- 172.21.15.28:0/4002990039 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff958021960 con 0x557e54bf8ea0 2024-09-19T11:23:21.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff842ffd640 1 -- 172.21.15.28:0/4002990039 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff95804de60 con 0x557e54bf8ea0 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 -- 172.21.15.28:0/4002990039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 msgr2=0x557e54bf9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4002990039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54bf9280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff958054860 tx=0x7ff958020ad0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 -- 172.21.15.28:0/4002990039 shutdown_connections 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4002990039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54bf9280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 -- 172.21.15.28:0/4002990039 >> 172.21.15.28:0/4002990039 conn(0x557e54d15c10 msgr2=0x557e54d13e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 -- 172.21.15.28:0/4002990039 shutdown_connections 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 -- 172.21.15.28:0/4002990039 wait complete. 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 Processor -- start 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 -- start start 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54d22fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.754+0000 7ff963c0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557e54bf9f00 con 0x557e54bf8ea0 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff95fc1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54d22fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff95fc1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54d22fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58904/0 (socket says 172.21.15.28:58904) 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 learned_addr learned my addr 172.21.15.28:0/2351150485 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e54d23530 con 0x557e54bf8ea0 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54d22fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff958020510 tx=0x7ff9580547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff8417fa640 1 -- 172.21.15.28:0/2351150485 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff958054a30 con 0x557e54bf8ea0 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff8417fa640 1 -- 172.21.15.28:0/2351150485 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9580143c0 con 0x557e54bf8ea0 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557e54d237c0 con 0x557e54bf8ea0 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff8417fa640 1 -- 172.21.15.28:0/2351150485 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff958020bd0 con 0x557e54bf8ea0 2024-09-19T11:23:21.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.755+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e54d24350 con 0x557e54bf8ea0 2024-09-19T11:23:21.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.756+0000 7ff8417fa640 1 -- 172.21.15.28:0/2351150485 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff958020d70 con 0x557e54bf8ea0 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.756+0000 7ff8417fa640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82803d050 0x7ff82803f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.756+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.756+0000 7ff8417fa640 1 -- 172.21.15.28:0/2351150485 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7ff95804b070 con 0x557e54bf8ea0 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.756+0000 7ff95f41a640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82803d050 0x7ff82803f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54d2e910 0x557e54d30da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d314f0 con 0x557e54d2e910 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff95ec19640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54d2e910 0x557e54d30da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff95f41a640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82803d050 0x7ff82803f510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff950000c00 tx=0x7ff95008e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff95ec19640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54d2e910 0x557e54d30da0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff954009050 con 0x557e54d2e910 2024-09-19T11:23:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff963c0e800 20 realm 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.757+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d2dd20 con 0x557e54d2e910 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff954009050 con 0x557e54d2e910 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d31fd0 con 0x557e54d2e910 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff954009050 con 0x557e54d2e910 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d328a0 con 0x557e54d2e910 2024-09-19T11:23:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff954009050 con 0x557e54d2e910 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54d331e0 0x557e54d355d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d35d10 con 0x557e54d331e0 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.758+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54d331e0 0x557e54d355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54d331e0 0x557e54d355d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff963c0e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff963c0e800 20 searching for the correct realm 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff95fc1b640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff9580123d0 0x7ff95803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff9580123d0 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.759+0000 7ff95f41a640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff9580123d0 0x7ff95803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.760+0000 7ff95f41a640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff9580123d0 0x7ff95803b350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.760+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.760+0000 7ff95f41a640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff95005c070 0x7ff9500951d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.760+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.760+0000 7ff95ec19640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff95005c070 0x7ff9500951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.760+0000 7ff95ec19640 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff95005c070 0x7ff9500951d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954009050 con 0x7ff95005c070 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954009050 con 0x557e54d2e910 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.761+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff9580123d0 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.762+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff9580123d0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.763+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.764+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.764+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.764+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.764+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.764+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.764+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.765+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff9580123d0 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.766+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.766+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.766+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.766+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.766+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.766+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff9580123d0 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d331e0 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.767+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff95005c070 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x557e54d2e910 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x557e54d376c0 con 0x7ff9580123d0 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d36790 con 0x557e54d2e910 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.768+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d37030 con 0x557e54d2e910 2024-09-19T11:23:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d363d0 con 0x557e54d331e0 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 zone default found 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 4 Realm: () 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 zonegroup default 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d36c60 con 0x7ff95005c070 2024-09-19T11:23:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.769+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d39f60 con 0x557e54d331e0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3a300 con 0x557e54d331e0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff963c0e800 20 started sync module instance, tier type = 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff963c0e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.770+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d393f0 con 0x7ff9580123d0 2024-09-19T11:23:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.771+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.771+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94001077393424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3abe0 con 0x7ff9580123d0 2024-09-19T11:23:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.771+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3b430 con 0x557e54d2e910 2024-09-19T11:23:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.772+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.772+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94001077399424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3ba30 con 0x557e54d2e910 2024-09-19T11:23:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.772+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3c530 con 0x557e54d331e0 2024-09-19T11:23:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.772+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94001077393424] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.773+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.773+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94001077403680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3c4f0 con 0x557e54d331e0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.773+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3cfc0 con 0x7ff9580123d0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.774+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94001077399424] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.774+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94001077403680] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.775+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.775+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94001077397296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3b480 con 0x7ff9580123d0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.775+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3d980 con 0x557e54d2e910 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.776+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.776+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94001077397296] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.776+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94001077409456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3e120 con 0x557e54d2e910 2024-09-19T11:23:21.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.776+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3cb00 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.777+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94001077409456] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.777+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.777+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94001077410672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3cb00 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.777+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3f1e0 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.778+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.778+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94001077415216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3f600 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.778+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d40050 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.778+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94001077410672] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94001077415216] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94001077418816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d3e710 con 0x7ff95005c070 2024-09-19T11:23:21.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 20 add_watcher() i=0 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 20 add_watcher() i=1 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 20 add_watcher() i=2 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 20 add_watcher() i=3 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 20 add_watcher() i=4 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 20 add_watcher() i=5 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.779+0000 7ff963c0e800 20 add_watcher() i=6 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.780+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94001077418816] v31'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.780+0000 7ff963c0e800 20 add_watcher() i=7 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.780+0000 7ff963c0e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.780+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d43040 con 0x557e54d2e910 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.780+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.780+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94001077416272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d40500 con 0x557e54d2e910 2024-09-19T11:23:21.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.781+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94001077416272] v31'48 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.781+0000 7ff815fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.781+0000 7ff963c0e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.781+0000 7ff963c0e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.781+0000 7ff963c0e800 5 note: GC not initialized 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff814fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff963c0e800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff963c0e800 20 Filter name: none 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff78e7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff963c0e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x557e54d75930 con 0x557e54d2e910 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.782+0000 7ff963c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.794 INFO:teuthology.orchestra.run.smithi028.stderr:could not fetch user info: no user info saved 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.783+0000 7ff815fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.783+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94001077416272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x557e54d2e910 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.784+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94001077416272] v31'49 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.784+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94001077393424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x7ff9580123d0 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.785+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94001077393424] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.785+0000 7ff963c0e800 20 remove_watcher() i=0 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.785+0000 7ff963c0e800 2 removed watcher, disabling cache 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.785+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94001077399424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x557e54d2e910 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.786+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94001077399424] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.787+0000 7ff963c0e800 20 remove_watcher() i=1 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.787+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94001077403680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x557e54d331e0 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.787+0000 7ff95fc1b640 1 -- 172.21.15.28:0/2351150485 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94001077403680] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff95803a070 con 0x557e54d331e0 2024-09-19T11:23:21.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.787+0000 7ff963c0e800 20 remove_watcher() i=2 2024-09-19T11:23:21.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.787+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94001077397296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x7ff9580123d0 2024-09-19T11:23:21.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.789+0000 7ff95f41a640 1 -- 172.21.15.28:0/2351150485 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94001077397296] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff9500a1030 con 0x7ff9580123d0 2024-09-19T11:23:21.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.789+0000 7ff963c0e800 20 remove_watcher() i=3 2024-09-19T11:23:21.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.789+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94001077409456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x557e54d2e910 2024-09-19T11:23:21.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.790+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94001077409456] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x557e54d2e910 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.790+0000 7ff963c0e800 20 remove_watcher() i=4 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.790+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94001077410672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x7ff95005c070 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.791+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94001077410672] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.791+0000 7ff963c0e800 20 remove_watcher() i=5 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.791+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94001077415216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x7ff95005c070 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.792+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94001077415216] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.792+0000 7ff963c0e800 20 remove_watcher() i=6 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.792+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94001077418816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x557e54d758f0 con 0x7ff95005c070 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94001077418816] v31'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff954029040 con 0x7ff95005c070 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 20 remove_watcher() i=7 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff95005c070 msgr2=0x7ff9500951d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff95005c070 0x7ff9500951d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54d2e910 msgr2=0x557e54d30da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54d2e910 0x557e54d30da0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff9580123d0 msgr2=0x7ff95803b350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff9580123d0 0x7ff95803b350 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54d331e0 msgr2=0x557e54d355d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54d331e0 0x557e54d355d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82803d050 msgr2=0x7ff82803f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82803d050 0x7ff82803f510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff950000c00 tx=0x7ff95008e000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 msgr2=0x557e54d22fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff95ec19640 1 -- 172.21.15.28:0/2351150485 reap_dead start 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54d22fd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff958020510 tx=0x7ff9580547c0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 shutdown_connections 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 --2- 172.21.15.28:0/2351150485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bf8ea0 0x557e54d22fd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 >> 172.21.15.28:0/2351150485 conn(0x557e54d15c10 msgr2=0x557e54d13e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 shutdown_connections 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.793+0000 7ff963c0e800 1 -- 172.21.15.28:0/2351150485 wait complete. 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83803d000 msgr2=0x7ff83803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83803d000 0x7ff83803f4c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x557e54cf3e90 tx=0x7ff950068910 comp rx=0 tx=0).stop 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54d0cd10 msgr2=0x557e54d0d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54d0cd10 0x557e54d0d130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x557e54cf3150 tx=0x7ff9580167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 shutdown_connections 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83803d000 0x7ff83803f4c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/3196060645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54d0cd10 0x557e54d0d130 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 >> 172.21.15.28:0/3196060645 conn(0x557e54d010c0 msgr2=0x557e54d014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 shutdown_connections 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/3196060645 wait complete. 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54bfd8c0 msgr2=0x557e54cf93e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54bfd8c0 0x557e54cf93e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54cfb780 msgr2=0x557e54cfdc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54cfb780 0x557e54cfdc20 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff84403d000 msgr2=0x7ff84403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff84403d000 0x7ff84403f4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff954001d70 tx=0x7ff9540012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 msgr2=0x557e54cf1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cf1d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff950045f50 tx=0x7ff950002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 shutdown_connections 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557e54cfb780 0x557e54cfdc20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff84403d000 0x7ff84403f4c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557e54bfd8c0 0x557e54cf93e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 --2- 172.21.15.28:0/4223077550 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557e54bfaea0 0x557e54cf1d90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 >> 172.21.15.28:0/4223077550 conn(0x557e54bf4a60 msgr2=0x557e54ce44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 shutdown_connections 2024-09-19T11:23:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.794+0000 7ff963c0e800 1 -- 172.21.15.28:0/4223077550 wait complete. 2024-09-19T11:23:21.804 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T11:23:21.805 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-09-19T11:23:21.805 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-09-19T11:23:21.805 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-09-19T11:23:21.862 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:21.862 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.861+0000 7fd3fb32c800 1 Processor -- start 2024-09-19T11:23:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.861+0000 7fd3fb32c800 1 -- start start 2024-09-19T11:23:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.861+0000 7fd3fb32c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c7f24b1bb0 0x55c7f24b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.861+0000 7fd3fb32c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c7f24b0f90 con 0x55c7f24b08e0 2024-09-19T11:23:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.861+0000 7fd3fb32c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55c7f233b7d0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.862+0000 7fd3f68ac640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c7f24b1bb0 0x55c7f24b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.862+0000 7fd3f68ac640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c7f24b1bb0 0x55c7f24b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58918/0 (socket says 172.21.15.28:58918) 2024-09-19T11:23:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.862+0000 7fd3f68ac640 1 -- 172.21.15.28:0/1600570546 learned_addr learned my addr 172.21.15.28:0/1600570546 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.862+0000 7fd3f70ad640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55c7f24b08e0 0x55c7f24b14a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:32988/0 (socket says 172.21.15.28:32988) 2024-09-19T11:23:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.862+0000 7fd3f68ac640 1 -- 172.21.15.28:0/1600570546 >> v1:172.21.15.22:6789/0 conn(0x55c7f24b08e0 legacy=0x55c7f24b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.862+0000 7fd3f60ab640 1 -- 172.21.15.28:0/1600570546 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306013934 0 0) 0x55c7f24b0f90 con 0x55c7f24b08e0 2024-09-19T11:23:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.862+0000 7fd3f68ac640 1 -- 172.21.15.28:0/1600570546 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f24b2060 con 0x55c7f24b1bb0 2024-09-19T11:23:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3f68ac640 1 --2- 172.21.15.28:0/1600570546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b1bb0 0x55c7f24b7750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000a520 tx=0x7fd3e0045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311aa78ed573521 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3f60ab640 1 -- 172.21.15.28:0/1600570546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e0055070 con 0x55c7f24b1bb0 2024-09-19T11:23:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3f60ab640 1 -- 172.21.15.28:0/1600570546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e00515b0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3f60ab640 1 -- 172.21.15.28:0/1600570546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e00518d0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3fb32c800 1 -- 172.21.15.28:0/1600570546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b1bb0 msgr2=0x55c7f24b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/1600570546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b1bb0 0x55c7f24b7750 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000a520 tx=0x7fd3e0045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3fb32c800 1 -- 172.21.15.28:0/1600570546 shutdown_connections 2024-09-19T11:23:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/1600570546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b1bb0 0x55c7f24b7750 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3fb32c800 1 -- 172.21.15.28:0/1600570546 >> 172.21.15.28:0/1600570546 conn(0x55c7f24abcb0 msgr2=0x55c7f24ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3fb32c800 1 -- 172.21.15.28:0/1600570546 shutdown_connections 2024-09-19T11:23:21.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.864+0000 7fd3fb32c800 1 -- 172.21.15.28:0/1600570546 wait complete. 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3fb32c800 1 Processor -- start 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3fb32c800 1 -- start start 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3fb32c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3fb32c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c7f2348dd0 con 0x55c7f24b08e0 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3f70ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3f70ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58922/0 (socket says 172.21.15.28:58922) 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2609767530 learned_addr learned my addr 172.21.15.28:0/2609767530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2609767530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f25a0840 con 0x55c7f24b08e0 2024-09-19T11:23:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.880+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/2609767530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a0250 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8006fa0 tx=0x7fd3e8001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47c4284c1f9165d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd2e560a640 1 -- 172.21.15.28:0/2609767530 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8003450 con 0x55c7f24b08e0 2024-09-19T11:23:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd2e560a640 1 -- 172.21.15.28:0/2609767530 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8003610 con 0x55c7f24b08e0 2024-09-19T11:23:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd2e560a640 1 -- 172.21.15.28:0/2609767530 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8003970 con 0x55c7f24b08e0 2024-09-19T11:23:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2609767530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 msgr2=0x55c7f25a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2609767530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a0250 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8006fa0 tx=0x7fd3e8001b00 comp rx=0 tx=0).stop 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2609767530 shutdown_connections 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2609767530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a0250 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2609767530 >> 172.21.15.28:0/2609767530 conn(0x55c7f24abcb0 msgr2=0x55c7f259bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2609767530 shutdown_connections 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.881+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2609767530 wait complete. 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3fb32c800 1 Processor -- start 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3fb32c800 1 -- start start 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3fb32c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3fb32c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c7f24b7ea0 con 0x55c7f24b08e0 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3f70ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3f70ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58924/0 (socket says 172.21.15.28:58924) 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3f70ad640 1 -- 172.21.15.28:0/151502371 learned_addr learned my addr 172.21.15.28:0/151502371 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3f70ad640 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f25a82b0 con 0x55c7f24b08e0 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.882+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a7d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8001330 tx=0x7fd3e8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.883+0000 7fd2e3e07640 1 -- 172.21.15.28:0/151502371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e8014a10 con 0x55c7f24b08e0 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.883+0000 7fd2e3e07640 1 -- 172.21.15.28:0/151502371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8014bb0 con 0x55c7f24b08e0 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.883+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c7f25a8540 con 0x55c7f24b08e0 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.883+0000 7fd2e3e07640 1 -- 172.21.15.28:0/151502371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3e801e3f0 con 0x55c7f24b08e0 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.883+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f25a8f40 con 0x55c7f24b08e0 2024-09-19T11:23:21.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd2e3e07640 1 -- 172.21.15.28:0/151502371 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd3e8010070 con 0x55c7f24b08e0 2024-09-19T11:23:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd2e3e07640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2dc03d000 0x7fd2dc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd2e3e07640 1 -- 172.21.15.28:0/151502371 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd3e8018070 con 0x55c7f24b08e0 2024-09-19T11:23:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd3f68ac640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2dc03d000 0x7fd2dc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f24b1bb0 0x55c7f25af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25af8e0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f24b1bb0 0x55c7f25af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd3f68ac640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2dc03d000 0x7fd2dc03f4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0057000 tx=0x7fd3e0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.884+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f24b1bb0 0x55c7f25af3a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.885+0000 7fd3f78ae640 1 -- 172.21.15.28:0/151502371 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.885+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25afc80 con 0x55c7f24b1bb0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.885+0000 7fd3f78ae640 1 -- 172.21.15.28:0/151502371 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.885+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25b1780 0x55c7f25b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.885+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25b4390 con 0x55c7f25b1780 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.885+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25b1780 0x55c7f25b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.885+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25b1780 0x55c7f25b3c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.886+0000 7fd3f70ad640 1 -- 172.21.15.28:0/151502371 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd3e801b070 con 0x55c7f25b1780 2024-09-19T11:23:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.886+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25b04d0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.886+0000 7fd3f78ae640 1 -- 172.21.15.28:0/151502371 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f24b1bb0 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.886+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25b0920 con 0x55c7f25b1780 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.886+0000 7fd3f70ad640 1 -- 172.21.15.28:0/151502371 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd3e801b070 con 0x55c7f25b1780 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3fb32c800 1 Processor -- start 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3fb32c800 1 -- start start 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3fb32c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f2426200 0x55c7f2424a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3fb32c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c7f23a74a0 con 0x55c7f2426200 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3f78ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f2426200 0x55c7f2424a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3f78ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f2426200 0x55c7f2424a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58936/0 (socket says 172.21.15.28:58936) 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3f78ae640 1 -- 172.21.15.28:0/3422077627 learned_addr learned my addr 172.21.15.28:0/3422077627 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3f78ae640 1 -- 172.21.15.28:0/3422077627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f2426720 con 0x55c7f2426200 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.887+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f2426200 0x55c7f2424a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55c7f25a9c80 tx=0x7fd3f000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.888+0000 7fd2e1602640 1 -- 172.21.15.28:0/3422077627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f001c020 con 0x55c7f2426200 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.888+0000 7fd2e1602640 1 -- 172.21.15.28:0/3422077627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f0007c90 con 0x55c7f2426200 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.888+0000 7fd2e1602640 1 -- 172.21.15.28:0/3422077627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f0018440 con 0x55c7f2426200 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.888+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c7f2424fb0 con 0x55c7f2426200 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.888+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f2425550 con 0x55c7f2426200 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.888+0000 7fd2e1602640 1 -- 172.21.15.28:0/3422077627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd3f001f020 con 0x55c7f2426200 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd2e1602640 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2d003d000 0x7fd2d003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd2e1602640 1 -- 172.21.15.28:0/3422077627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd3f0055a70 con 0x55c7f2426200 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2d003d000 0x7fd2d003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3fb32c800 1 Processor -- start 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3fb32c800 1 -- start start 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2d003d000 0x7fd2d003f4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55c7f25b0870 tx=0x7fd3e8014fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3fb32c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3fb32c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c7f25a0c70 con 0x55c7f25d19d0 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3f78ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3f78ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58946/0 (socket says 172.21.15.28:58946) 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.889+0000 7fd3f78ae640 1 -- 172.21.15.28:0/3942918672 learned_addr learned my addr 172.21.15.28:0/3942918672 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd3f78ae640 1 -- 172.21.15.28:0/3942918672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f25d23c0 con 0x55c7f25d19d0 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/3942918672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d1dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0053df0 tx=0x7fd3f0053fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6994b8089f2b4080 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd2daffd640 1 -- 172.21.15.28:0/3942918672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f0051070 con 0x55c7f25d19d0 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd2daffd640 1 -- 172.21.15.28:0/3942918672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f0016c50 con 0x55c7f25d19d0 2024-09-19T11:23:21.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd2daffd640 1 -- 172.21.15.28:0/3942918672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f0013320 con 0x55c7f25d19d0 2024-09-19T11:23:21.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3942918672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 msgr2=0x55c7f25d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/3942918672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d1dd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0053df0 tx=0x7fd3f0053fc0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3942918672 shutdown_connections 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.890+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/3942918672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d1dd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3942918672 >> 172.21.15.28:0/3942918672 conn(0x55c7f25cddd0 msgr2=0x55c7f25cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3942918672 shutdown_connections 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3942918672 wait complete. 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3fb32c800 1 Processor -- start 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3fb32c800 1 -- start start 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3fb32c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3fb32c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c7f25d29b0 con 0x55c7f25d19d0 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3f78ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3f78ae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58948/0 (socket says 172.21.15.28:58948) 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.891+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 learned_addr learned my addr 172.21.15.28:0/2697529990 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.892+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f25d94c0 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.892+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d8f60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0018fd0 tx=0x7fd3f004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.892+0000 7fd2d97fa640 1 -- 172.21.15.28:0/2697529990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f0013ea0 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.892+0000 7fd2d97fa640 1 -- 172.21.15.28:0/2697529990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f002e070 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.892+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c7f25d9750 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.892+0000 7fd2d97fa640 1 -- 172.21.15.28:0/2697529990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f00208c0 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.892+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f25da2e0 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.893+0000 7fd2d97fa640 1 -- 172.21.15.28:0/2697529990 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd3f0016040 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.893+0000 7fd2d97fa640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2c003d000 0x7fd2c003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.893+0000 7fd2d97fa640 1 -- 172.21.15.28:0/2697529990 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd3f0055040 con 0x55c7f25d19d0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.893+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2c003d000 0x7fd2c003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.893+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2c003d000 0x7fd2c003f4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8002020 tx=0x7fd3e8035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.893+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f25e48e0 0x55c7f25e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25e74f0 con 0x55c7f25e48e0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3f68ac640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f25e48e0 0x55c7f25e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3f68ac640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f25e48e0 0x55c7f25e6da0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3fb32c800 20 realm 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.894+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25e7920 con 0x55c7f25e48e0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25e81f0 con 0x55c7f25e48e0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25e8590 con 0x55c7f25e48e0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25e92f0 0x55c7f25eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ebcf0 con 0x55c7f25e92f0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25e92f0 0x55c7f25eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.895+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25e92f0 0x55c7f25eb6e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3fb32c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3fb32c800 20 searching for the correct realm 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3f78ae640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd3f0021020 0x7fd3f0021860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3f0021020 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd3f0021020 0x7fd3f0021860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.896+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd3f0021020 0x7fd3f0021860 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.897+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.897+0000 7fd3f70ad640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3e8055850 0x7fd3e8053f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.897+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.897+0000 7fd3f68ac640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3e8055850 0x7fd3e8053f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.897+0000 7fd3f68ac640 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3e8055850 0x7fd3e8053f10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.897+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.897+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e48e0 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e48e0 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.898+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3f0021020 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e48e0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.899+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e48e0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3f0021020 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.900+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e48e0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.901+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e48e0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.902+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.902+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.902+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.902+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3f0021020 2024-09-19T11:23:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e48e0 2024-09-19T11:23:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.903+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3e8055850 2024-09-19T11:23:21.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x7fd3f0021020 2024-09-19T11:23:21.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ed620 con 0x55c7f25e92f0 2024-09-19T11:23:21.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.904+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7fd3f006b040 con 0x7fd3e8055850 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7fd3f006b040 con 0x55c7f25e48e0 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7fd3f006b040 con 0x7fd3f0021020 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ec5b0 con 0x55c7f25e48e0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.905+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ec950 con 0x55c7f25e48e0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ecd70 con 0x55c7f25e92f0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 zone default found 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 4 Realm: () 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 zonegroup default 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25edb10 con 0x7fd3e8055850 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.906+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25efdf0 con 0x55c7f25e92f0 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25efdf0 con 0x55c7f25e92f0 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3fb32c800 20 started sync module instance, tier type = 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3fb32c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.907+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25ef280 con 0x7fd3f0021020 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.908+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.908+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94317253162608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f0220 con 0x7fd3f0021020 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.908+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f0a70 con 0x55c7f25e48e0 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.909+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94317253162608] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.909+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.909+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94317253163664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f0b50 con 0x55c7f25e48e0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.909+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f1650 con 0x55c7f25e92f0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.910+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94317253163664] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.911+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.911+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94317253169472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f1610 con 0x55c7f25e92f0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.911+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f2100 con 0x7fd3f0021020 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.911+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94317253169472] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.912+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.912+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94317253166528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f2100 con 0x7fd3f0021020 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.912+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f2ba0 con 0x55c7f25e48e0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.913+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94317253166528] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.913+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.913+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94317253174928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f2b60 con 0x55c7f25e48e0 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.913+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f3610 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.914+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94317253174928] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.914+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.914+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94317253172192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f3610 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.914+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f4050 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.915+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94317253172192] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.915+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.915+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94317253177696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f4230 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.915+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f4a80 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94317253177696] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94317253181280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f48d0 con 0x7fd3e8055850 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 20 add_watcher() i=0 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 20 add_watcher() i=1 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 20 add_watcher() i=2 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 20 add_watcher() i=3 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 20 add_watcher() i=4 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 20 add_watcher() i=5 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.916+0000 7fd3fb32c800 20 add_watcher() i=6 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.917+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94317253181280] v31'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0057030 con 0x7fd3e8055850 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.917+0000 7fd3fb32c800 20 add_watcher() i=7 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.917+0000 7fd3fb32c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.917+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f7e40 con 0x55c7f25e48e0 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.918+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd3e0057030 con 0x55c7f25e48e0 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.918+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94317253182336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f25f5360 con 0x55c7f25e48e0 2024-09-19T11:23:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94317253182336] v31'50 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd2adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 5 note: GC not initialized 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd2acfe1640 20 reqs_thread_entry: start 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 20 Filter name: none 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd2267fc640 20 reqs_thread_entry: start 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.919+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f262ace0 con 0x55c7f25e48e0 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2629380 con 0x55c7f25e48e0 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2629380 con 0x7fd3e8055850 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x7fd3e8055850 2024-09-19T11:23:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-09-19T11:23:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-09-19T11:23:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-09-19T11:23:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-09-19T11:23:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.920+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2629cd0 con 0x55c7f25e48e0 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.921+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~391] v31'1 uv1 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3fb32c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3fb32c800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3fb32c800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94317253400560 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f262e5d0 con 0x55c7f25e48e0 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify (1) cookie 94317253174928 notify 133143986176 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55c7f262a070 con 0x55c7f25e48e0 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd2daffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986176 cookie 94317253174928 notifier 4292 bl.length()=567 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd2daffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd2daffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 notify.4 [notify cookie 94317253400560 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd2daffd640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fd2bc003ea0 con 0x55c7f25e48e0 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== watch-notify(notify_complete (2) cookie 94317253400560 notify 133143986176 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c7f262a800 con 0x55c7f25e48e0 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.922+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2629780 con 0x55c7f25e48e0 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.923+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.923+0000 7fd3fb32c800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.923+0000 7fd3fb32c800 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.923+0000 7fd3fb32c800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.923+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94317253400448 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f262e9e0 con 0x7fd3f0021020 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== watch-notify(notify (1) cookie 94317253166528 notify 133143986176 ret 0) ==== 211+0+0 (crc 0 0 0) 0x7fd3e805c040 con 0x7fd3f0021020 2024-09-19T11:23:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd2daffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986176 cookie 94317253166528 notifier 4292 bl.length()=169 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd2daffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd2daffd640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd2daffd640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fd2bc003ea0 con 0x7fd3f0021020 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(69 notify.3 [notify cookie 94317253400448 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== watch-notify(notify_complete (2) cookie 94317253400448 notify 133143986176 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fd3e804f040 con 0x7fd3f0021020 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.924+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2629780 con 0x7fd3e8055850 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x7fd3e8055850 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd3fb32c800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd3fb32c800 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd3fb32c800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94317253400448 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c7f262d2c0 con 0x7fd3f0021020 2024-09-19T11:23:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== watch-notify(notify (1) cookie 94317253166528 notify 133143986177 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7fd3e804f040 con 0x7fd3f0021020 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd2da7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986177 cookie 94317253166528 notifier 4292 bl.length()=177 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd2da7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd2da7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(72 notify.3 [notify cookie 94317253400448 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.925+0000 7fd2da7fc640 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fd22c003400 con 0x7fd3f0021020 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.926+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.926+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== watch-notify(notify_complete (2) cookie 94317253400448 notify 133143986177 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c7f25a8f40 con 0x7fd3f0021020 2024-09-19T11:23:21.938 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:21.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.927+0000 7fd2adfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:21.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.927+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94317253182336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x55c7f25e48e0 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.928+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94317253182336] v31'51 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.928+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94317253162608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x7fd3f0021020 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.929+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94317253162608] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.929+0000 7fd3fb32c800 20 remove_watcher() i=0 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.929+0000 7fd3fb32c800 2 removed watcher, disabling cache 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.929+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94317253163664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x55c7f25e48e0 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.930+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94317253163664] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.930+0000 7fd3fb32c800 20 remove_watcher() i=1 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.930+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94317253169472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x55c7f25e92f0 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.931+0000 7fd3f78ae640 1 -- 172.21.15.28:0/2697529990 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94317253169472] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3f00083e0 con 0x55c7f25e92f0 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.931+0000 7fd3fb32c800 20 remove_watcher() i=2 2024-09-19T11:23:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.931+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94317253166528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x7fd3f0021020 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.932+0000 7fd3f70ad640 1 -- 172.21.15.28:0/2697529990 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94317253166528] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e801b070 con 0x7fd3f0021020 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.932+0000 7fd3fb32c800 20 remove_watcher() i=3 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.932+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94317253174928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x55c7f25e48e0 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.933+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94317253174928] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x55c7f25e48e0 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.933+0000 7fd3fb32c800 20 remove_watcher() i=4 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.933+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94317253172192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x7fd3e8055850 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.934+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94317253172192] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x7fd3e8055850 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.934+0000 7fd3fb32c800 20 remove_watcher() i=5 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.934+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94317253177696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x7fd3e8055850 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.935+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94317253177696] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x7fd3e8055850 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.935+0000 7fd3fb32c800 20 remove_watcher() i=6 2024-09-19T11:23:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.935+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94317253181280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c7f2628af0 con 0x7fd3e8055850 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94317253181280] v31'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3e0080040 con 0x7fd3e8055850 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 20 remove_watcher() i=7 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3e8055850 msgr2=0x7fd3e8053f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3e8055850 0x7fd3e8053f10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f25e48e0 msgr2=0x55c7f25e6da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f25e48e0 0x55c7f25e6da0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd3f0021020 msgr2=0x7fd3f0021860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd3f0021020 0x7fd3f0021860 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25e92f0 msgr2=0x55c7f25eb6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25e92f0 0x55c7f25eb6e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2c003d000 msgr2=0x7fd2c003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2c003d000 0x7fd2c003f4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8002020 tx=0x7fd3e8035000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 msgr2=0x55c7f25d8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3f68ac640 1 -- 172.21.15.28:0/2697529990 reap_dead start 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d8f60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0018fd0 tx=0x7fd3f004e000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 shutdown_connections 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/2697529990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f25d19d0 0x55c7f25d8f60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 >> 172.21.15.28:0/2697529990 conn(0x55c7f25cddd0 msgr2=0x55c7f25cbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 shutdown_connections 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.936+0000 7fd3fb32c800 1 -- 172.21.15.28:0/2697529990 wait complete. 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2d003d000 msgr2=0x7fd2d003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2d003d000 0x7fd2d003f4c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55c7f25b0870 tx=0x7fd3e8014fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f2426200 msgr2=0x55c7f2424a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f2426200 0x55c7f2424a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55c7f25a9c80 tx=0x7fd3f000bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 shutdown_connections 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2d003d000 0x7fd2d003f4c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/3422077627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f2426200 0x55c7f2424a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 >> 172.21.15.28:0/3422077627 conn(0x55c7f25b7100 msgr2=0x55c7f25b74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 shutdown_connections 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/3422077627 wait complete. 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f24b1bb0 msgr2=0x55c7f25af3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f24b1bb0 0x55c7f25af3a0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25b1780 msgr2=0x55c7f25b3c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25b1780 0x55c7f25b3c00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2dc03d000 msgr2=0x7fd2dc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2dc03d000 0x7fd2dc03f4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0057000 tx=0x7fd3e0052000 comp rx=0 tx=0).stop 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 msgr2=0x55c7f25a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a7d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8001330 tx=0x7fd3e8001250 comp rx=0 tx=0).stop 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 shutdown_connections 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c7f25b1780 0x55c7f25b3c00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd2dc03d000 0x7fd2dc03f4c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c7f24b1bb0 0x55c7f25af3a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 --2- 172.21.15.28:0/151502371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c7f24b08e0 0x55c7f25a7d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 >> 172.21.15.28:0/151502371 conn(0x55c7f24abcb0 msgr2=0x55c7f25a1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 shutdown_connections 2024-09-19T11:23:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.937+0000 7fd3fb32c800 1 -- 172.21.15.28:0/151502371 wait complete. 2024-09-19T11:23:21.950 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:21.950 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-09-19T11:23:21.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-09-19T11:23:21.951 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-09-19T11:23:22.000 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.000 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.999+0000 7fa92ef2c800 1 Processor -- start 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:21.999+0000 7fa92ef2c800 1 -- start start 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92ef2c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5610f398afd0 0x5610f398d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92ef2c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5610f39915f0 con 0x5610f398d900 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92ef2c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5610f38157d0 con 0x5610f398afd0 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92ab3e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5610f398afd0 0x5610f398d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92a33d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5610f398d900 0x5610f398fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33000/0 (socket says 172.21.15.28:33000) 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92a33d640 1 -- 172.21.15.28:0/2697908933 learned_addr learned my addr 172.21.15.28:0/2697908933 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2697908933 >> v1:172.21.15.22:6789/0 conn(0x5610f398d900 legacy=0x5610f398fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa929b3c640 1 -- 172.21.15.28:0/2697908933 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821626253 0 0) 0x5610f39915f0 con 0x5610f398d900 2024-09-19T11:23:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.000+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2697908933 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3992820 con 0x5610f398afd0 2024-09-19T11:23:22.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.002+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2697908933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a470 tx=0x7fa91c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c3af060f20fea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.002+0000 7fa929b3c640 1 -- 172.21.15.28:0/2697908933 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c053570 con 0x5610f398afd0 2024-09-19T11:23:22.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.002+0000 7fa929b3c640 1 -- 172.21.15.28:0/2697908933 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c058070 con 0x5610f398afd0 2024-09-19T11:23:22.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.002+0000 7fa929b3c640 1 -- 172.21.15.28:0/2697908933 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c053880 con 0x5610f398afd0 2024-09-19T11:23:22.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.003+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2697908933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 msgr2=0x5610f398d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.003+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2697908933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398d3c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a470 tx=0x7fa91c045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.003+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2697908933 shutdown_connections 2024-09-19T11:23:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.003+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2697908933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398d3c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.003+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2697908933 >> 172.21.15.28:0/2697908933 conn(0x5610f3984a70 msgr2=0x5610f3986ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.003+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2697908933 shutdown_connections 2024-09-19T11:23:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.003+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2697908933 wait complete. 2024-09-19T11:23:22.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.018+0000 7fa92ef2c800 1 Processor -- start 2024-09-19T11:23:22.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.018+0000 7fa92ef2c800 1 -- start start 2024-09-19T11:23:22.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.018+0000 7fa92ef2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.018+0000 7fa92ef2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610f3822dd0 con 0x5610f398afd0 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.018+0000 7fa92ab3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.019+0000 7fa92ab3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58968/0 (socket says 172.21.15.28:58968) 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.019+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2278722116 learned_addr learned my addr 172.21.15.28:0/2278722116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.019+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2278722116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f398ad30 con 0x5610f398afd0 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.019+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2278722116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398a740 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa91c045f50 tx=0x7fa91c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e3bfccdefdbd8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.019+0000 7fa818e09640 1 -- 172.21.15.28:0/2278722116 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c053c20 con 0x5610f398afd0 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.019+0000 7fa818e09640 1 -- 172.21.15.28:0/2278722116 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c063030 con 0x5610f398afd0 2024-09-19T11:23:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.019+0000 7fa818e09640 1 -- 172.21.15.28:0/2278722116 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c053440 con 0x5610f398afd0 2024-09-19T11:23:22.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2278722116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 msgr2=0x5610f398a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2278722116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398a740 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa91c045f50 tx=0x7fa91c051000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2278722116 shutdown_connections 2024-09-19T11:23:22.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2278722116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f398a740 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2278722116 >> 172.21.15.28:0/2278722116 conn(0x5610f3984a70 msgr2=0x5610f3a75ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2278722116 shutdown_connections 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2278722116 wait complete. 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 Processor -- start 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 -- start start 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f3a83e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.020+0000 7fa92ef2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610f3991ac0 con 0x5610f398afd0 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa92ab3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f3a83e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa92ab3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f3a83e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58976/0 (socket says 172.21.15.28:58976) 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2832634113 learned_addr learned my addr 172.21.15.28:0/2832634113 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3a84360 con 0x5610f398afd0 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f3a83e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa91c045f20 tx=0x7fa91c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa817606640 1 -- 172.21.15.28:0/2832634113 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c053570 con 0x5610f398afd0 2024-09-19T11:23:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa817606640 1 -- 172.21.15.28:0/2832634113 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c053710 con 0x5610f398afd0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa817606640 1 -- 172.21.15.28:0/2832634113 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa91c068420 con 0x5610f398afd0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.021+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610f3a845f0 con 0x5610f398afd0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.022+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f3a84f70 con 0x5610f398afd0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.022+0000 7fa817606640 1 -- 172.21.15.28:0/2832634113 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa91c0685c0 con 0x5610f398afd0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.022+0000 7fa817606640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa81003d000 0x7fa81003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa817606640 1 -- 172.21.15.28:0/2832634113 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fa91c070070 con 0x5610f398afd0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92a33d640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa81003d000 0x7fa81003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f398d900 0x5610f3a8b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3a85bb0 con 0x5610f398d900 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92b33f640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f398d900 0x5610f3a8b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92a33d640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa81003d000 0x7fa81003f4c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa920000c00 tx=0x7fa9200013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92b33f640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f398d900 0x5610f3a8b450 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92b33f640 1 -- 172.21.15.28:0/2832634113 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa92400a4d0 con 0x5610f398d900 2024-09-19T11:23:22.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.023+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3a85490 con 0x5610f398d900 2024-09-19T11:23:22.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.024+0000 7fa92b33f640 1 -- 172.21.15.28:0/2832634113 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa92400a4d0 con 0x5610f398d900 2024-09-19T11:23:22.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.024+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3a8d8c0 0x5610f3a8fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.024+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3a90380 con 0x5610f3a8d8c0 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.024+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3a8d8c0 0x5610f3a8fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.024+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3a8d8c0 0x5610f3a8fcb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.024+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2832634113 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa91c051070 con 0x5610f3a8d8c0 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.024+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3a8c6a0 con 0x5610f398d900 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.025+0000 7fa92b33f640 1 -- 172.21.15.28:0/2832634113 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa92400a4d0 con 0x5610f398d900 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.025+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3a8cb00 con 0x5610f3a8d8c0 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.025+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2832634113 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa91c051070 con 0x5610f3a8d8c0 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.025+0000 7fa92ef2c800 1 Processor -- start 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.025+0000 7fa92ef2c800 1 -- start start 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.026+0000 7fa92ef2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3a9ec30 0x5610f3a9f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.026+0000 7fa92ef2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610f38814a0 con 0x5610f3a9ec30 2024-09-19T11:23:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.026+0000 7fa92b33f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3a9ec30 0x5610f3a9f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.026+0000 7fa92b33f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3a9ec30 0x5610f3a9f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58984/0 (socket says 172.21.15.28:58984) 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.026+0000 7fa92b33f640 1 -- 172.21.15.28:0/220753835 learned_addr learned my addr 172.21.15.28:0/220753835 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.026+0000 7fa92b33f640 1 -- 172.21.15.28:0/220753835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3a9f690 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.026+0000 7fa92b33f640 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3a9ec30 0x5610f3a9f050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5610f3a85140 tx=0x7fa9240167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.027+0000 7fa814e01640 1 -- 172.21.15.28:0/220753835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa924013640 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.027+0000 7fa814e01640 1 -- 172.21.15.28:0/220753835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9240137e0 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.027+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610f3a9fec0 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.027+0000 7fa814e01640 1 -- 172.21.15.28:0/220753835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa924013ae0 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.027+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f3aa0460 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.027+0000 7fa814e01640 1 -- 172.21.15.28:0/220753835 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa924011040 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa814e01640 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa80403d000 0x7fa80403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa814e01640 1 -- 172.21.15.28:0/220753835 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fa92400e040 con 0x5610f3a9ec30 2024-09-19T11:23:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa80403d000 0x7fa80403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92ef2c800 1 Processor -- start 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92ef2c800 1 -- start start 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa80403d000 0x7fa80403f4c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5610f3a8ba60 tx=0x7fa91c05f890 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92ef2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3aabab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92ef2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610f3989180 con 0x5610f3aab6b0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92b33f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3aabab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92b33f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3aabab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:58992/0 (socket says 172.21.15.28:58992) 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.028+0000 7fa92b33f640 1 -- 172.21.15.28:0/235693981 learned_addr learned my addr 172.21.15.28:0/235693981 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92b33f640 1 -- 172.21.15.28:0/235693981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3aac0a0 con 0x5610f3aab6b0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92b33f640 1 --2- 172.21.15.28:0/235693981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3aabab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9240547f0 tx=0x7fa92401fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0e641e19670859 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa80e7fc640 1 -- 172.21.15.28:0/235693981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa924044030 con 0x5610f3aab6b0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa80e7fc640 1 -- 172.21.15.28:0/235693981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa924021a90 con 0x5610f3aab6b0 2024-09-19T11:23:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa80e7fc640 1 -- 172.21.15.28:0/235693981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa92401f4b0 con 0x5610f3aab6b0 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92ef2c800 1 -- 172.21.15.28:0/235693981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 msgr2=0x5610f3aabab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/235693981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3aabab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa9240547f0 tx=0x7fa92401fe70 comp rx=0 tx=0).stop 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92ef2c800 1 -- 172.21.15.28:0/235693981 shutdown_connections 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/235693981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3aabab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92ef2c800 1 -- 172.21.15.28:0/235693981 >> 172.21.15.28:0/235693981 conn(0x5610f3aa7ab0 msgr2=0x5610f3aa5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92ef2c800 1 -- 172.21.15.28:0/235693981 shutdown_connections 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.029+0000 7fa92ef2c800 1 -- 172.21.15.28:0/235693981 wait complete. 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92ef2c800 1 Processor -- start 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92ef2c800 1 -- start start 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92ef2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3ab2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92ef2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610f3aac6c0 con 0x5610f3aab6b0 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92b33f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3ab2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92b33f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3ab2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59008/0 (socket says 172.21.15.28:59008) 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 learned_addr learned my addr 172.21.15.28:0/2871488788 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610f3ab3290 con 0x5610f3aab6b0 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.030+0000 7fa92b33f640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3ab2d30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa924021a60 tx=0x7fa92404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.031+0000 7fa80cff9640 1 -- 172.21.15.28:0/2871488788 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa924021570 con 0x5610f3aab6b0 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.031+0000 7fa80cff9640 1 -- 172.21.15.28:0/2871488788 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa924056070 con 0x5610f3aab6b0 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.031+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610f3ab3520 con 0x5610f3aab6b0 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.031+0000 7fa80cff9640 1 -- 172.21.15.28:0/2871488788 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9240459c0 con 0x5610f3aab6b0 2024-09-19T11:23:22.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.031+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610f3ab3520 con 0x5610f3aab6b0 2024-09-19T11:23:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.031+0000 7fa80cff9640 1 -- 172.21.15.28:0/2871488788 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa924045b60 con 0x5610f3aab6b0 2024-09-19T11:23:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.032+0000 7fa80cff9640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa7f403cfb0 0x7fa7f403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.032+0000 7fa80cff9640 1 -- 172.21.15.28:0/2871488788 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fa924034c80 con 0x5610f3aab6b0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.032+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa7f403cfb0 0x7fa7f403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.032+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa7f403cfb0 0x7fa7f403f470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a330 tx=0x7fa91c08a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.032+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.032+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f3abe3f0 0x5610f3ac08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac1000 con 0x5610f3abe3f0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92a33d640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f3abe3f0 0x5610f3ac08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92a33d640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f3abe3f0 0x5610f3ac08b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa92000f5f0 con 0x5610f3abe3f0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 20 realm 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3abd740 con 0x5610f3abe3f0 2024-09-19T11:23:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa92000f5f0 con 0x5610f3abe3f0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.033+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac1af0 con 0x5610f3abe3f0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa92000f5f0 con 0x5610f3abe3f0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac23c0 con 0x5610f3abe3f0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa92000f5f0 con 0x5610f3abe3f0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92ef2c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3ac2de0 0x5610f3ac51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac5940 con 0x5610f3ac2de0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92b33f640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3ac2de0 0x5610f3ac51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.034+0000 7fa92b33f640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3ac2de0 0x5610f3ac51d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92ef2c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92ef2c800 20 searching for the correct realm 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92b33f640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa92404d340 0x7fa92404d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa92404d340 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa92404d340 0x7fa92404d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.035+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa92404d340 0x7fa92404d7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92ab3e640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa91c0a5040 0x7fa91c08f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92a33d640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa91c0a5040 0x7fa91c08f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92a33d640 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa91c0a5040 0x7fa91c08f6b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92000f5f0 con 0x7fa91c0a5040 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.036+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92000f5f0 con 0x5610f3abe3f0 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa92404d340 2024-09-19T11:23:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.037+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.038+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa92404d340 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.039+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.039+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.039+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.039+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.039+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.039+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.040+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.040+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.040+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.040+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa92002b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.041+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.042+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.042+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa92404d340 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.042+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.042+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.042+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92002b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.042+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa92404d340 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.043+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3ac2de0 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa91c0a5040 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x5610f3abe3f0 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7270 con 0x7fa92404d340 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.044+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac6300 con 0x5610f3abe3f0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac6ba0 con 0x5610f3abe3f0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac76f0 con 0x5610f3ac2de0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 zone default found 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 4 Realm: () 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 zonegroup default 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.045+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac7a90 con 0x7fa91c0a5040 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac98a0 con 0x5610f3ac2de0 2024-09-19T11:23:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac9d10 con 0x5610f3ac2de0 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 20 started sync module instance, tier type = 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.046+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac8d40 con 0x7fa92404d340 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.047+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.047+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94630807638320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ac89a0 con 0x7fa92404d340 2024-09-19T11:23:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.047+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acab30 con 0x5610f3abe3f0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.048+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94630807638320] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.048+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.048+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94630807642480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acaaf0 con 0x5610f3abe3f0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.048+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acb5a0 con 0x5610f3ac2de0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.049+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.049+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94630807646352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acb5a0 con 0x5610f3ac2de0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.049+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acc580 con 0x7fa92404d340 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.050+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94630807642480] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.050+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94630807646352] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.051+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.051+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94630807648704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acc090 con 0x7fa92404d340 2024-09-19T11:23:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.051+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acce50 con 0x5610f3abe3f0 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.052+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.052+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94630807652768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acd270 con 0x5610f3abe3f0 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.052+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acdac0 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.052+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94630807648704] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.053+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.053+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94630807655024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acdeb0 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.053+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ace700 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.053+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94630807652768] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.054+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.054+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94630807658160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ace780 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.054+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acefd0 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.054+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94630807655024] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94630807658160] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94630807659216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3aceba0 con 0x7fa91c0a5040 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 20 add_watcher() i=0 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 20 add_watcher() i=1 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 20 add_watcher() i=2 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 20 add_watcher() i=3 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 20 add_watcher() i=4 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 20 add_watcher() i=5 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.055+0000 7fa92ef2c800 20 add_watcher() i=6 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.056+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94630807659216] v31'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.056+0000 7fa92ef2c800 20 add_watcher() i=7 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.056+0000 7fa92ef2c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.056+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ad1d50 con 0x5610f3abe3f0 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.056+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.056+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94630807660272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acefc0 con 0x5610f3abe3f0 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.057+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94630807660272] v31'52 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.057+0000 7fa7e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.057+0000 7fa92ef2c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.057+0000 7fa92ef2c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.057+0000 7fa92ef2c800 5 note: GC not initialized 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa75bfff640 20 reqs_thread_entry: start 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 20 Filter name: none 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa759ffb640 20 reqs_thread_entry: start 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:23:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3b04ce0 con 0x7fa91c0a5040 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.058+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3b033c0 con 0x5610f3abe3f0 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) ==== 197+0+23 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 20 rados_obj.operate() r=0 bl.length=7 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-19T11:23:22.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5610f3b037b0 con 0x5610f3abe3f0 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.059+0000 7fa92ef2c800 20 get_user_info_from_index found 0 xattrs 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.060+0000 7fa7e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.060+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94630807660272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ad0b60 con 0x5610f3abe3f0 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.061+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94630807660272] v31'53 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.061+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94630807638320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ace6f0 con 0x7fa92404d340 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.062+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94630807638320] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.062+0000 7fa92ef2c800 20 remove_watcher() i=0 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.062+0000 7fa92ef2c800 2 removed watcher, disabling cache 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.062+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94630807642480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ace6f0 con 0x5610f3abe3f0 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.063+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94630807642480] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.063+0000 7fa92ef2c800 20 remove_watcher() i=1 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.063+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94630807646352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3aca170 con 0x5610f3ac2de0 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.064+0000 7fa92b33f640 1 -- 172.21.15.28:0/2871488788 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94630807646352] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa924053020 con 0x5610f3ac2de0 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.064+0000 7fa92ef2c800 20 remove_watcher() i=2 2024-09-19T11:23:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.064+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94630807648704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ace6f0 con 0x7fa92404d340 2024-09-19T11:23:22.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.065+0000 7fa92ab3e640 1 -- 172.21.15.28:0/2871488788 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94630807648704] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa91c051070 con 0x7fa92404d340 2024-09-19T11:23:22.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.065+0000 7fa92ef2c800 20 remove_watcher() i=3 2024-09-19T11:23:22.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.065+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94630807652768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3aca170 con 0x5610f3abe3f0 2024-09-19T11:23:22.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.066+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94630807652768] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x5610f3abe3f0 2024-09-19T11:23:22.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.066+0000 7fa92ef2c800 20 remove_watcher() i=4 2024-09-19T11:23:22.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.066+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94630807655024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3ace6f0 con 0x7fa91c0a5040 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.067+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94630807655024] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.067+0000 7fa92ef2c800 20 remove_watcher() i=5 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.067+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94630807658160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3acc9a0 con 0x7fa91c0a5040 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.068+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94630807658160] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.068+0000 7fa92ef2c800 20 remove_watcher() i=6 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.068+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94630807659216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5610f3aca170 con 0x7fa91c0a5040 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94630807659216] v31'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa92003b040 con 0x7fa91c0a5040 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 20 remove_watcher() i=7 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa91c0a5040 msgr2=0x7fa91c08f6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa91c0a5040 0x7fa91c08f6b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f3abe3f0 msgr2=0x5610f3ac08b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f3abe3f0 0x5610f3ac08b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa92404d340 msgr2=0x7fa92404d7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa92404d340 0x7fa92404d7c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3ac2de0 msgr2=0x5610f3ac51d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3ac2de0 0x5610f3ac51d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa7f403cfb0 msgr2=0x7fa7f403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.069+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa7f403cfb0 0x7fa7f403f470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00a330 tx=0x7fa91c08a000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 msgr2=0x5610f3ab2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3ab2d30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa924021a60 tx=0x7fa92404e000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92a33d640 1 -- 172.21.15.28:0/2871488788 reap_dead start 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 shutdown_connections 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2871488788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3aab6b0 0x5610f3ab2d30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 >> 172.21.15.28:0/2871488788 conn(0x5610f3aa7ab0 msgr2=0x5610f3aa5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 shutdown_connections 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2871488788 wait complete. 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa80403d000 msgr2=0x7fa80403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa80403d000 0x7fa80403f4c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5610f3a8ba60 tx=0x7fa91c05f890 comp rx=0 tx=0).stop 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3a9ec30 msgr2=0x5610f3a9f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3a9ec30 0x5610f3a9f050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5610f3a85140 tx=0x7fa9240167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 shutdown_connections 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa80403d000 0x7fa80403f4c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/220753835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f3a9ec30 0x5610f3a9f050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 >> 172.21.15.28:0/220753835 conn(0x5610f3a92fe0 msgr2=0x5610f3a933c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 shutdown_connections 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/220753835 wait complete. 2024-09-19T11:23:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f398d900 msgr2=0x5610f3a8b450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f398d900 0x5610f3a8b450 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3a8d8c0 msgr2=0x5610f3a8fcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.070+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3a8d8c0 0x5610f3a8fcb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa81003d000 msgr2=0x7fa81003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa81003d000 0x7fa81003f4c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa920000c00 tx=0x7fa9200013b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 msgr2=0x5610f3a83e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f3a83e00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa91c045f20 tx=0x7fa91c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 shutdown_connections 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610f3a8d8c0 0x5610f3a8fcb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa81003d000 0x7fa81003f4c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610f398d900 0x5610f3a8b450 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 --2- 172.21.15.28:0/2832634113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610f398afd0 0x5610f3a83e00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 >> 172.21.15.28:0/2832634113 conn(0x5610f3984a70 msgr2=0x5610f3a75620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 shutdown_connections 2024-09-19T11:23:22.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.071+0000 7fa92ef2c800 1 -- 172.21.15.28:0/2832634113 wait complete. 2024-09-19T11:23:22.081 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-09-19T11:23:22.082 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.082 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.082 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:22.129 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.129 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.128+0000 7fcef41af800 1 Processor -- start 2024-09-19T11:23:22.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.128+0000 7fcef41af800 1 -- start start 2024-09-19T11:23:22.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef41af800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5608d41e8850 0x5608d41f1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef41af800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5608d41f18b0 con 0x5608d41e6d30 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef41af800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5608d40737d0 con 0x5608d41e8850 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef1498640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5608d41e8850 0x5608d41f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef1c99640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5608d41e6d30 0x5608d41e7130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33008/0 (socket says 172.21.15.28:33008) 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef1498640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5608d41e8850 0x5608d41f1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59020/0 (socket says 172.21.15.28:59020) 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef1c99640 1 -- 172.21.15.28:0/2421304926 learned_addr learned my addr 172.21.15.28:0/2421304926 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef1498640 1 -- 172.21.15.28:0/2421304926 >> v1:172.21.15.22:6789/0 conn(0x5608d41e6d30 legacy=0x5608d41e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.129+0000 7fcef1498640 1 -- 172.21.15.28:0/2421304926 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d41f2a90 con 0x5608d41e8850 2024-09-19T11:23:22.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.131+0000 7fcef1498640 1 --2- 172.21.15.28:0/2421304926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e8850 0x5608d41f1370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcee800a470 tx=0x7fcee8045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723dd170280c97cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.131+0000 7fcef0c97640 1 -- 172.21.15.28:0/2421304926 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee8053570 con 0x5608d41e8850 2024-09-19T11:23:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.131+0000 7fcef0c97640 1 -- 172.21.15.28:0/2421304926 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee8058070 con 0x5608d41e8850 2024-09-19T11:23:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.131+0000 7fcef0c97640 1 -- 172.21.15.28:0/2421304926 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee8053880 con 0x5608d41e8850 2024-09-19T11:23:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.132+0000 7fcef41af800 1 -- 172.21.15.28:0/2421304926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e8850 msgr2=0x5608d41f1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.132+0000 7fcef41af800 1 --2- 172.21.15.28:0/2421304926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e8850 0x5608d41f1370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcee800a470 tx=0x7fcee8045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.132+0000 7fcef41af800 1 -- 172.21.15.28:0/2421304926 shutdown_connections 2024-09-19T11:23:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.132+0000 7fcef41af800 1 --2- 172.21.15.28:0/2421304926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e8850 0x5608d41f1370 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.132+0000 7fcef41af800 1 -- 172.21.15.28:0/2421304926 >> 172.21.15.28:0/2421304926 conn(0x5608d41e2a60 msgr2=0x5608d41e4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.132+0000 7fcef41af800 1 -- 172.21.15.28:0/2421304926 shutdown_connections 2024-09-19T11:23:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.132+0000 7fcef41af800 1 -- 172.21.15.28:0/2421304926 wait complete. 2024-09-19T11:23:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.146+0000 7fcef41af800 1 Processor -- start 2024-09-19T11:23:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.147+0000 7fcef41af800 1 -- start start 2024-09-19T11:23:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.147+0000 7fcef41af800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.147+0000 7fcef41af800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608d4080dd0 con 0x5608d41e6d30 2024-09-19T11:23:22.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.147+0000 7fcef1c99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.147+0000 7fcef1c99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42da520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59030/0 (socket says 172.21.15.28:59030) 2024-09-19T11:23:22.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.147+0000 7fcef1c99640 1 -- 172.21.15.28:0/973698788 learned_addr learned my addr 172.21.15.28:0/973698788 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.147+0000 7fcef1c99640 1 -- 172.21.15.28:0/973698788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d42dab10 con 0x5608d41e6d30 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcef1c99640 1 --2- 172.21.15.28:0/973698788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42da520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcee4001800 tx=0x7fcee4001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebb8ecb7c4b2327 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcddfe07640 1 -- 172.21.15.28:0/973698788 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee4004520 con 0x5608d41e6d30 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcddfe07640 1 -- 172.21.15.28:0/973698788 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee40046c0 con 0x5608d41e6d30 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcddfe07640 1 -- 172.21.15.28:0/973698788 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee40049e0 con 0x5608d41e6d30 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcef41af800 1 -- 172.21.15.28:0/973698788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 msgr2=0x5608d42da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcef41af800 1 --2- 172.21.15.28:0/973698788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42da520 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcee4001800 tx=0x7fcee4001cc0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcef41af800 1 -- 172.21.15.28:0/973698788 shutdown_connections 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcef41af800 1 --2- 172.21.15.28:0/973698788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42da520 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcef41af800 1 -- 172.21.15.28:0/973698788 >> 172.21.15.28:0/973698788 conn(0x5608d41e2a60 msgr2=0x5608d42d5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.148+0000 7fcef41af800 1 -- 172.21.15.28:0/973698788 shutdown_connections 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef41af800 1 -- 172.21.15.28:0/973698788 wait complete. 2024-09-19T11:23:22.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef41af800 1 Processor -- start 2024-09-19T11:23:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef41af800 1 -- start start 2024-09-19T11:23:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef41af800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42e2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef41af800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608d41f1d50 con 0x5608d41e6d30 2024-09-19T11:23:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef1c99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef1c99640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42e2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59046/0 (socket says 172.21.15.28:59046) 2024-09-19T11:23:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef1c99640 1 -- 172.21.15.28:0/1494393198 learned_addr learned my addr 172.21.15.28:0/1494393198 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.149+0000 7fcef1c99640 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d42e25d0 con 0x5608d41e6d30 2024-09-19T11:23:22.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.150+0000 7fcef1c99640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42e2070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000f80 tx=0x7fcee4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.150+0000 7fcdde604640 1 -- 172.21.15.28:0/1494393198 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee4022020 con 0x5608d41e6d30 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.150+0000 7fcdde604640 1 -- 172.21.15.28:0/1494393198 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee4014890 con 0x5608d41e6d30 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.150+0000 7fcdde604640 1 -- 172.21.15.28:0/1494393198 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcee4014bb0 con 0x5608d41e6d30 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.150+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608d42e2860 con 0x5608d41e6d30 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.150+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d42e3260 con 0x5608d41e6d30 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcdde604640 1 -- 172.21.15.28:0/1494393198 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fcee4025020 con 0x5608d41e6d30 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcdde604640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdd803d000 0x7fcdd803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcdde604640 1 -- 172.21.15.28:0/1494393198 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fcee4018030 con 0x5608d41e6d30 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcef1498640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdd803d000 0x7fcdd803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d41e8850 0x5608d42e96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d42e9c00 con 0x5608d41e8850 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcef249a640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d41e8850 0x5608d42e96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcef1498640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdd803d000 0x7fcdd803f4c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcee8057000 tx=0x7fcee8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.151+0000 7fcef249a640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d41e8850 0x5608d42e96c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.152+0000 7fcef249a640 1 -- 172.21.15.28:0/1494393198 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fceec0083e0 con 0x5608d41e8850 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.152+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d42e9fa0 con 0x5608d41e8850 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.152+0000 7fcef249a640 1 -- 172.21.15.28:0/1494393198 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fceec0083e0 con 0x5608d41e8850 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.152+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d42ebaa0 0x5608d42edf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.152+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d42ee6b0 con 0x5608d42ebaa0 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.152+0000 7fcef1c99640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d42ebaa0 0x5608d42edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.153+0000 7fcef1c99640 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d42ebaa0 0x5608d42edf20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.153+0000 7fcef1c99640 1 -- 172.21.15.28:0/1494393198 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fcee401b070 con 0x5608d42ebaa0 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.153+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d42ea7f0 con 0x5608d41e8850 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.153+0000 7fcef249a640 1 -- 172.21.15.28:0/1494393198 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fceec0083e0 con 0x5608d41e8850 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.153+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d42eabe0 con 0x5608d42ebaa0 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.153+0000 7fcef1c99640 1 -- 172.21.15.28:0/1494393198 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fcee401b070 con 0x5608d42ebaa0 2024-09-19T11:23:22.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef41af800 1 Processor -- start 2024-09-19T11:23:22.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef41af800 1 -- start start 2024-09-19T11:23:22.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef41af800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d42fd070 0x5608d42fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef41af800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608d40df4a0 con 0x5608d42fd070 2024-09-19T11:23:22.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef249a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d42fd070 0x5608d42fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef249a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d42fd070 0x5608d42fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59060/0 (socket says 172.21.15.28:59060) 2024-09-19T11:23:22.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef249a640 1 -- 172.21.15.28:0/129725598 learned_addr learned my addr 172.21.15.28:0/129725598 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.154+0000 7fcef249a640 1 -- 172.21.15.28:0/129725598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d42fdad0 con 0x5608d42fd070 2024-09-19T11:23:22.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.155+0000 7fcef249a640 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d42fd070 0x5608d42fd490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5608d42e3070 tx=0x7fceec00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.155+0000 7fcdd77fe640 1 -- 172.21.15.28:0/129725598 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceec01c020 con 0x5608d42fd070 2024-09-19T11:23:22.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.155+0000 7fcdd77fe640 1 -- 172.21.15.28:0/129725598 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceec007c90 con 0x5608d42fd070 2024-09-19T11:23:22.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.155+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608d42fe300 con 0x5608d42fd070 2024-09-19T11:23:22.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.155+0000 7fcdd77fe640 1 -- 172.21.15.28:0/129725598 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceec018440 con 0x5608d42fd070 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.155+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d42fe8a0 con 0x5608d42fd070 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcdd77fe640 1 -- 172.21.15.28:0/129725598 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fceec01f020 con 0x5608d42fd070 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcdd77fe640 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdcc03d000 0x7fcdcc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcdd77fe640 1 -- 172.21.15.28:0/129725598 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fceec055980 con 0x5608d42fd070 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcef1c99640 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdcc03d000 0x7fcdcc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcef41af800 1 Processor -- start 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcef1c99640 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdcc03d000 0x7fcdcc03f4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5608d42eea80 tx=0x7fcee4006820 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcef41af800 1 -- start start 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcef41af800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4309ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.156+0000 7fcef41af800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608d42daf40 con 0x5608d4309ad0 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcef249a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4309ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcef249a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4309ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59062/0 (socket says 172.21.15.28:59062) 2024-09-19T11:23:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcef249a640 1 -- 172.21.15.28:0/1541206370 learned_addr learned my addr 172.21.15.28:0/1541206370 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcef249a640 1 -- 172.21.15.28:0/1541206370 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d430a4c0 con 0x5608d4309ad0 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcef249a640 1 --2- 172.21.15.28:0/1541206370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4309ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fceec053dc0 tx=0x7fceec015330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52901721197dedd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcdd57fa640 1 -- 172.21.15.28:0/1541206370 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceec028c20 con 0x5608d4309ad0 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcdd57fa640 1 -- 172.21.15.28:0/1541206370 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceec0173a0 con 0x5608d4309ad0 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.157+0000 7fcdd57fa640 1 -- 172.21.15.28:0/1541206370 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceec028300 con 0x5608d4309ad0 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 -- 172.21.15.28:0/1541206370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 msgr2=0x5608d4309ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 --2- 172.21.15.28:0/1541206370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4309ed0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fceec053dc0 tx=0x7fceec015330 comp rx=0 tx=0).stop 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 -- 172.21.15.28:0/1541206370 shutdown_connections 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 --2- 172.21.15.28:0/1541206370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4309ed0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 -- 172.21.15.28:0/1541206370 >> 172.21.15.28:0/1541206370 conn(0x5608d4305ed0 msgr2=0x5608d4304110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 -- 172.21.15.28:0/1541206370 shutdown_connections 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 -- 172.21.15.28:0/1541206370 wait complete. 2024-09-19T11:23:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 Processor -- start 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 -- start start 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4311080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef41af800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608d430aae0 con 0x5608d4309ad0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef249a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4311080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef249a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4311080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59076/0 (socket says 172.21.15.28:59076) 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.158+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 learned_addr learned my addr 172.21.15.28:0/2564982567 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.159+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d43115e0 con 0x5608d4309ad0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.159+0000 7fcef249a640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4311080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fceec015a50 tx=0x7fceec053d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.159+0000 7fcdc37fe640 1 -- 172.21.15.28:0/2564982567 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceec02f070 con 0x5608d4309ad0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.159+0000 7fcdc37fe640 1 -- 172.21.15.28:0/2564982567 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceec020aa0 con 0x5608d4309ad0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.159+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608d4311870 con 0x5608d4309ad0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.159+0000 7fcdc37fe640 1 -- 172.21.15.28:0/2564982567 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceec016a10 con 0x5608d4309ad0 2024-09-19T11:23:22.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.159+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d43121f0 con 0x5608d4309ad0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcdc37fe640 1 -- 172.21.15.28:0/2564982567 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fceec032070 con 0x5608d4309ad0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcdc37fe640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdbc03d000 0x7fcdbc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcdc37fe640 1 -- 172.21.15.28:0/2564982567 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fceec03c7f0 con 0x5608d4309ad0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcef1c99640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdbc03d000 0x7fcdbc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d431c900 0x5608d431edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d431f510 con 0x5608d431c900 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.160+0000 7fcef1c99640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdbc03d000 0x7fcdbc03f4c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcee4014fd0 tx=0x7fcee4052d00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef1498640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d431c900 0x5608d431edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef1498640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d431c900 0x5608d431edc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 20 realm 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d431f940 con 0x5608d431c900 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.161+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d4320210 con 0x5608d431c900 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef41af800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d43205b0 con 0x5608d431c900 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef41af800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d4321310 0x5608d4323700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d4323c40 con 0x5608d4321310 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef249a640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d4321310 0x5608d4323700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.162+0000 7fcef249a640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d4321310 0x5608d4323700 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fceec031030 con 0x5608d4321310 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef41af800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef41af800 20 searching for the correct realm 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec031030 con 0x5608d4321310 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef249a640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fceec034060 0x7fceec004280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fceec034060 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef1c99640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fceec034060 0x7fceec004280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef1c99640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fceec034060 0x7fceec004280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef1c99640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcee4027020 0x7fcee401e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.163+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.164+0000 7fcef1498640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcee4027020 0x7fcee401e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.164+0000 7fcef1498640 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcee4027020 0x7fcee401e3f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.164+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.164+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.165+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fceec034060 2024-09-19T11:23:22.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec031030 con 0x5608d4321310 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.166+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fceec034060 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec031030 con 0x5608d4321310 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.167+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.168+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.168+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.168+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.168+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.168+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.168+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.169+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fceec034060 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.170+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.171+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.171+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fceec034060 2024-09-19T11:23:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.171+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.171+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d4321310 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.171+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.171+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fcee4027020 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x5608d431c900 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5608d43255b0 con 0x7fceec034060 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d43246b0 con 0x5608d431c900 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.172+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d4324f50 con 0x5608d431c900 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d43242f0 con 0x5608d4321310 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 zone default found 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 4 Realm: () 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 zonegroup default 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d4325f30 con 0x7fcee4027020 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d4327df0 con 0x5608d4321310 2024-09-19T11:23:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.173+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef41af800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d4328230 con 0x5608d4321310 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef41af800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef41af800 20 started sync module instance, tier type = 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef41af800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.174+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4327160 con 0x7fceec034060 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.175+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.175+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94595919801520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4328660 con 0x7fceec034060 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.175+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4328eb0 con 0x5608d431c900 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.176+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94595919801520] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.176+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.176+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94595919809024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d43292d0 con 0x5608d431c900 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.176+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4329c90 con 0x5608d4321310 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.177+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.178+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94595919812576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432a0b0 con 0x5608d4321310 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.178+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432ab90 con 0x7fceec034060 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.178+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94595919809024] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.179+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94595919812576] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.179+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.179+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94595919810080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432a0b0 con 0x7fceec034060 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.179+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432b4a0 con 0x5608d431c900 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.180+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94595919810080] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.180+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.180+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94595919818672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432a0b0 con 0x5608d431c900 2024-09-19T11:23:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.180+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432bf10 con 0x7fcee4027020 2024-09-19T11:23:22.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.181+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94595919818672] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.181+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.181+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94595919821312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432bed0 con 0x7fcee4027020 2024-09-19T11:23:22.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.181+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432c950 con 0x7fcee4027020 2024-09-19T11:23:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.182+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.182+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94595919823936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432cdc0 con 0x7fcee4027020 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.182+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432d8b0 con 0x7fcee4027020 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.182+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94595919821312] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94595919823936] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94595919813632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432cdc0 con 0x7fcee4027020 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 20 add_watcher() i=0 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 20 add_watcher() i=1 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 20 add_watcher() i=2 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 20 add_watcher() i=3 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 20 add_watcher() i=4 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 20 add_watcher() i=5 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.183+0000 7fcef41af800 20 add_watcher() i=6 2024-09-19T11:23:22.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.184+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94595919813632] v31'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.184+0000 7fcef41af800 20 add_watcher() i=7 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.184+0000 7fcef41af800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.184+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d43308a0 con 0x5608d431c900 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.185+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.185+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94595919825632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432dd20 con 0x5608d431c900 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94595919825632] v31'54 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcda37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcef41af800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcef41af800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcef41af800 5 note: GC not initialized 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcda27fc640 20 reqs_thread_entry: start 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcef41af800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcef41af800 20 Filter name: none 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.186+0000 7fcd13fff640 20 reqs_thread_entry: start 2024-09-19T11:23:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.187+0000 7fcef41af800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.187+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608d4363100 con 0x5608d431c900 2024-09-19T11:23:22.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.187+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.187+0000 7fcef41af800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:22.196 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:22.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.188+0000 7fcda37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.188+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94595919825632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4361ac0 con 0x5608d431c900 2024-09-19T11:23:22.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.189+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94595919825632] v31'55 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.189+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94595919801520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432cab0 con 0x7fceec034060 2024-09-19T11:23:22.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.190+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94595919801520] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.190+0000 7fcef41af800 20 remove_watcher() i=0 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.190+0000 7fcef41af800 2 removed watcher, disabling cache 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.190+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94595919809024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4326cb0 con 0x5608d431c900 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.191+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94595919809024] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.191+0000 7fcef41af800 20 remove_watcher() i=1 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.191+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94595919812576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4328a00 con 0x5608d4321310 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.192+0000 7fcef249a640 1 -- 172.21.15.28:0/2564982567 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94595919812576] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fceec062040 con 0x5608d4321310 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.192+0000 7fcef41af800 20 remove_watcher() i=2 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.192+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94595919810080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d4328a00 con 0x7fceec034060 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.193+0000 7fcef1c99640 1 -- 172.21.15.28:0/2564982567 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94595919810080] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee401b070 con 0x7fceec034060 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.193+0000 7fcef41af800 20 remove_watcher() i=3 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.193+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94595919818672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d43297e0 con 0x5608d431c900 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.194+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94595919818672] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x5608d431c900 2024-09-19T11:23:22.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.194+0000 7fcef41af800 20 remove_watcher() i=4 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.194+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94595919821312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d43297e0 con 0x7fcee4027020 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.195+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94595919821312] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.195+0000 7fcef41af800 20 remove_watcher() i=5 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.195+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94595919823936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d432ba00 con 0x7fcee4027020 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.196+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94595919823936] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.196+0000 7fcef41af800 20 remove_watcher() i=6 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.196+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94595919813632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608d43297e0 con 0x7fcee4027020 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94595919813632] v31'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcee8051070 con 0x7fcee4027020 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 20 remove_watcher() i=7 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcee4027020 msgr2=0x7fcee401e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcee4027020 0x7fcee401e3f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d431c900 msgr2=0x5608d431edc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d431c900 0x5608d431edc0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fceec034060 msgr2=0x7fceec004280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fceec034060 0x7fceec004280 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d4321310 msgr2=0x5608d4323700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d4321310 0x5608d4323700 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdbc03d000 msgr2=0x7fcdbc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdbc03d000 0x7fcdbc03f4c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcee4014fd0 tx=0x7fcee4052d00 comp rx=0 tx=0).stop 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 msgr2=0x5608d4311080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 --2- 172.21.15.28:0/2564982567 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d4309ad0 0x5608d4311080 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fceec015a50 tx=0x7fceec053d20 comp rx=0 tx=0).stop 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 reap_dead start 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef1498640 1 -- 172.21.15.28:0/2564982567 reap_dead start 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 shutdown_connections 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 >> 172.21.15.28:0/2564982567 conn(0x5608d4305ed0 msgr2=0x5608d43040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 shutdown_connections 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.197+0000 7fcef41af800 1 -- 172.21.15.28:0/2564982567 wait complete. 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdcc03d000 msgr2=0x7fcdcc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdcc03d000 0x7fcdcc03f4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x5608d42eea80 tx=0x7fcee4006820 comp rx=0 tx=0).stop 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d42fd070 msgr2=0x5608d42fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d42fd070 0x5608d42fd490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x5608d42e3070 tx=0x7fceec00bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 shutdown_connections 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdcc03d000 0x7fcdcc03f4c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/129725598 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d42fd070 0x5608d42fd490 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 >> 172.21.15.28:0/129725598 conn(0x5608d42f1420 msgr2=0x5608d42f1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 shutdown_connections 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/129725598 wait complete. 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d41e8850 msgr2=0x5608d42e96c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d41e8850 0x5608d42e96c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d42ebaa0 msgr2=0x5608d42edf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d42ebaa0 0x5608d42edf20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdd803d000 msgr2=0x7fcdd803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdd803d000 0x7fcdd803f4c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcee8057000 tx=0x7fcee8052000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 msgr2=0x5608d42e2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42e2070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000f80 tx=0x7fcee4001250 comp rx=0 tx=0).stop 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 shutdown_connections 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5608d42ebaa0 0x5608d42edf20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcdd803d000 0x7fcdd803f4c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5608d41e8850 0x5608d42e96c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 --2- 172.21.15.28:0/1494393198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608d41e6d30 0x5608d42e2070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 >> 172.21.15.28:0/1494393198 conn(0x5608d41e2a60 msgr2=0x5608d42dba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 shutdown_connections 2024-09-19T11:23:22.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.198+0000 7fcef41af800 1 -- 172.21.15.28:0/1494393198 wait complete. 2024-09-19T11:23:22.209 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:22.209 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-09-19T11:23:22.209 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-09-19T11:23:22.209 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-09-19T11:23:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.254+0000 7f9dd4c87800 1 Processor -- start 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd4c87800 1 -- start start 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd4c87800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5581ce1c4e00 0x5581ce1ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd4c87800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5581ce1c52b0 con 0x5581ce1c1ce0 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd4c87800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5581ce04e7d0 con 0x5581ce1c4e00 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd1f77640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5581ce1c4e00 0x5581ce1ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd2778640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5581ce1c1ce0 0x5581ce1c46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33024/0 (socket says 172.21.15.28:33024) 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd1f77640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5581ce1c4e00 0x5581ce1ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59086/0 (socket says 172.21.15.28:59086) 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd2778640 1 -- 172.21.15.28:0/2626601523 learned_addr learned my addr 172.21.15.28:0/2626601523 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd1f77640 1 -- 172.21.15.28:0/2626601523 >> v1:172.21.15.22:6789/0 conn(0x5581ce1c1ce0 legacy=0x5581ce1c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd1f77640 1 -- 172.21.15.28:0/2626601523 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ce1c5480 con 0x5581ce1c4e00 2024-09-19T11:23:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.255+0000 7f9dd1776640 1 -- 172.21.15.28:0/2626601523 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3981776904 0 0) 0x5581ce1c52b0 con 0x5581ce1c1ce0 2024-09-19T11:23:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.257+0000 7f9dd1f77640 1 --2- 172.21.15.28:0/2626601523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c4e00 0x5581ce1ca750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800a470 tx=0x7f9dc8045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b61afd7875d766 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.257+0000 7f9dd1776640 1 -- 172.21.15.28:0/2626601523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc8053570 con 0x5581ce1c4e00 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.257+0000 7f9dd1776640 1 -- 172.21.15.28:0/2626601523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc8058070 con 0x5581ce1c4e00 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.257+0000 7f9dd1776640 1 -- 172.21.15.28:0/2626601523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc8053880 con 0x5581ce1c4e00 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.258+0000 7f9dd4c87800 1 -- 172.21.15.28:0/2626601523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c4e00 msgr2=0x5581ce1ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.258+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/2626601523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c4e00 0x5581ce1ca750 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9dc800a470 tx=0x7f9dc8045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.258+0000 7f9dd4c87800 1 -- 172.21.15.28:0/2626601523 shutdown_connections 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.258+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/2626601523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c4e00 0x5581ce1ca750 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.258+0000 7f9dd4c87800 1 -- 172.21.15.28:0/2626601523 >> 172.21.15.28:0/2626601523 conn(0x5581ce1bd1c0 msgr2=0x5581ce1bf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.258+0000 7f9dd4c87800 1 -- 172.21.15.28:0/2626601523 shutdown_connections 2024-09-19T11:23:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.258+0000 7f9dd4c87800 1 -- 172.21.15.28:0/2626601523 wait complete. 2024-09-19T11:23:22.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.273+0000 7f9dd4c87800 1 Processor -- start 2024-09-19T11:23:22.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.273+0000 7f9dd4c87800 1 -- start start 2024-09-19T11:23:22.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9dd4c87800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce1c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9dd4c87800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581ce05bdd0 con 0x5581ce1c1ce0 2024-09-19T11:23:22.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9dd2778640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce1c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9dd2778640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce1c3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59098/0 (socket says 172.21.15.28:59098) 2024-09-19T11:23:22.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9dd2778640 1 -- 172.21.15.28:0/3139482778 learned_addr learned my addr 172.21.15.28:0/3139482778 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9dd2778640 1 -- 172.21.15.28:0/3139482778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ce1c4470 con 0x5581ce1c1ce0 2024-09-19T11:23:22.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9dd2778640 1 --2- 172.21.15.28:0/3139482778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce1c3e80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4001ea0 tx=0x7f9dc400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4d9371dee2b5f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.274+0000 7f9cc0608640 1 -- 172.21.15.28:0/3139482778 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc4008070 con 0x5581ce1c1ce0 2024-09-19T11:23:22.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9cc0608640 1 -- 172.21.15.28:0/3139482778 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc400d500 con 0x5581ce1c1ce0 2024-09-19T11:23:22.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9cc0608640 1 -- 172.21.15.28:0/3139482778 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc400d880 con 0x5581ce1c1ce0 2024-09-19T11:23:22.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3139482778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 msgr2=0x5581ce1c3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3139482778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce1c3e80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4001ea0 tx=0x7f9dc400b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3139482778 shutdown_connections 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3139482778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce1c3e80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3139482778 >> 172.21.15.28:0/3139482778 conn(0x5581ce1bd1c0 msgr2=0x5581ce2aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3139482778 shutdown_connections 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.275+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3139482778 wait complete. 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd4c87800 1 Processor -- start 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd4c87800 1 -- start start 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd4c87800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce2bd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd4c87800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581ce1caf90 con 0x5581ce1c1ce0 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd2778640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce2bd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd2778640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce2bd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59114/0 (socket says 172.21.15.28:59114) 2024-09-19T11:23:22.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd2778640 1 -- 172.21.15.28:0/3189522888 learned_addr learned my addr 172.21.15.28:0/3189522888 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.276+0000 7f9dd2778640 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ce2bd5d0 con 0x5581ce1c1ce0 2024-09-19T11:23:22.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.277+0000 7f9dd2778640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce2bd070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40012f0 tx=0x7f9dc4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.277+0000 7f9cbee05640 1 -- 172.21.15.28:0/3189522888 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc400f9d0 con 0x5581ce1c1ce0 2024-09-19T11:23:22.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.277+0000 7f9cbee05640 1 -- 172.21.15.28:0/3189522888 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc400fb70 con 0x5581ce1c1ce0 2024-09-19T11:23:22.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.277+0000 7f9cbee05640 1 -- 172.21.15.28:0/3189522888 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc400fe50 con 0x5581ce1c1ce0 2024-09-19T11:23:22.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.277+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5581ce2bd860 con 0x5581ce1c1ce0 2024-09-19T11:23:22.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.277+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ce2be260 con 0x5581ce1c1ce0 2024-09-19T11:23:22.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.278+0000 7f9cbee05640 1 -- 172.21.15.28:0/3189522888 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9dc4017040 con 0x5581ce1c1ce0 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.278+0000 7f9cbee05640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cb803d1a0 0x7f9cb803f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.278+0000 7f9cbee05640 1 -- 172.21.15.28:0/3189522888 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f9dc4015020 con 0x5581ce1c1ce0 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.278+0000 7f9dd1f77640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cb803d1a0 0x7f9cb803f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.278+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce1c4e00 0x5581ce2c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.278+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2c4c00 con 0x5581ce1c4e00 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.278+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce1c4e00 0x5581ce2c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd1f77640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cb803d1a0 0x7f9cb803f660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8057000 tx=0x7f9dc8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce1c4e00 0x5581ce2c46c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd2f79640 1 -- 172.21.15.28:0/3189522888 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce1c4e00 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2c4fa0 con 0x5581ce1c4e00 2024-09-19T11:23:22.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd2f79640 1 -- 172.21.15.28:0/3189522888 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce1c4e00 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2c6aa0 0x5581ce2c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2c96b0 con 0x5581ce2c6aa0 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.279+0000 7f9dd2778640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2c6aa0 0x5581ce2c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.280+0000 7f9dd2778640 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2c6aa0 0x5581ce2c8f20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.280+0000 7f9dd2778640 1 -- 172.21.15.28:0/3189522888 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9dc402d070 con 0x5581ce2c6aa0 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.280+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2c57f0 con 0x5581ce1c4e00 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.280+0000 7f9dd2f79640 1 -- 172.21.15.28:0/3189522888 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce1c4e00 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.280+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2c5c40 con 0x5581ce2c6aa0 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd2778640 1 -- 172.21.15.28:0/3189522888 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9dc402d070 con 0x5581ce2c6aa0 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd4c87800 1 Processor -- start 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd4c87800 1 -- start start 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd4c87800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2d8020 0x5581ce2d8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd4c87800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581ce0ba4a0 con 0x5581ce2d8020 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd2f79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2d8020 0x5581ce2d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd2f79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2d8020 0x5581ce2d8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59116/0 (socket says 172.21.15.28:59116) 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.281+0000 7f9dd2f79640 1 -- 172.21.15.28:0/3956102991 learned_addr learned my addr 172.21.15.28:0/3956102991 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.282+0000 7f9dd2f79640 1 -- 172.21.15.28:0/3956102991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ce2d8a80 con 0x5581ce2d8020 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.282+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2d8020 0x5581ce2d8440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5581ce2be070 tx=0x7f9dcc0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.282+0000 7f9cb7fff640 1 -- 172.21.15.28:0/3956102991 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc013640 con 0x5581ce2d8020 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.282+0000 7f9cb7fff640 1 -- 172.21.15.28:0/3956102991 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc0137e0 con 0x5581ce2d8020 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.282+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5581ce2d92b0 con 0x5581ce2d8020 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.282+0000 7f9cb7fff640 1 -- 172.21.15.28:0/3956102991 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc013ae0 con 0x5581ce2d8020 2024-09-19T11:23:22.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.282+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ce2d9850 con 0x5581ce2d8020 2024-09-19T11:23:22.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.283+0000 7f9cb7fff640 1 -- 172.21.15.28:0/3956102991 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9dcc011040 con 0x5581ce2d8020 2024-09-19T11:23:22.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.283+0000 7f9cb7fff640 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cac03d000 0x7f9cac03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.283+0000 7f9cb7fff640 1 -- 172.21.15.28:0/3956102991 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f9dcc00e040 con 0x5581ce2d8020 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.283+0000 7f9dd2778640 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cac03d000 0x7f9cac03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd4c87800 1 Processor -- start 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd2778640 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cac03d000 0x7f9cac03f4c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40531e0 tx=0x7f9dc40011b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd4c87800 1 -- start start 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd4c87800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd4c87800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581ce1c28d0 con 0x5581ce2e4a50 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd2f79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd2f79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2e4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59122/0 (socket says 172.21.15.28:59122) 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1803094041 learned_addr learned my addr 172.21.15.28:0/1803094041 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1803094041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ce2e5440 con 0x5581ce2e4a50 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.284+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/1803094041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2e4e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc054810 tx=0x7f9dcc0143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dd0b344ea4347a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9cb5ffb640 1 -- 172.21.15.28:0/1803094041 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc051070 con 0x5581ce2e4a50 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9cb5ffb640 1 -- 172.21.15.28:0/1803094041 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc053440 con 0x5581ce2e4a50 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9cb5ffb640 1 -- 172.21.15.28:0/1803094041 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc021a40 con 0x5581ce2e4a50 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1803094041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 msgr2=0x5581ce2e4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1803094041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2e4e50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc054810 tx=0x7f9dcc0143c0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1803094041 shutdown_connections 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1803094041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2e4e50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1803094041 >> 172.21.15.28:0/1803094041 conn(0x5581ce2e0e70 msgr2=0x5581ce2df0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1803094041 shutdown_connections 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1803094041 wait complete. 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 Processor -- start 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.285+0000 7f9dd4c87800 1 -- start start 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd4c87800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2ec050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd4c87800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5581ce2e5a60 con 0x5581ce2e4a50 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd2f79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2ec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd2f79640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2ec050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59128/0 (socket says 172.21.15.28:59128) 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 learned_addr learned my addr 172.21.15.28:0/1140170987 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ce2ec5b0 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2ec050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc054740 tx=0x7f9dcc054770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9ca3fff640 1 -- 172.21.15.28:0/1140170987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc013c80 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9ca3fff640 1 -- 172.21.15.28:0/1140170987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc013e40 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9ca3fff640 1 -- 172.21.15.28:0/1140170987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dcc045130 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5581ce2ec840 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.286+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ce2ed3d0 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.287+0000 7f9ca3fff640 1 -- 172.21.15.28:0/1140170987 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9dcc0452f0 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.287+0000 7f9ca3fff640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c9c03d000 0x7f9c9c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.287+0000 7f9ca3fff640 1 -- 172.21.15.28:0/1140170987 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f9dcc034ca0 con 0x5581ce2e4a50 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.287+0000 7f9dd2778640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c9c03d000 0x7f9c9c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.288+0000 7f9dd2778640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c9c03d000 0x7f9c9c03f4c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400efd0 tx=0x7f9dc4041000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.288+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.288+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce2f7970 0x5581ce2f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.288+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2fa580 con 0x5581ce2f7970 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.288+0000 7f9dd1f77640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce2f7970 0x5581ce2f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.288+0000 7f9dd1f77640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce2f7970 0x5581ce2f9e30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 realm 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2fa9b0 con 0x5581ce2f7970 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2fb280 con 0x5581ce2f7970 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2fb620 con 0x5581ce2f7970 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.289+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2fc380 0x5581ce2fe770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2fed80 con 0x5581ce2fc380 2024-09-19T11:23:22.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2fc380 0x5581ce2fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2fc380 0x5581ce2fe770 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd4c87800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd4c87800 20 searching for the correct realm 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd2f79640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9dcc056050 0x7f9dcc04d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.290+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dcc056050 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.291+0000 7f9dd2778640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9dcc056050 0x7f9dcc04d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.291+0000 7f9dd2778640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9dcc056050 0x7f9dcc04d790 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.291+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.291+0000 7f9dd2778640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9dc4053530 0x7f9dc4053970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.291+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.291+0000 7f9dd1f77640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9dc4053530 0x7f9dc4053970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.292+0000 7f9dd1f77640 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9dc4053530 0x7f9dc4053970 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.292+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.292+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.292+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.292+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.292+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.292+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dcc056050 2024-09-19T11:23:22.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.293+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dcc056050 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.294+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.295+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.296+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.296+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.296+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.296+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.296+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dcc056050 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.297+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.298+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.298+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.298+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.298+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.298+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.298+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dcc056050 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2fc380 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dc4053530 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x5581ce2f7970 2024-09-19T11:23:22.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.299+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3006b0 con 0x7f9dcc056050 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2ff740 con 0x5581ce2f7970 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce2fff90 con 0x5581ce2f7970 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.300+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce300b30 con 0x5581ce2fc380 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 zone default found 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 4 Realm: () 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 zonegroup default 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce300ed0 con 0x7f9dc4053530 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce303110 con 0x5581ce2fc380 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.301+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3034b0 con 0x5581ce2fc380 2024-09-19T11:23:22.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.302+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.302+0000 7f9dd4c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.302+0000 7f9dd4c87800 20 started sync module instance, tier type = 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.302+0000 7f9dd4c87800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.302+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce302550 con 0x7f9dcc056050 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.303+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.303+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94015998404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce302d60 con 0x7f9dcc056050 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.303+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3041f0 con 0x5581ce2f7970 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.304+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94015998404928] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.304+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.304+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94015998408784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3045e0 con 0x5581ce2f7970 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.304+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce305010 con 0x5581ce2fc380 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.305+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94015998408784] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.305+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.305+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94015998409840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce304fd0 con 0x5581ce2fc380 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.305+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce305a50 con 0x7f9dcc056050 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.306+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94015998409840] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.306+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.306+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94015998413600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce304f40 con 0x7f9dcc056050 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.306+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce306940 con 0x5581ce2f7970 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.307+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.307+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94015998420624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3071e0 con 0x5581ce2f7970 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.307+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94015998413600] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.307+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce305eb0 con 0x7f9dc4053530 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.308+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94015998420624] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.308+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.308+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94015998422064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce307770 con 0x7f9dc4053530 2024-09-19T11:23:22.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.308+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce308130 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.309+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94015998422064] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.310+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.310+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94015998423120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce308130 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.310+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce308b50 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94015998423120] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94015998428736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce308d30 con 0x7f9dc4053530 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 20 add_watcher() i=0 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 20 add_watcher() i=1 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 20 add_watcher() i=2 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 20 add_watcher() i=3 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 20 add_watcher() i=4 2024-09-19T11:23:22.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 20 add_watcher() i=5 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.311+0000 7f9dd4c87800 20 add_watcher() i=6 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.312+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94015998428736] v31'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.312+0000 7f9dd4c87800 20 add_watcher() i=7 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.312+0000 7f9dd4c87800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.312+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce30c100 con 0x5581ce2f7970 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.312+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.312+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94015998430240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3094f0 con 0x5581ce2f7970 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.313+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94015998430240] v31'56 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.313+0000 7f9c88fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.313+0000 7f9dd4c87800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.313+0000 7f9dd4c87800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.313+0000 7f9dd4c87800 5 note: GC not initialized 2024-09-19T11:23:22.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.313+0000 7f9c037fe640 20 reqs_thread_entry: start 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 20 Filter name: none 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9c017fa640 20 reqs_thread_entry: start 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33edc0 con 0x5581ce2f7970 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.314+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce3416f0 con 0x7f9dcc056050 2024-09-19T11:23:22.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.315+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.315+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce345260 con 0x5581ce2f7970 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v31'2 uv2 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd4c87800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd4c87800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd4c87800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94015998676336 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5581ce341f40 con 0x5581ce2f7970 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94015998420624 notify 133143986177 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f9dc8081040 con 0x5581ce2f7970 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9cb5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986177 cookie 94015998420624 notifier 4313 bl.length()=567 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9cb5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9cb5ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(65 notify.4 [notify cookie 94015998676336 out=8b] v0'0 uv33 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9cb5ffb640 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f9c98003ea0 con 0x5581ce2f7970 2024-09-19T11:23:22.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.316+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94015998676336 notify 133143986177 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5581ce1c5480 con 0x5581ce2f7970 2024-09-19T11:23:22.328 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.317+0000 7f9c88fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.317+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94015998430240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x5581ce2f7970 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.318+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94015998430240] v31'57 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.318+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94015998404928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x7f9dcc056050 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.319+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94015998404928] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.319+0000 7f9dd4c87800 20 remove_watcher() i=0 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.319+0000 7f9dd4c87800 2 removed watcher, disabling cache 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.319+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94015998408784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x5581ce2f7970 2024-09-19T11:23:22.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.321+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94015998408784] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.321+0000 7f9dd4c87800 20 remove_watcher() i=1 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.321+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94015998409840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x5581ce2fc380 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.322+0000 7f9dd2f79640 1 -- 172.21.15.28:0/1140170987 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94015998409840] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dcc00a4d0 con 0x5581ce2fc380 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.322+0000 7f9dd4c87800 20 remove_watcher() i=2 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.322+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94015998413600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x7f9dcc056050 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.323+0000 7f9dd2778640 1 -- 172.21.15.28:0/1140170987 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94015998413600] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc4039070 con 0x7f9dcc056050 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.323+0000 7f9dd4c87800 20 remove_watcher() i=3 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.323+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94015998420624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x5581ce2f7970 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.324+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94015998420624] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x5581ce2f7970 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.324+0000 7f9dd4c87800 20 remove_watcher() i=4 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.324+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94015998422064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x7f9dc4053530 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.325+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94015998422064] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.325+0000 7f9dd4c87800 20 remove_watcher() i=5 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.325+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94015998423120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x7f9dc4053530 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.326+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94015998423120] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.326+0000 7f9dd4c87800 20 remove_watcher() i=6 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.326+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94015998428736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5581ce33cc20 con 0x7f9dc4053530 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94015998428736] v31'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9dc8051070 con 0x7f9dc4053530 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 20 remove_watcher() i=7 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9dc4053530 msgr2=0x7f9dc4053970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9dc4053530 0x7f9dc4053970 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce2f7970 msgr2=0x5581ce2f9e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce2f7970 0x5581ce2f9e30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9dcc056050 msgr2=0x7f9dcc04d790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9dcc056050 0x7f9dcc04d790 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2fc380 msgr2=0x5581ce2fe770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2fc380 0x5581ce2fe770 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c9c03d000 msgr2=0x7f9c9c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c9c03d000 0x7f9c9c03f4c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400efd0 tx=0x7f9dc4041000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 msgr2=0x5581ce2ec050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd1f77640 1 -- 172.21.15.28:0/1140170987 reap_dead start 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2ec050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc054740 tx=0x7f9dcc054770 comp rx=0 tx=0).stop 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 shutdown_connections 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/1140170987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2e4a50 0x5581ce2ec050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 >> 172.21.15.28:0/1140170987 conn(0x5581ce2e0e70 msgr2=0x5581ce2df080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 shutdown_connections 2024-09-19T11:23:22.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.327+0000 7f9dd4c87800 1 -- 172.21.15.28:0/1140170987 wait complete. 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cac03d000 msgr2=0x7f9cac03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cac03d000 0x7f9cac03f4c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40531e0 tx=0x7f9dc40011b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2d8020 msgr2=0x5581ce2d8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2d8020 0x5581ce2d8440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5581ce2be070 tx=0x7f9dcc0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 shutdown_connections 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cac03d000 0x7f9cac03f4c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3956102991 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce2d8020 0x5581ce2d8440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 >> 172.21.15.28:0/3956102991 conn(0x5581ce2cc3d0 msgr2=0x5581ce2cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 shutdown_connections 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3956102991 wait complete. 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce1c4e00 msgr2=0x5581ce2c46c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce1c4e00 0x5581ce2c46c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2c6aa0 msgr2=0x5581ce2c8f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2c6aa0 0x5581ce2c8f20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cb803d1a0 msgr2=0x7f9cb803f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cb803d1a0 0x7f9cb803f660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8057000 tx=0x7f9dc8052000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 msgr2=0x5581ce2bd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce2bd070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40012f0 tx=0x7f9dc4000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 shutdown_connections 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5581ce2c6aa0 0x5581ce2c8f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9cb803d1a0 0x7f9cb803f660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5581ce1c4e00 0x5581ce2c46c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 --2- 172.21.15.28:0/3189522888 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5581ce1c1ce0 0x5581ce2bd070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 >> 172.21.15.28:0/3189522888 conn(0x5581ce1bd1c0 msgr2=0x5581ce2ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 shutdown_connections 2024-09-19T11:23:22.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.328+0000 7f9dd4c87800 1 -- 172.21.15.28:0/3189522888 wait complete. 2024-09-19T11:23:22.339 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:22.339 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.340 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.340 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f3177346800 1 Processor -- start 2024-09-19T11:23:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f3177346800 1 -- start start 2024-09-19T11:23:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f3177346800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560869496280 0x56086952a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f3177346800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56086952a910 con 0x560869497610 2024-09-19T11:23:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f3177346800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5608693ac7d0 con 0x560869496280 2024-09-19T11:23:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f317462f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560869496280 0x56086952a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f317462f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560869496280 0x56086952a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59130/0 (socket says 172.21.15.28:59130) 2024-09-19T11:23:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f317462f640 1 -- 172.21.15.28:0/3267445401 learned_addr learned my addr 172.21.15.28:0/3267445401 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f317462f640 1 -- 172.21.15.28:0/3267445401 >> v1:172.21.15.22:6789/0 conn(0x560869497610 legacy=0x560869495a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.385+0000 7f317462f640 1 -- 172.21.15.28:0/3267445401 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086952baf0 con 0x560869496280 2024-09-19T11:23:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.387+0000 7f317462f640 1 --2- 172.21.15.28:0/3267445401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086952a3d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f315c00a520 tx=0x7f315c045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a4c676bd3150d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.387+0000 7f3173e2e640 1 -- 172.21.15.28:0/3267445401 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f315c055070 con 0x560869496280 2024-09-19T11:23:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.387+0000 7f3173e2e640 1 -- 172.21.15.28:0/3267445401 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315c0515b0 con 0x560869496280 2024-09-19T11:23:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3173e2e640 1 -- 172.21.15.28:0/3267445401 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f315c0518d0 con 0x560869496280 2024-09-19T11:23:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3177346800 1 -- 172.21.15.28:0/3267445401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 msgr2=0x56086952a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3177346800 1 --2- 172.21.15.28:0/3267445401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086952a3d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f315c00a520 tx=0x7f315c045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3177346800 1 -- 172.21.15.28:0/3267445401 shutdown_connections 2024-09-19T11:23:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3177346800 1 --2- 172.21.15.28:0/3267445401 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086952a3d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3177346800 1 -- 172.21.15.28:0/3267445401 >> 172.21.15.28:0/3267445401 conn(0x56086951cc60 msgr2=0x56086951f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3177346800 1 -- 172.21.15.28:0/3267445401 shutdown_connections 2024-09-19T11:23:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.388+0000 7f3177346800 1 -- 172.21.15.28:0/3267445401 wait complete. 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.403+0000 7f3177346800 1 Processor -- start 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3177346800 1 -- start start 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3177346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x5608696134d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3177346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608693b9dd0 con 0x560869496280 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3174e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x5608696134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3174e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x5608696134d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59136/0 (socket says 172.21.15.28:59136) 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3174e30640 1 -- 172.21.15.28:0/3003921672 learned_addr learned my addr 172.21.15.28:0/3003921672 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3174e30640 1 -- 172.21.15.28:0/3003921672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560869613ac0 con 0x560869496280 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.404+0000 7f3174e30640 1 --2- 172.21.15.28:0/3003921672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x5608696134d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f316c001680 tx=0x7f316c001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231e0bc006b34cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3062e0d640 1 -- 172.21.15.28:0/3003921672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c016070 con 0x560869496280 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3062e0d640 1 -- 172.21.15.28:0/3003921672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c00f640 con 0x560869496280 2024-09-19T11:23:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3062e0d640 1 -- 172.21.15.28:0/3003921672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c00f960 con 0x560869496280 2024-09-19T11:23:22.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3177346800 1 -- 172.21.15.28:0/3003921672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 msgr2=0x5608696134d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3177346800 1 --2- 172.21.15.28:0/3003921672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x5608696134d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f316c001680 tx=0x7f316c001b40 comp rx=0 tx=0).stop 2024-09-19T11:23:22.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3177346800 1 -- 172.21.15.28:0/3003921672 shutdown_connections 2024-09-19T11:23:22.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3177346800 1 --2- 172.21.15.28:0/3003921672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x5608696134d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3177346800 1 -- 172.21.15.28:0/3003921672 >> 172.21.15.28:0/3003921672 conn(0x56086951cc60 msgr2=0x56086960ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3177346800 1 -- 172.21.15.28:0/3003921672 shutdown_connections 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.405+0000 7f3177346800 1 -- 172.21.15.28:0/3003921672 wait complete. 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3177346800 1 Processor -- start 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3177346800 1 -- start start 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3177346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086961b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3177346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56086952ade0 con 0x560869496280 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3174e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086961b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3174e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086961b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59138/0 (socket says 172.21.15.28:59138) 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3174e30640 1 -- 172.21.15.28:0/2811039512 learned_addr learned my addr 172.21.15.28:0/2811039512 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.406+0000 7f3174e30640 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086961b640 con 0x560869496280 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.407+0000 7f3174e30640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086961b0e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x560869614460 tx=0x7f316c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.407+0000 7f306160a640 1 -- 172.21.15.28:0/2811039512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c021020 con 0x560869496280 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.407+0000 7f306160a640 1 -- 172.21.15.28:0/2811039512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c0117c0 con 0x560869496280 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.407+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56086961b8d0 con 0x560869496280 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.407+0000 7f306160a640 1 -- 172.21.15.28:0/2811039512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316c016040 con 0x560869496280 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.407+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086961c2d0 con 0x560869496280 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f306160a640 1 -- 172.21.15.28:0/2811039512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f316c024020 con 0x560869496280 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f306160a640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f305c03d000 0x7f305c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f306160a640 1 -- 172.21.15.28:0/2811039512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f316c053ce0 con 0x560869496280 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f317462f640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f305c03d000 0x7f305c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869497610 0x560869622730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869622c70 con 0x560869497610 2024-09-19T11:23:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f3175631640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869497610 0x560869622730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.408+0000 7f317462f640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f305c03d000 0x7f305c03f4c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f315c057000 tx=0x7f315c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.409+0000 7f3175631640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869497610 0x560869622730 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.409+0000 7f3175631640 1 -- 172.21.15.28:0/2811039512 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f31640083e0 con 0x560869497610 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.409+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869623010 con 0x560869497610 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.409+0000 7f3175631640 1 -- 172.21.15.28:0/2811039512 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f31640083e0 con 0x560869497610 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.409+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560869624ad0 0x560869626f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.409+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869627700 con 0x560869624ad0 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.409+0000 7f3174e30640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560869624ad0 0x560869626f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.410+0000 7f3174e30640 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560869624ad0 0x560869626f70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.410+0000 7f3174e30640 1 -- 172.21.15.28:0/2811039512 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f316c00f070 con 0x560869624ad0 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.410+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869623860 con 0x560869497610 2024-09-19T11:23:22.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.410+0000 7f3175631640 1 -- 172.21.15.28:0/2811039512 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f31640083e0 con 0x560869497610 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.410+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869623c50 con 0x560869624ad0 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.410+0000 7f3174e30640 1 -- 172.21.15.28:0/2811039512 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f316c00f070 con 0x560869624ad0 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3177346800 1 Processor -- start 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3177346800 1 -- start start 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3177346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869636070 0x560869636490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3177346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608694184a0 con 0x560869636070 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3175631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869636070 0x560869636490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3175631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869636070 0x560869636490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59144/0 (socket says 172.21.15.28:59144) 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3175631640 1 -- 172.21.15.28:0/1409862379 learned_addr learned my addr 172.21.15.28:0/1409862379 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.411+0000 7f3175631640 1 -- 172.21.15.28:0/1409862379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560869636ad0 con 0x560869636070 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.412+0000 7f3175631640 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869636070 0x560869636490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56086961bf70 tx=0x7f316400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.412+0000 7f305a7fc640 1 -- 172.21.15.28:0/1409862379 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316401c020 con 0x560869636070 2024-09-19T11:23:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.412+0000 7f305a7fc640 1 -- 172.21.15.28:0/1409862379 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3164007c90 con 0x560869636070 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.412+0000 7f305a7fc640 1 -- 172.21.15.28:0/1409862379 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3164018440 con 0x560869636070 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.412+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560869637300 con 0x560869636070 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.412+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608696378a0 con 0x560869636070 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.413+0000 7f305a7fc640 1 -- 172.21.15.28:0/1409862379 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f316401f020 con 0x560869636070 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.413+0000 7f305a7fc640 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304c03d000 0x7f304c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.413+0000 7f305a7fc640 1 -- 172.21.15.28:0/1409862379 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f31640559c0 con 0x560869636070 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.413+0000 7f3174e30640 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304c03d000 0x7f304c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.413+0000 7f3177346800 1 Processor -- start 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.413+0000 7f3174e30640 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304c03d000 0x7f304c03f4c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56086961d230 tx=0x7f316c02d590 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.413+0000 7f3177346800 1 -- start start 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f3177346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x560869642f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f3177346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560869613ef0 con 0x560869642b30 2024-09-19T11:23:22.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f3175631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x560869642f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f3175631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x560869642f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59148/0 (socket says 172.21.15.28:59148) 2024-09-19T11:23:22.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f3175631640 1 -- 172.21.15.28:0/3963640194 learned_addr learned my addr 172.21.15.28:0/3963640194 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f3175631640 1 -- 172.21.15.28:0/3963640194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560869643520 con 0x560869642b30 2024-09-19T11:23:22.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f3175631640 1 --2- 172.21.15.28:0/3963640194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x560869642f30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3164053d80 tx=0x7f3164053f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d9efdf34c016d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f304bfff640 1 -- 172.21.15.28:0/3963640194 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316404a070 con 0x560869642b30 2024-09-19T11:23:22.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f304bfff640 1 -- 172.21.15.28:0/3963640194 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31640156f0 con 0x560869642b30 2024-09-19T11:23:22.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.414+0000 7f304bfff640 1 -- 172.21.15.28:0/3963640194 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3164052070 con 0x560869642b30 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 -- 172.21.15.28:0/3963640194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 msgr2=0x560869642f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 --2- 172.21.15.28:0/3963640194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x560869642f30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3164053d80 tx=0x7f3164053f40 comp rx=0 tx=0).stop 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 -- 172.21.15.28:0/3963640194 shutdown_connections 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 --2- 172.21.15.28:0/3963640194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x560869642f30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 -- 172.21.15.28:0/3963640194 >> 172.21.15.28:0/3963640194 conn(0x56086963ef30 msgr2=0x56086963d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 -- 172.21.15.28:0/3963640194 shutdown_connections 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 -- 172.21.15.28:0/3963640194 wait complete. 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 Processor -- start 2024-09-19T11:23:22.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 -- start start 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x56086964a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.415+0000 7f3177346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560869643b40 con 0x560869642b30 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f3175631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x56086964a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f3175631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x56086964a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59154/0 (socket says 172.21.15.28:59154) 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 learned_addr learned my addr 172.21.15.28:0/2128320738 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086964a6a0 con 0x560869642b30 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f3175631640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x56086964a140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3164048040 tx=0x7f3164053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f304a7fc640 1 -- 172.21.15.28:0/2128320738 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3164030030 con 0x560869642b30 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f304a7fc640 1 -- 172.21.15.28:0/2128320738 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316404b040 con 0x560869642b30 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56086964a930 con 0x560869642b30 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f304a7fc640 1 -- 172.21.15.28:0/2128320738 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f316404a050 con 0x560869642b30 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.416+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086964b2b0 con 0x560869642b30 2024-09-19T11:23:22.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.417+0000 7f304a7fc640 1 -- 172.21.15.28:0/2128320738 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3164033070 con 0x560869642b30 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.417+0000 7f304a7fc640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304003d000 0x7f304003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.417+0000 7f304a7fc640 1 -- 172.21.15.28:0/2128320738 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f3164031070 con 0x560869642b30 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.417+0000 7f3174e30640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304003d000 0x7f304003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.417+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.417+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869655a70 0x560869657f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869658680 con 0x560869655a70 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3174e30640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304003d000 0x7f304003f4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f316c02d6a0 tx=0x7f316c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f317462f640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869655a70 0x560869657f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f317462f640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869655a70 0x560869657f30 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 20 realm 2024-09-19T11:23:22.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869658ab0 con 0x560869655a70 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.418+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869659380 con 0x560869655a70 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869659720 con 0x560869655a70 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56086965a480 0x56086965c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56086965cdb0 con 0x56086965a480 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3175631640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56086965a480 0x56086965c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.419+0000 7f3175631640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56086965a480 0x56086965c870 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3177346800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3177346800 20 searching for the correct realm 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3175631640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3164035030 0x7f3164037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f3164035030 2024-09-19T11:23:22.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3174e30640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3164035030 0x7f3164037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.420+0000 7f3174e30640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3164035030 0x7f3164037a80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.421+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.421+0000 7f3174e30640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f316c026020 0x7f316c0532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.421+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.421+0000 7f317462f640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f316c026020 0x7f316c0532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.421+0000 7f317462f640 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f316c026020 0x7f316c0532a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.421+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.421+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x560869655a70 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x560869655a70 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.422+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f3164035030 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.423+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.423+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.423+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.423+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x560869655a70 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.423+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.423+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x560869655a70 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f3164035030 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.424+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.425+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.425+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.425+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.425+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x560869655a70 2024-09-19T11:23:22.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x560869655a70 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.426+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f3164035030 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x560869655a70 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.427+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f316c026020 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x7f3164035030 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56086965e750 con 0x56086965a480 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.428+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f316406b040 con 0x7f316c026020 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f316406b040 con 0x560869655a70 2024-09-19T11:23:22.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f316406b040 con 0x7f3164035030 2024-09-19T11:23:22.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.429+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56086965d720 con 0x560869655a70 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56086965db10 con 0x560869655a70 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56086965df30 con 0x56086965a480 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 zone default found 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 4 Realm: () 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 zonegroup default 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56086965ec20 con 0x7f316c026020 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.430+0000 7f3177346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869660b40 con 0x56086965a480 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560869660fa0 con 0x56086965a480 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 20 started sync module instance, tier type = 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.431+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56086965fe50 con 0x7f3164035030 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.432+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.432+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94594128017824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869660fa0 con 0x7f3164035030 2024-09-19T11:23:22.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.432+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696617f0 con 0x560869655a70 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.433+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94594128017824] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.434+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.434+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94594128024384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869661c10 con 0x560869655a70 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.434+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696625d0 con 0x56086965a480 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.435+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94594128024384] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.435+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.435+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94594128025440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696625d0 con 0x56086965a480 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.435+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869663040 con 0x7f3164035030 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.436+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94594128025440] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.436+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.436+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94594128027936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869662540 con 0x7f3164035030 2024-09-19T11:23:22.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.436+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869663f00 con 0x560869655a70 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.437+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.437+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94594128034384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869663ed0 con 0x560869655a70 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.437+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869664ee0 con 0x7f316c026020 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.437+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94594128027936] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.438+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.438+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94594128034384] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.438+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94594128038288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869664ea0 con 0x7f316c026020 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.438+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696642f0 con 0x7f316c026020 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.439+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94594128038288] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.439+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.439+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94594128032816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869665440 con 0x7f316c026020 2024-09-19T11:23:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.439+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869665d30 con 0x7f316c026020 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94594128041024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696663c0 con 0x7f316c026020 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 20 add_watcher() i=0 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 20 add_watcher() i=1 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 20 add_watcher() i=2 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 20 add_watcher() i=3 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 20 add_watcher() i=4 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 20 add_watcher() i=5 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94594128032816] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.440+0000 7f3177346800 20 add_watcher() i=6 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.441+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94594128041024] v31'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c057030 con 0x7f316c026020 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.441+0000 7f3177346800 20 add_watcher() i=7 2024-09-19T11:23:22.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.441+0000 7f3177346800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.441+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5608696692e0 con 0x560869655a70 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.442+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.442+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94594128043520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696666d0 con 0x560869655a70 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.442+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94594128043520] v31'58 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f302b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f3177346800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f3177346800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f3177346800 5 note: GC not initialized 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f302a7e4640 20 reqs_thread_entry: start 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f3177346800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f3177346800 20 Filter name: none 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f30237fe640 20 reqs_thread_entry: start 2024-09-19T11:23:22.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f3177346800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.443+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56086969bb60 con 0x560869655a70 2024-09-19T11:23:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.444+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f315c057030 con 0x560869655a70 2024-09-19T11:23:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.444+0000 7f3177346800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:22.455 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:22.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.445+0000 7f302b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.445+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94594128043520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869652dd0 con 0x560869655a70 2024-09-19T11:23:22.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.446+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94594128043520] v31'59 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f315c089040 con 0x560869655a70 2024-09-19T11:23:22.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.446+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94594128017824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869665e00 con 0x7f3164035030 2024-09-19T11:23:22.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.447+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94594128017824] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.447+0000 7f3177346800 20 remove_watcher() i=0 2024-09-19T11:23:22.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.447+0000 7f3177346800 2 removed watcher, disabling cache 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.447+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94594128024384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56086965f9a0 con 0x560869655a70 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.448+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94594128024384] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c089040 con 0x560869655a70 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.448+0000 7f3177346800 20 remove_watcher() i=1 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.448+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94594128025440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560869661340 con 0x56086965a480 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.449+0000 7f3175631640 1 -- 172.21.15.28:0/2128320738 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94594128025440] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316404d070 con 0x56086965a480 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.449+0000 7f3177346800 20 remove_watcher() i=2 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.449+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94594128027936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56086965f9a0 con 0x7f3164035030 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.450+0000 7f3174e30640 1 -- 172.21.15.28:0/2128320738 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94594128027936] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f316c00f070 con 0x7f3164035030 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.450+0000 7f3177346800 20 remove_watcher() i=3 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.450+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94594128034384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696617f0 con 0x560869655a70 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.451+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94594128034384] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c089040 con 0x560869655a70 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.451+0000 7f3177346800 20 remove_watcher() i=4 2024-09-19T11:23:22.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.451+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94594128038288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696617f0 con 0x7f316c026020 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.452+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94594128038288] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c089040 con 0x7f316c026020 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.452+0000 7f3177346800 20 remove_watcher() i=5 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.452+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94594128032816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696617f0 con 0x7f316c026020 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.453+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94594128032816] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c089040 con 0x7f316c026020 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.453+0000 7f3177346800 20 remove_watcher() i=6 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.453+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94594128041024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5608696617f0 con 0x7f316c026020 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94594128041024] v31'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f315c089040 con 0x7f316c026020 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 20 remove_watcher() i=7 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f316c026020 msgr2=0x7f316c0532a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f316c026020 0x7f316c0532a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869655a70 msgr2=0x560869657f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869655a70 0x560869657f30 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3164035030 msgr2=0x7f3164037a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3164035030 0x7f3164037a80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56086965a480 msgr2=0x56086965c870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56086965a480 0x56086965c870 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304003d000 msgr2=0x7f304003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304003d000 0x7f304003f4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f316c02d6a0 tx=0x7f316c04e000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 msgr2=0x56086964a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f317462f640 1 -- 172.21.15.28:0/2128320738 reap_dead start 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x56086964a140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3164048040 tx=0x7f3164053ce0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 shutdown_connections 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 --2- 172.21.15.28:0/2128320738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869642b30 0x56086964a140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.454+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 >> 172.21.15.28:0/2128320738 conn(0x56086963ef30 msgr2=0x56086963d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 shutdown_connections 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2128320738 wait complete. 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304c03d000 msgr2=0x7f304c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304c03d000 0x7f304c03f4c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56086961d230 tx=0x7f316c02d590 comp rx=0 tx=0).stop 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869636070 msgr2=0x560869636490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869636070 0x560869636490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56086961bf70 tx=0x7f316400bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 shutdown_connections 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f304c03d000 0x7f304c03f4c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/1409862379 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869636070 0x560869636490 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 >> 172.21.15.28:0/1409862379 conn(0x56086962a420 msgr2=0x56086962a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 shutdown_connections 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/1409862379 wait complete. 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869497610 msgr2=0x560869622730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869497610 0x560869622730 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560869624ad0 msgr2=0x560869626f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560869624ad0 0x560869626f70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f305c03d000 msgr2=0x7f305c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f305c03d000 0x7f305c03f4c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f315c057000 tx=0x7f315c052000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 msgr2=0x56086961b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086961b0e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x560869614460 tx=0x7f316c001250 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 shutdown_connections 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560869624ad0 0x560869626f70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f305c03d000 0x7f305c03f4c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560869497610 0x560869622730 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 --2- 172.21.15.28:0/2811039512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560869496280 0x56086961b0e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.455+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 >> 172.21.15.28:0/2811039512 conn(0x56086951cc60 msgr2=0x5608696149c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.456+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 shutdown_connections 2024-09-19T11:23:22.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.456+0000 7f3177346800 1 -- 172.21.15.28:0/2811039512 wait complete. 2024-09-19T11:23:22.466 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:22.466 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-09-19T11:23:22.467 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-09-19T11:23:22.467 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-09-19T11:23:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85f725800 1 Processor -- start 2024-09-19T11:23:22.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85f725800 1 -- start start 2024-09-19T11:23:22.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85f725800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55bb5c84cba0 0x55bb5c84cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85f725800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55bb5c84e2b0 con 0x55bb5c84d5d0 2024-09-19T11:23:22.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85f725800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55bb5c6d87d0 con 0x55bb5c84cba0 2024-09-19T11:23:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85b333640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55bb5c84cba0 0x55bb5c84cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85b333640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55bb5c84cba0 0x55bb5c84cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59164/0 (socket says 172.21.15.28:59164) 2024-09-19T11:23:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.512+0000 7fc85b333640 1 -- 172.21.15.28:0/1002868989 learned_addr learned my addr 172.21.15.28:0/1002868989 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.513+0000 7fc85b333640 1 -- 172.21.15.28:0/1002868989 >> v1:172.21.15.22:6789/0 conn(0x55bb5c84d5d0 legacy=0x55bb5c84da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.513+0000 7fc85b333640 1 -- 172.21.15.28:0/1002868989 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5c854750 con 0x55bb5c84cba0 2024-09-19T11:23:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.514+0000 7fc85b333640 1 --2- 172.21.15.28:0/1002868989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84cfa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00a520 tx=0x7fc84c045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aac0e440109ab23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.514+0000 7fc85a331640 1 -- 172.21.15.28:0/1002868989 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc84c055070 con 0x55bb5c84cba0 2024-09-19T11:23:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.514+0000 7fc85a331640 1 -- 172.21.15.28:0/1002868989 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84c0515b0 con 0x55bb5c84cba0 2024-09-19T11:23:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85a331640 1 -- 172.21.15.28:0/1002868989 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc84c0518d0 con 0x55bb5c84cba0 2024-09-19T11:23:22.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85f725800 1 -- 172.21.15.28:0/1002868989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 msgr2=0x55bb5c84cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85f725800 1 --2- 172.21.15.28:0/1002868989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84cfa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00a520 tx=0x7fc84c045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85f725800 1 -- 172.21.15.28:0/1002868989 shutdown_connections 2024-09-19T11:23:22.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85f725800 1 --2- 172.21.15.28:0/1002868989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84cfa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85f725800 1 -- 172.21.15.28:0/1002868989 >> 172.21.15.28:0/1002868989 conn(0x55bb5c8471c0 msgr2=0x55bb5c849640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85f725800 1 -- 172.21.15.28:0/1002868989 shutdown_connections 2024-09-19T11:23:22.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.515+0000 7fc85f725800 1 -- 172.21.15.28:0/1002868989 wait complete. 2024-09-19T11:23:22.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.530+0000 7fc85f725800 1 Processor -- start 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85f725800 1 -- start start 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85f725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85f725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bb5c6e5dd0 con 0x55bb5c84cba0 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85b333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85b333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59174/0 (socket says 172.21.15.28:59174) 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85b333640 1 -- 172.21.15.28:0/2140394843 learned_addr learned my addr 172.21.15.28:0/2140394843 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85b333640 1 -- 172.21.15.28:0/2140394843 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5c84c2f0 con 0x55bb5c84cba0 2024-09-19T11:23:22.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.531+0000 7fc85b333640 1 --2- 172.21.15.28:0/2140394843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84bd00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc84c025da0 tx=0x7fc84c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc42b6fb2f04421 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc74960a640 1 -- 172.21.15.28:0/2140394843 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc84c051950 con 0x55bb5c84cba0 2024-09-19T11:23:22.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc74960a640 1 -- 172.21.15.28:0/2140394843 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84c051af0 con 0x55bb5c84cba0 2024-09-19T11:23:22.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc74960a640 1 -- 172.21.15.28:0/2140394843 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc84c051dd0 con 0x55bb5c84cba0 2024-09-19T11:23:22.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc85f725800 1 -- 172.21.15.28:0/2140394843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 msgr2=0x55bb5c84bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc85f725800 1 --2- 172.21.15.28:0/2140394843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84bd00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc84c025da0 tx=0x7fc84c051000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc85f725800 1 -- 172.21.15.28:0/2140394843 shutdown_connections 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc85f725800 1 --2- 172.21.15.28:0/2140394843 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c84bd00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc85f725800 1 -- 172.21.15.28:0/2140394843 >> 172.21.15.28:0/2140394843 conn(0x55bb5c8471c0 msgr2=0x55bb5c937bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc85f725800 1 -- 172.21.15.28:0/2140394843 shutdown_connections 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.532+0000 7fc85f725800 1 -- 172.21.15.28:0/2140394843 wait complete. 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85f725800 1 Processor -- start 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85f725800 1 -- start start 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85f725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c944d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85f725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bb5c853a40 con 0x55bb5c84cba0 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85b333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c944d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85b333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c944d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59180/0 (socket says 172.21.15.28:59180) 2024-09-19T11:23:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85b333640 1 -- 172.21.15.28:0/1364774673 learned_addr learned my addr 172.21.15.28:0/1364774673 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.533+0000 7fc85b333640 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5c945290 con 0x55bb5c84cba0 2024-09-19T11:23:22.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.534+0000 7fc85b333640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c944d30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc84c067040 tx=0x7fc84c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.534+0000 7fc747e07640 1 -- 172.21.15.28:0/1364774673 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc84c073070 con 0x55bb5c84cba0 2024-09-19T11:23:22.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.534+0000 7fc747e07640 1 -- 172.21.15.28:0/1364774673 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84c05a450 con 0x55bb5c84cba0 2024-09-19T11:23:22.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.534+0000 7fc747e07640 1 -- 172.21.15.28:0/1364774673 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc84c05a730 con 0x55bb5c84cba0 2024-09-19T11:23:22.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.534+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bb5c945520 con 0x55bb5c84cba0 2024-09-19T11:23:22.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.534+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5c946130 con 0x55bb5c84cba0 2024-09-19T11:23:22.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc747e07640 1 -- 172.21.15.28:0/1364774673 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc84c076070 con 0x55bb5c84cba0 2024-09-19T11:23:22.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc747e07640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc74003d000 0x7fc74003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc747e07640 1 -- 172.21.15.28:0/1364774673 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc84c062070 con 0x55bb5c84cba0 2024-09-19T11:23:22.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc85ab32640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc74003d000 0x7fc74003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c84d5d0 0x55bb5c94c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c94c8c0 con 0x55bb5c84d5d0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc85bb34640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c84d5d0 0x55bb5c94c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc85ab32640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc74003d000 0x7fc74003f4c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc850001e50 tx=0x7fc8500013d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.535+0000 7fc85bb34640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c84d5d0 0x55bb5c94c380 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.536+0000 7fc85bb34640 1 -- 172.21.15.28:0/1364774673 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c84d5d0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.536+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c94cc60 con 0x55bb5c84d5d0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.536+0000 7fc85bb34640 1 -- 172.21.15.28:0/1364774673 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c84d5d0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.536+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c94e760 0x55bb5c950be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.536+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c951370 con 0x55bb5c94e760 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.536+0000 7fc85b333640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c94e760 0x55bb5c950be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.536+0000 7fc85b333640 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c94e760 0x55bb5c950be0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.537+0000 7fc85b333640 1 -- 172.21.15.28:0/1364774673 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc84c075030 con 0x55bb5c94e760 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.537+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c94d4b0 con 0x55bb5c84d5d0 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.537+0000 7fc85bb34640 1 -- 172.21.15.28:0/1364774673 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c84d5d0 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.537+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c94d900 con 0x55bb5c94e760 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.537+0000 7fc85b333640 1 -- 172.21.15.28:0/1364774673 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc84c075030 con 0x55bb5c94e760 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.538+0000 7fc85f725800 1 Processor -- start 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.538+0000 7fc85f725800 1 -- start start 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.538+0000 7fc85f725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c95fce0 0x55bb5c960100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.538+0000 7fc85f725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bb5c7444a0 con 0x55bb5c95fce0 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.538+0000 7fc85bb34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c95fce0 0x55bb5c960100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.538+0000 7fc85bb34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c95fce0 0x55bb5c960100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59196/0 (socket says 172.21.15.28:59196) 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.538+0000 7fc85bb34640 1 -- 172.21.15.28:0/2061010700 learned_addr learned my addr 172.21.15.28:0/2061010700 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.539+0000 7fc85bb34640 1 -- 172.21.15.28:0/2061010700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5c960740 con 0x55bb5c95fce0 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.539+0000 7fc85bb34640 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c95fce0 0x55bb5c960100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55bb5c945bc0 tx=0x7fc8540167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.539+0000 7fc745602640 1 -- 172.21.15.28:0/2061010700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc854013640 con 0x55bb5c95fce0 2024-09-19T11:23:22.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.539+0000 7fc745602640 1 -- 172.21.15.28:0/2061010700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8540137e0 con 0x55bb5c95fce0 2024-09-19T11:23:22.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.539+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bb5c960f70 con 0x55bb5c95fce0 2024-09-19T11:23:22.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.539+0000 7fc745602640 1 -- 172.21.15.28:0/2061010700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc854013ae0 con 0x55bb5c95fce0 2024-09-19T11:23:22.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.539+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5c961510 con 0x55bb5c95fce0 2024-09-19T11:23:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.540+0000 7fc745602640 1 -- 172.21.15.28:0/2061010700 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc854011040 con 0x55bb5c95fce0 2024-09-19T11:23:22.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.540+0000 7fc745602640 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc73403d000 0x7fc73403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.540+0000 7fc745602640 1 -- 172.21.15.28:0/2061010700 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc85400e040 con 0x55bb5c95fce0 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.540+0000 7fc85b333640 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc73403d000 0x7fc73403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.540+0000 7fc85f725800 1 Processor -- start 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.540+0000 7fc85f725800 1 -- start start 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.540+0000 7fc85b333640 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc73403d000 0x7fc73403f4c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc84c064e50 tx=0x7fc84c096fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc85f725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c96cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc85f725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bb5c940130 con 0x55bb5c96c760 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc85bb34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c96cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc85bb34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c96cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59210/0 (socket says 172.21.15.28:59210) 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc85bb34640 1 -- 172.21.15.28:0/400613961 learned_addr learned my addr 172.21.15.28:0/400613961 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc85bb34640 1 -- 172.21.15.28:0/400613961 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5c96d150 con 0x55bb5c96c760 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc85bb34640 1 --2- 172.21.15.28:0/400613961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c96cb60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8540547f0 tx=0x7fc85401fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddaf19e1aedf9536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc73effd640 1 -- 172.21.15.28:0/400613961 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc854051070 con 0x55bb5c96c760 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc73effd640 1 -- 172.21.15.28:0/400613961 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854053440 con 0x55bb5c96c760 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.541+0000 7fc73effd640 1 -- 172.21.15.28:0/400613961 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc854021a90 con 0x55bb5c96c760 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 -- 172.21.15.28:0/400613961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 msgr2=0x55bb5c96cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 --2- 172.21.15.28:0/400613961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c96cb60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc8540547f0 tx=0x7fc85401fe70 comp rx=0 tx=0).stop 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 -- 172.21.15.28:0/400613961 shutdown_connections 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 --2- 172.21.15.28:0/400613961 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c96cb60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 -- 172.21.15.28:0/400613961 >> 172.21.15.28:0/400613961 conn(0x55bb5c968b60 msgr2=0x55bb5c966da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 -- 172.21.15.28:0/400613961 shutdown_connections 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 -- 172.21.15.28:0/400613961 wait complete. 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 Processor -- start 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 -- start start 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c973db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85f725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bb5c96d770 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.542+0000 7fc85bb34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c973db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc85bb34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c973db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59216/0 (socket says 172.21.15.28:59216) 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 learned_addr learned my addr 172.21.15.28:0/1215524785 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb5c974310 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc85bb34640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c973db0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8540547f0 tx=0x7fc854054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc73d7fa640 1 -- 172.21.15.28:0/1215524785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc854021440 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc73d7fa640 1 -- 172.21.15.28:0/1215524785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85404c070 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bb5c9745a0 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc73d7fa640 1 -- 172.21.15.28:0/1215524785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8540459c0 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.543+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb5c975130 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.544+0000 7fc73d7fa640 1 -- 172.21.15.28:0/1215524785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc854045b60 con 0x55bb5c96c760 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.544+0000 7fc73d7fa640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc72403d000 0x7fc72403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.544+0000 7fc73d7fa640 1 -- 172.21.15.28:0/1215524785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc8540539d0 con 0x55bb5c96c760 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.544+0000 7fc85b333640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc72403d000 0x7fc72403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.545+0000 7fc85b333640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc72403d000 0x7fc72403f4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005570 tx=0x7fc84c097340 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.545+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.545+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c97f6f0 0x55bb5c981bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.545+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c982300 con 0x55bb5c97f6f0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.545+0000 7fc85ab32640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c97f6f0 0x55bb5c981bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.545+0000 7fc85ab32640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c97f6f0 0x55bb5c981bb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc850010580 con 0x55bb5c97f6f0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 realm 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c97ea40 con 0x55bb5c97f6f0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc850010580 con 0x55bb5c97f6f0 2024-09-19T11:23:22.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c982df0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc850010580 con 0x55bb5c97f6f0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9836c0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc850010580 con 0x55bb5c97f6f0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c9840e0 0x55bb5c9864d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c986c10 con 0x55bb5c9840e0 2024-09-19T11:23:22.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.546+0000 7fc85bb34640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c9840e0 0x55bb5c9864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85bb34640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c9840e0 0x55bb5c9864d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85f725800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85f725800 20 searching for the correct realm 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85bb34640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc85404d2a0 0x7fc85403d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc85404d2a0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.547+0000 7fc85b333640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc85404d2a0 0x7fc85403d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.548+0000 7fc85b333640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc85404d2a0 0x7fc85403d7f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.548+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.548+0000 7fc85b333640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc84c0a6040 0x7fc84c050610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.548+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.548+0000 7fc85ab32640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc84c0a6040 0x7fc84c050610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.548+0000 7fc85ab32640 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc84c0a6040 0x7fc84c050610 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc850010580 con 0x7fc84c0a6040 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc850010580 con 0x55bb5c97f6f0 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.549+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc85404d2a0 2024-09-19T11:23:22.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.550+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc85404d2a0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.551+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.552+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.552+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.552+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.552+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.552+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.552+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.553+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.554+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.554+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc85404d2a0 2024-09-19T11:23:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.554+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.554+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.554+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.554+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.555+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.555+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.555+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.555+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.555+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.555+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc85404d2a0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c9840e0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc84c0a6040 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x55bb5c97f6f0 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988560 con 0x7fc85404d2a0 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.556+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9875d0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c987e70 con 0x55bb5c97f6f0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9889e0 con 0x55bb5c9840e0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 zone default found 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 4 Realm: () 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 zonegroup default 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.557+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c988de0 con 0x7fc84c0a6040 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98ac10 con 0x55bb5c9840e0 2024-09-19T11:23:22.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98b0a0 con 0x55bb5c9840e0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 20 started sync module instance, tier type = 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.558+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98a0b0 con 0x7fc85404d2a0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.559+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.559+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94263200750592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98b500 con 0x7fc85404d2a0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.559+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98bd50 con 0x55bb5c97f6f0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.561+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94263200750592] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.561+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.561+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94263200757920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98c170 con 0x55bb5c97f6f0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.561+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98c9c0 con 0x55bb5c9840e0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.562+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94263200757920] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.562+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.562+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94263200759456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98c8d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.562+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98d3c0 con 0x7fc85404d2a0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.563+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94263200759456] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.563+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.563+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94263200763536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98d7b0 con 0x7fc85404d2a0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.563+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98e1f0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.564+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.564+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94263200767168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98e1f0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.564+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98f1f0 con 0x7fc84c0a6040 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.564+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94263200763536] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.565+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94263200767168] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.565+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.565+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94263200771296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98e3d0 con 0x7fc84c0a6040 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.565+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98fa90 con 0x7fc84c0a6040 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.566+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.566+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94263200773600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98e3d0 con 0x7fc84c0a6040 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.566+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9905e0 con 0x7fc84c0a6040 2024-09-19T11:23:22.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.566+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94263200771296] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94263200773600] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94263200774656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c98c320 con 0x7fc84c0a6040 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 20 add_watcher() i=0 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 20 add_watcher() i=1 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 20 add_watcher() i=2 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 20 add_watcher() i=3 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 20 add_watcher() i=4 2024-09-19T11:23:22.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 20 add_watcher() i=5 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.567+0000 7fc85f725800 20 add_watcher() i=6 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.568+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94263200774656] v31'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.568+0000 7fc85f725800 20 add_watcher() i=7 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.568+0000 7fc85f725800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.568+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9935d0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.568+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.568+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94263200776400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c990a50 con 0x55bb5c97f6f0 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.569+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94263200776400] v31'60 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.569+0000 7fc711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 5 note: GC not initialized 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc710fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 20 Filter name: none 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc68a7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.570+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c6370 con 0x55bb5c97f6f0 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85f725800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85f725800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85f725800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85f725800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c8ca0 con 0x7fc85404d2a0 2024-09-19T11:23:22.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.571+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9cc470 con 0x55bb5c97f6f0 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v31'3 uv3 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc85f725800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc85f725800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc85f725800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94263201021488 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c7c90 con 0x55bb5c97f6f0 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94263200767168 notify 133143986178 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55bb5c84e2b0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc73effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986178 cookie 94263200767168 notifier 4327 bl.length()=567 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc73effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc73effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc73effd640 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fc720003ea0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.572+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(65 notify.4 [notify cookie 94263201021488 out=8b] v0'0 uv39 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.573+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.573+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94263201021488 notify 133143986178 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fc85000fcc0 con 0x55bb5c97f6f0 2024-09-19T11:23:22.584 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.574+0000 7fc711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.574+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94263200776400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x55bb5c97f6f0 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.575+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94263200776400] v31'61 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.575+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94263200750592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x7fc85404d2a0 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.576+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94263200750592] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.576+0000 7fc85f725800 20 remove_watcher() i=0 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.576+0000 7fc85f725800 2 removed watcher, disabling cache 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.576+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94263200757920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x55bb5c97f6f0 2024-09-19T11:23:22.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.577+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94263200757920] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.577+0000 7fc85f725800 20 remove_watcher() i=1 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.577+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94263200759456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x55bb5c9840e0 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.578+0000 7fc85bb34640 1 -- 172.21.15.28:0/1215524785 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94263200759456] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85400a4d0 con 0x55bb5c9840e0 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.578+0000 7fc85f725800 20 remove_watcher() i=2 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.578+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94263200763536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x7fc85404d2a0 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.579+0000 7fc85b333640 1 -- 172.21.15.28:0/1215524785 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94263200763536] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc84c075030 con 0x7fc85404d2a0 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.579+0000 7fc85f725800 20 remove_watcher() i=3 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.579+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94263200767168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x55bb5c97f6f0 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.580+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94263200767168] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x55bb5c97f6f0 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.580+0000 7fc85f725800 20 remove_watcher() i=4 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.580+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94263200771296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x7fc84c0a6040 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.581+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94263200771296] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.581+0000 7fc85f725800 20 remove_watcher() i=5 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.581+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94263200773600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x7fc84c0a6040 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.582+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94263200773600] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.582+0000 7fc85f725800 20 remove_watcher() i=6 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.582+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94263200774656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55bb5c9c4210 con 0x7fc84c0a6040 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94263200774656] v31'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc85002a040 con 0x7fc84c0a6040 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 20 remove_watcher() i=7 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc84c0a6040 msgr2=0x7fc84c050610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc84c0a6040 0x7fc84c050610 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c97f6f0 msgr2=0x55bb5c981bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c97f6f0 0x55bb5c981bb0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc85404d2a0 msgr2=0x7fc85403d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc85404d2a0 0x7fc85403d7f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c9840e0 msgr2=0x55bb5c9864d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.583+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c9840e0 0x55bb5c9864d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc72403d000 msgr2=0x7fc72403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc72403d000 0x7fc72403f4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005570 tx=0x7fc84c097340 comp rx=0 tx=0).stop 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 msgr2=0x55bb5c973db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85ab32640 1 -- 172.21.15.28:0/1215524785 reap_dead start 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c973db0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc8540547f0 tx=0x7fc854054780 comp rx=0 tx=0).stop 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 shutdown_connections 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 --2- 172.21.15.28:0/1215524785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c96c760 0x55bb5c973db0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 >> 172.21.15.28:0/1215524785 conn(0x55bb5c968b60 msgr2=0x55bb5c966d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 shutdown_connections 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/1215524785 wait complete. 2024-09-19T11:23:22.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc73403d000 msgr2=0x7fc73403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc73403d000 0x7fc73403f4c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc84c064e50 tx=0x7fc84c096fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c95fce0 msgr2=0x55bb5c960100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c95fce0 0x55bb5c960100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55bb5c945bc0 tx=0x7fc8540167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 shutdown_connections 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc73403d000 0x7fc73403f4c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 --2- 172.21.15.28:0/2061010700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c95fce0 0x55bb5c960100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 >> 172.21.15.28:0/2061010700 conn(0x55bb5c954090 msgr2=0x55bb5c954470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 shutdown_connections 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/2061010700 wait complete. 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.584+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c84d5d0 msgr2=0x55bb5c94c380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c84d5d0 0x55bb5c94c380 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c94e760 msgr2=0x55bb5c950be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c94e760 0x55bb5c950be0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc74003d000 msgr2=0x7fc74003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc74003d000 0x7fc74003f4c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc850001e50 tx=0x7fc8500013d0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 msgr2=0x55bb5c944d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c944d30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc84c067040 tx=0x7fc84c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 shutdown_connections 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55bb5c94e760 0x55bb5c950be0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc74003d000 0x7fc74003f4c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55bb5c84d5d0 0x55bb5c94c380 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 --2- 172.21.15.28:0/1364774673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bb5c84cba0 0x55bb5c944d30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 >> 172.21.15.28:0/1364774673 conn(0x55bb5c8471c0 msgr2=0x55bb5c9363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 shutdown_connections 2024-09-19T11:23:22.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.585+0000 7fc85f725800 1 -- 172.21.15.28:0/1364774673 wait complete. 2024-09-19T11:23:22.595 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:22.595 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.596 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.596 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:22.641 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.641 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc13c722800 1 Processor -- start 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc13c722800 1 -- start start 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc13c722800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5625875470a0 0x56258754ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc13c722800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56258754f5f0 con 0x562587546bb0 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc13c722800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5625873d37d0 con 0x5625875470a0 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc137bd1640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5625875470a0 0x56258754ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc1383d2640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x562587546bb0 0x56258754b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33048/0 (socket says 172.21.15.28:33048) 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc1383d2640 1 -- 172.21.15.28:0/3251811195 learned_addr learned my addr 172.21.15.28:0/3251811195 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc137bd1640 1 -- 172.21.15.28:0/3251811195 >> v1:172.21.15.22:6789/0 conn(0x562587546bb0 legacy=0x56258754b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc137bd1640 1 -- 172.21.15.28:0/3251811195 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562587550820 con 0x5625875470a0 2024-09-19T11:23:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.641+0000 7fc1373d0640 1 -- 172.21.15.28:0/3251811195 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 36543163 0 0) 0x56258754f5f0 con 0x562587546bb0 2024-09-19T11:23:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.643+0000 7fc137bd1640 1 --2- 172.21.15.28:0/3251811195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625875470a0 0x56258754ef70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00a470 tx=0x7fc12c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829693cbcd2a2cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.643+0000 7fc1373d0640 1 -- 172.21.15.28:0/3251811195 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc12c053570 con 0x5625875470a0 2024-09-19T11:23:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.643+0000 7fc1373d0640 1 -- 172.21.15.28:0/3251811195 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12c058070 con 0x5625875470a0 2024-09-19T11:23:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.643+0000 7fc1373d0640 1 -- 172.21.15.28:0/3251811195 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc12c053880 con 0x5625875470a0 2024-09-19T11:23:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.644+0000 7fc13c722800 1 -- 172.21.15.28:0/3251811195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625875470a0 msgr2=0x56258754ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.644+0000 7fc13c722800 1 --2- 172.21.15.28:0/3251811195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625875470a0 0x56258754ef70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00a470 tx=0x7fc12c045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.644+0000 7fc13c722800 1 -- 172.21.15.28:0/3251811195 shutdown_connections 2024-09-19T11:23:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.644+0000 7fc13c722800 1 --2- 172.21.15.28:0/3251811195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625875470a0 0x56258754ef70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.644+0000 7fc13c722800 1 -- 172.21.15.28:0/3251811195 >> 172.21.15.28:0/3251811195 conn(0x5625875421c0 msgr2=0x562587544640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.644+0000 7fc13c722800 1 -- 172.21.15.28:0/3251811195 shutdown_connections 2024-09-19T11:23:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.644+0000 7fc13c722800 1 -- 172.21.15.28:0/3251811195 wait complete. 2024-09-19T11:23:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.659+0000 7fc13c722800 1 Processor -- start 2024-09-19T11:23:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.659+0000 7fc13c722800 1 -- start start 2024-09-19T11:23:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.659+0000 7fc13c722800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587548fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.659+0000 7fc13c722800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625873e0dd0 con 0x562587546bb0 2024-09-19T11:23:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc1383d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587548fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc1383d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587548fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59230/0 (socket says 172.21.15.28:59230) 2024-09-19T11:23:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc1383d2640 1 -- 172.21.15.28:0/963307605 learned_addr learned my addr 172.21.15.28:0/963307605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc1383d2640 1 -- 172.21.15.28:0/963307605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562587549590 con 0x562587546bb0 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc1383d2640 1 --2- 172.21.15.28:0/963307605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587548fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc128001ea0 tx=0x7fc12800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fb8db7023ce527 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc02660c640 1 -- 172.21.15.28:0/963307605 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc128008070 con 0x562587546bb0 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc02660c640 1 -- 172.21.15.28:0/963307605 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12800d500 con 0x562587546bb0 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.660+0000 7fc02660c640 1 -- 172.21.15.28:0/963307605 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc12800d880 con 0x562587546bb0 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 -- 172.21.15.28:0/963307605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 msgr2=0x562587548fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 --2- 172.21.15.28:0/963307605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587548fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc128001ea0 tx=0x7fc12800b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 -- 172.21.15.28:0/963307605 shutdown_connections 2024-09-19T11:23:22.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 --2- 172.21.15.28:0/963307605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587548fa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 -- 172.21.15.28:0/963307605 >> 172.21.15.28:0/963307605 conn(0x5625875421c0 msgr2=0x562587633ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 -- 172.21.15.28:0/963307605 shutdown_connections 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 -- 172.21.15.28:0/963307605 wait complete. 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 Processor -- start 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.661+0000 7fc13c722800 1 -- start start 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc13c722800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587641e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc13c722800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56258754fac0 con 0x562587546bb0 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc1383d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587641e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc1383d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587641e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59238/0 (socket says 172.21.15.28:59238) 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc1383d2640 1 -- 172.21.15.28:0/3635469873 learned_addr learned my addr 172.21.15.28:0/3635469873 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc1383d2640 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562587642390 con 0x562587546bb0 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc1383d2640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587641e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc128000f60 tx=0x7fc128000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc024e09640 1 -- 172.21.15.28:0/3635469873 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc12800f8d0 con 0x562587546bb0 2024-09-19T11:23:22.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc024e09640 1 -- 172.21.15.28:0/3635469873 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc12800fa70 con 0x562587546bb0 2024-09-19T11:23:22.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562587642620 con 0x562587546bb0 2024-09-19T11:23:22.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.662+0000 7fc024e09640 1 -- 172.21.15.28:0/3635469873 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc12800fd50 con 0x562587546bb0 2024-09-19T11:23:22.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.663+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562587643020 con 0x562587546bb0 2024-09-19T11:23:22.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.663+0000 7fc024e09640 1 -- 172.21.15.28:0/3635469873 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc128017040 con 0x562587546bb0 2024-09-19T11:23:22.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc024e09640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc02003d000 0x7fc02003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc024e09640 1 -- 172.21.15.28:0/3635469873 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc128015020 con 0x562587546bb0 2024-09-19T11:23:22.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc137bd1640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc02003d000 0x7fc02003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625875470a0 0x562587649480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5625876499c0 con 0x5625875470a0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc138bd3640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625875470a0 0x562587649480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc137bd1640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc02003d000 0x7fc02003f4c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc12c057000 tx=0x7fc12c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.664+0000 7fc138bd3640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625875470a0 0x562587649480 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.665+0000 7fc138bd3640 1 -- 172.21.15.28:0/3635469873 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc13000a4d0 con 0x5625875470a0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.665+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587649d60 con 0x5625875470a0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.665+0000 7fc138bd3640 1 -- 172.21.15.28:0/3635469873 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc13000a4d0 con 0x5625875470a0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.665+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56258764b860 0x56258764dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.665+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56258764e470 con 0x56258764b860 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.665+0000 7fc1383d2640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56258764b860 0x56258764dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.665+0000 7fc1383d2640 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56258764b860 0x56258764dce0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.666+0000 7fc1383d2640 1 -- 172.21.15.28:0/3635469873 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc12802d070 con 0x56258764b860 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.666+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56258764a5b0 con 0x5625875470a0 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.666+0000 7fc138bd3640 1 -- 172.21.15.28:0/3635469873 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc13000a4d0 con 0x5625875470a0 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.666+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56258764a9d0 con 0x56258764b860 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.666+0000 7fc1383d2640 1 -- 172.21.15.28:0/3635469873 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc12802d070 con 0x56258764b860 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc13c722800 1 Processor -- start 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc13c722800 1 -- start start 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc13c722800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56258765cdc0 0x56258765d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc13c722800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56258743f4a0 con 0x56258765cdc0 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc138bd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56258765cdc0 0x56258765d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc138bd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56258765cdc0 0x56258765d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59250/0 (socket says 172.21.15.28:59250) 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc138bd3640 1 -- 172.21.15.28:0/397004440 learned_addr learned my addr 172.21.15.28:0/397004440 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc138bd3640 1 -- 172.21.15.28:0/397004440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258765d820 con 0x56258765cdc0 2024-09-19T11:23:22.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.667+0000 7fc138bd3640 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56258765cdc0 0x56258765d1e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x562587642e30 tx=0x7fc1300167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.668+0000 7fc01dffb640 1 -- 172.21.15.28:0/397004440 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130013640 con 0x56258765cdc0 2024-09-19T11:23:22.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.668+0000 7fc01dffb640 1 -- 172.21.15.28:0/397004440 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1300137e0 con 0x56258765cdc0 2024-09-19T11:23:22.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.668+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56258765e050 con 0x56258765cdc0 2024-09-19T11:23:22.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.668+0000 7fc01dffb640 1 -- 172.21.15.28:0/397004440 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130013ac0 con 0x56258765cdc0 2024-09-19T11:23:22.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.668+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258765e5f0 con 0x56258765cdc0 2024-09-19T11:23:22.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc01dffb640 1 -- 172.21.15.28:0/397004440 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc130011040 con 0x56258765cdc0 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc01dffb640 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc01003d000 0x7fc01003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc01dffb640 1 -- 172.21.15.28:0/397004440 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc13000e040 con 0x56258765cdc0 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc1383d2640 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc01003d000 0x7fc01003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc13c722800 1 Processor -- start 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc1383d2640 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc01003d000 0x7fc01003f4c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc12800ed30 tx=0x7fc12800ff70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc13c722800 1 -- start start 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc13c722800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587669c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.669+0000 7fc13c722800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625875479f0 con 0x562587669820 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc138bd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587669c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc138bd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587669c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59256/0 (socket says 172.21.15.28:59256) 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc138bd3640 1 -- 172.21.15.28:0/2144500446 learned_addr learned my addr 172.21.15.28:0/2144500446 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc138bd3640 1 -- 172.21.15.28:0/2144500446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258766a210 con 0x562587669820 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc138bd3640 1 --2- 172.21.15.28:0/2144500446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587669c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc130054860 tx=0x7fc130054a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e15feeff3e62ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc0177fe640 1 -- 172.21.15.28:0/2144500446 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130044030 con 0x562587669820 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc0177fe640 1 -- 172.21.15.28:0/2144500446 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc130013c60 con 0x562587669820 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc0177fe640 1 -- 172.21.15.28:0/2144500446 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc13004de60 con 0x562587669820 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc13c722800 1 -- 172.21.15.28:0/2144500446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 msgr2=0x562587669c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.670+0000 7fc13c722800 1 --2- 172.21.15.28:0/2144500446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587669c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc130054860 tx=0x7fc130054a30 comp rx=0 tx=0).stop 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 -- 172.21.15.28:0/2144500446 shutdown_connections 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 --2- 172.21.15.28:0/2144500446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587669c20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 -- 172.21.15.28:0/2144500446 >> 172.21.15.28:0/2144500446 conn(0x562587665c40 msgr2=0x562587663e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 -- 172.21.15.28:0/2144500446 shutdown_connections 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 -- 172.21.15.28:0/2144500446 wait complete. 2024-09-19T11:23:22.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 Processor -- start 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 -- start start 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587670e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc13c722800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56258766a830 con 0x562587669820 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc138bd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587670e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc138bd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587670e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59272/0 (socket says 172.21.15.28:59272) 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 learned_addr learned my addr 172.21.15.28:0/387419993 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.671+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562587671390 con 0x562587669820 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.672+0000 7fc138bd3640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587670e30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc130016cd0 tx=0x7fc1300547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.672+0000 7fc015ffb640 1 -- 172.21.15.28:0/387419993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130053530 con 0x562587669820 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.672+0000 7fc015ffb640 1 -- 172.21.15.28:0/387419993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1300145d0 con 0x562587669820 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.672+0000 7fc015ffb640 1 -- 172.21.15.28:0/387419993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc130020d00 con 0x562587669820 2024-09-19T11:23:22.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.672+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562587671620 con 0x562587669820 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.672+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625876721b0 con 0x562587669820 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.672+0000 7fc015ffb640 1 -- 172.21.15.28:0/387419993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc13001fa90 con 0x562587669820 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc015ffb640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc00803d150 0x7fc00803f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc015ffb640 1 -- 172.21.15.28:0/387419993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc130034c80 con 0x562587669820 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc1383d2640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc00803d150 0x7fc00803f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56258767c720 0x56258767ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56258767f330 con 0x56258767c720 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc137bd1640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56258767c720 0x56258767ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.673+0000 7fc1383d2640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc00803d150 0x7fc00803f610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc128053460 tx=0x7fc128041000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc137bd1640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56258767c720 0x56258767ebe0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 20 realm 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56258767f760 con 0x56258767c720 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587680030 con 0x56258767c720 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.674+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5625876803d0 con 0x56258767c720 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc13c722800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562587681130 0x562587683520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587683a60 con 0x562587681130 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc138bd3640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562587681130 0x562587683520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc138bd3640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562587681130 0x562587683520 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc13c722800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.675+0000 7fc13c722800 20 searching for the correct realm 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc138bd3640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc1300123d0 0x7fc13003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc1300123d0 2024-09-19T11:23:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc1383d2640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc1300123d0 0x7fc13003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc1383d2640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc1300123d0 0x7fc13003b7c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc1383d2640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc12803c040 0x7fc12804c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.676+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.677+0000 7fc137bd1640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc12803c040 0x7fc12804c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.677+0000 7fc137bd1640 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc12803c040 0x7fc12804c580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.677+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.677+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.677+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.677+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc1300123d0 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.678+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.679+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc1300123d0 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.680+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.681+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc1300123d0 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.682+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.683+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.683+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.683+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.683+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.683+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.683+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc1300123d0 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.684+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.684+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x562587681130 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.684+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.684+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc12803c040 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.684+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.684+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x56258767c720 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5625876853d0 con 0x7fc1300123d0 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5625876844d0 con 0x56258767c720 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587684d70 con 0x56258767c720 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.685+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587684110 con 0x562587681130 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 zone default found 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 4 Realm: () 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 zonegroup default 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587685d50 con 0x7fc12803c040 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587687be0 con 0x562587681130 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.686+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562587688070 con 0x562587681130 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.687+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.687+0000 7fc13c722800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.687+0000 7fc13c722800 20 started sync module instance, tier type = 2024-09-19T11:23:22.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.687+0000 7fc13c722800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.687+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562587686f80 con 0x7fc1300123d0 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.688+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.688+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94719185548192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562587686be0 con 0x7fc1300123d0 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.688+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562587688fa0 con 0x56258767c720 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.689+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94719185548192] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.689+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.689+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94719185554160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5625876893c0 con 0x56258767c720 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.689+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562587689d80 con 0x562587681130 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.690+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94719185554160] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.690+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.690+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94719185558192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5625876893c0 con 0x562587681130 2024-09-19T11:23:22.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.690+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768a7f0 con 0x7fc1300123d0 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.691+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94719185558192] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.692+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.692+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94719185555216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562587689330 con 0x7fc1300123d0 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.692+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768b6b0 con 0x56258767c720 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.693+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.693+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94719185564160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768bad0 con 0x56258767c720 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.693+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768c5b0 con 0x7fc12803c040 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.693+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94719185555216] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.694+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94719185564160] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.694+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.694+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94719185562592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768b620 con 0x7fc12803c040 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.694+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768ce80 con 0x7fc12803c040 2024-09-19T11:23:22.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.695+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.695+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94719185570256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768b620 con 0x7fc12803c040 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.695+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d9e0 con 0x7fc12803c040 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.695+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94719185562592] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94719185570256] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94719185571312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d4f0 con 0x7fc12803c040 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 20 add_watcher() i=0 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 20 add_watcher() i=1 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 20 add_watcher() i=2 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 20 add_watcher() i=3 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 20 add_watcher() i=4 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 20 add_watcher() i=5 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.696+0000 7fc13c722800 20 add_watcher() i=6 2024-09-19T11:23:22.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.697+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94719185571312] v31'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.697+0000 7fc13c722800 20 add_watcher() i=7 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.697+0000 7fc13c722800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.697+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5625876909d0 con 0x56258767c720 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.697+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.697+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94719185573104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768de30 con 0x56258767c720 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.698+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94719185573104] v31'62 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.698+0000 7fbfee7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.698+0000 7fc13c722800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.698+0000 7fc13c722800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.698+0000 7fc13c722800 5 note: GC not initialized 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.698+0000 7fbfed7e2640 20 reqs_thread_entry: start 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.699+0000 7fc13c722800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.699+0000 7fc13c722800 20 Filter name: none 2024-09-19T11:23:22.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.699+0000 7fbf5effd640 20 reqs_thread_entry: start 2024-09-19T11:23:22.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.699+0000 7fc13c722800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.699+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5625876c32b0 con 0x56258767c720 2024-09-19T11:23:22.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.699+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.699+0000 7fc13c722800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:22.710 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.700+0000 7fbfee7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.700+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94719185573104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5625876c2140 con 0x56258767c720 2024-09-19T11:23:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.701+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94719185573104] v31'63 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.701+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94719185548192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d4f0 con 0x7fc1300123d0 2024-09-19T11:23:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.702+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94719185548192] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.702+0000 7fc13c722800 20 remove_watcher() i=0 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.702+0000 7fc13c722800 2 removed watcher, disabling cache 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.702+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94719185554160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d4f0 con 0x56258767c720 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.703+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94719185554160] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.703+0000 7fc13c722800 20 remove_watcher() i=1 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.703+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94719185558192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562587688af0 con 0x562587681130 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.704+0000 7fc138bd3640 1 -- 172.21.15.28:0/387419993 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94719185558192] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc130053020 con 0x562587681130 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.704+0000 7fc13c722800 20 remove_watcher() i=2 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.704+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94719185555216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d4f0 con 0x7fc1300123d0 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.705+0000 7fc1383d2640 1 -- 172.21.15.28:0/387419993 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94719185555216] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc128039070 con 0x7fc1300123d0 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.705+0000 7fc13c722800 20 remove_watcher() i=3 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.705+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94719185564160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d4f0 con 0x56258767c720 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.706+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94719185564160] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x56258767c720 2024-09-19T11:23:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.706+0000 7fc13c722800 20 remove_watcher() i=4 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.706+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94719185562592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768b200 con 0x7fc12803c040 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.708+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94719185562592] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.708+0000 7fc13c722800 20 remove_watcher() i=5 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.708+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94719185570256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d4f0 con 0x7fc12803c040 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.709+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94719185570256] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.709+0000 7fc13c722800 20 remove_watcher() i=6 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.709+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94719185571312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56258768d4f0 con 0x7fc12803c040 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94719185571312] v31'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc12c051070 con 0x7fc12803c040 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 20 remove_watcher() i=7 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc12803c040 msgr2=0x7fc12804c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc12803c040 0x7fc12804c580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56258767c720 msgr2=0x56258767ebe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56258767c720 0x56258767ebe0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc1300123d0 msgr2=0x7fc13003b7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc1300123d0 0x7fc13003b7c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562587681130 msgr2=0x562587683520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562587681130 0x562587683520 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc00803d150 msgr2=0x7fc00803f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc00803d150 0x7fc00803f610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc128053460 tx=0x7fc128041000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc137bd1640 1 -- 172.21.15.28:0/387419993 reap_dead start 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 msgr2=0x562587670e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587670e30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc130016cd0 tx=0x7fc1300547c0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 shutdown_connections 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/387419993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587669820 0x562587670e30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 >> 172.21.15.28:0/387419993 conn(0x562587665c40 msgr2=0x562587663e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 shutdown_connections 2024-09-19T11:23:22.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/387419993 wait complete. 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc01003d000 msgr2=0x7fc01003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc01003d000 0x7fc01003f4c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc12800ed30 tx=0x7fc12800ff70 comp rx=0 tx=0).stop 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56258765cdc0 msgr2=0x56258765d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56258765cdc0 0x56258765d1e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x562587642e30 tx=0x7fc1300167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 shutdown_connections 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc01003d000 0x7fc01003f4c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 --2- 172.21.15.28:0/397004440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56258765cdc0 0x56258765d1e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 >> 172.21.15.28:0/397004440 conn(0x562587651170 msgr2=0x562587651550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 shutdown_connections 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.710+0000 7fc13c722800 1 -- 172.21.15.28:0/397004440 wait complete. 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625875470a0 msgr2=0x562587649480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625875470a0 0x562587649480 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56258764b860 msgr2=0x56258764dce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56258764b860 0x56258764dce0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc02003d000 msgr2=0x7fc02003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc02003d000 0x7fc02003f4c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc12c057000 tx=0x7fc12c052000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 msgr2=0x562587641e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587641e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc128000f60 tx=0x7fc128000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 shutdown_connections 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56258764b860 0x56258764dce0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc02003d000 0x7fc02003f4c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625875470a0 0x562587649480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 --2- 172.21.15.28:0/3635469873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562587546bb0 0x562587641e30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 >> 172.21.15.28:0/3635469873 conn(0x5625875421c0 msgr2=0x562587633620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 shutdown_connections 2024-09-19T11:23:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.711+0000 7fc13c722800 1 -- 172.21.15.28:0/3635469873 wait complete. 2024-09-19T11:23:22.722 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:22.722 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-09-19T11:23:22.722 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-09-19T11:23:22.722 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-09-19T11:23:22.772 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.772 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0f928800 1 Processor -- start 2024-09-19T11:23:22.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0f928800 1 -- start start 2024-09-19T11:23:22.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0f928800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d81b38ed50 0x55d81b38f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0f928800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d81b390250 con 0x55d81b38f690 2024-09-19T11:23:22.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0f928800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55d81b21b7d0 con 0x55d81b38ed50 2024-09-19T11:23:22.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0b66f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d81b38ed50 0x55d81b38f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0ae6e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55d81b38f690 0x55d81b38fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33064/0 (socket says 172.21.15.28:33064) 2024-09-19T11:23:22.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0b66f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d81b38ed50 0x55d81b38f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59280/0 (socket says 172.21.15.28:59280) 2024-09-19T11:23:22.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.772+0000 7efd0ae6e640 1 -- 172.21.15.28:0/770273185 learned_addr learned my addr 172.21.15.28:0/770273185 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:22.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.773+0000 7efd0b66f640 1 -- 172.21.15.28:0/770273185 >> v1:172.21.15.22:6789/0 conn(0x55d81b38f690 legacy=0x55d81b38fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.773+0000 7efd0a66d640 1 -- 172.21.15.28:0/770273185 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948214582 0 0) 0x55d81b390250 con 0x55d81b38f690 2024-09-19T11:23:22.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.773+0000 7efd0b66f640 1 -- 172.21.15.28:0/770273185 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b390420 con 0x55d81b38ed50 2024-09-19T11:23:22.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0b66f640 1 --2- 172.21.15.28:0/770273185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b38f150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00a470 tx=0x7efcfc045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5e38c7604d4d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0a66d640 1 -- 172.21.15.28:0/770273185 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcfc053570 con 0x55d81b38ed50 2024-09-19T11:23:22.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0a66d640 1 -- 172.21.15.28:0/770273185 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcfc058070 con 0x55d81b38ed50 2024-09-19T11:23:22.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0a66d640 1 -- 172.21.15.28:0/770273185 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcfc053880 con 0x55d81b38ed50 2024-09-19T11:23:22.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0f928800 1 -- 172.21.15.28:0/770273185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 msgr2=0x55d81b38f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0f928800 1 --2- 172.21.15.28:0/770273185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b38f150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00a470 tx=0x7efcfc045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:22.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0f928800 1 -- 172.21.15.28:0/770273185 shutdown_connections 2024-09-19T11:23:22.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0f928800 1 --2- 172.21.15.28:0/770273185 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b38f150 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0f928800 1 -- 172.21.15.28:0/770273185 >> 172.21.15.28:0/770273185 conn(0x55d81b38aa80 msgr2=0x55d81b38cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0f928800 1 -- 172.21.15.28:0/770273185 shutdown_connections 2024-09-19T11:23:22.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.775+0000 7efd0f928800 1 -- 172.21.15.28:0/770273185 wait complete. 2024-09-19T11:23:22.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0f928800 1 Processor -- start 2024-09-19T11:23:22.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0f928800 1 -- start start 2024-09-19T11:23:22.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0f928800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b482450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0f928800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d81b228dd0 con 0x55d81b38ed50 2024-09-19T11:23:22.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0b66f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b482450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0b66f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b482450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59294/0 (socket says 172.21.15.28:59294) 2024-09-19T11:23:22.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0b66f640 1 -- 172.21.15.28:0/3215750190 learned_addr learned my addr 172.21.15.28:0/3215750190 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.791+0000 7efd0b66f640 1 -- 172.21.15.28:0/3215750190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b482a40 con 0x55d81b38ed50 2024-09-19T11:23:22.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.792+0000 7efd0b66f640 1 --2- 172.21.15.28:0/3215750190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b482450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcfc005870 tx=0x7efcfc051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc54f11928077fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.792+0000 7efbf960a640 1 -- 172.21.15.28:0/3215750190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcfc05c580 con 0x55d81b38ed50 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.792+0000 7efbf960a640 1 -- 172.21.15.28:0/3215750190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcfc063030 con 0x55d81b38ed50 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.792+0000 7efbf960a640 1 -- 172.21.15.28:0/3215750190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcfc05c850 con 0x55d81b38ed50 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 -- 172.21.15.28:0/3215750190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 msgr2=0x55d81b482450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 --2- 172.21.15.28:0/3215750190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b482450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcfc005870 tx=0x7efcfc051000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 -- 172.21.15.28:0/3215750190 shutdown_connections 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 --2- 172.21.15.28:0/3215750190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b482450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 -- 172.21.15.28:0/3215750190 >> 172.21.15.28:0/3215750190 conn(0x55d81b38aa80 msgr2=0x55d81b47ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 -- 172.21.15.28:0/3215750190 shutdown_connections 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 -- 172.21.15.28:0/3215750190 wait complete. 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 Processor -- start 2024-09-19T11:23:22.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 -- start start 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b489dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0f928800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d81b3909b0 con 0x55d81b38ed50 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0b66f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b489dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0b66f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b489dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59298/0 (socket says 172.21.15.28:59298) 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.793+0000 7efd0b66f640 1 -- 172.21.15.28:0/1841759496 learned_addr learned my addr 172.21.15.28:0/1841759496 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.794+0000 7efd0b66f640 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b48a330 con 0x55d81b38ed50 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.794+0000 7efd0b66f640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b489dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efcfc025c20 tx=0x7efcfc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.794+0000 7efbf7e07640 1 -- 172.21.15.28:0/1841759496 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcfc05cb70 con 0x55d81b38ed50 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.794+0000 7efbf7e07640 1 -- 172.21.15.28:0/1841759496 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcfc05cd10 con 0x55d81b38ed50 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.794+0000 7efbf7e07640 1 -- 172.21.15.28:0/1841759496 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcfc053440 con 0x55d81b38ed50 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.794+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d81b48a5c0 con 0x55d81b38ed50 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.794+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81b48b1d0 con 0x55d81b38ed50 2024-09-19T11:23:22.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efbf7e07640 1 -- 172.21.15.28:0/1841759496 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7efcfc0535e0 con 0x55d81b38ed50 2024-09-19T11:23:22.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efbf7e07640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbf003cfb0 0x7efbf003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efbf7e07640 1 -- 172.21.15.28:0/1841759496 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7efcfc070070 con 0x55d81b38ed50 2024-09-19T11:23:22.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efd0ae6e640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbf003cfb0 0x7efbf003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b38f690 0x55d81b491420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b491960 con 0x55d81b38f690 2024-09-19T11:23:22.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efd0be70640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b38f690 0x55d81b491420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.795+0000 7efd0ae6e640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbf003cfb0 0x7efbf003f470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd00000c00 tx=0x7efd000013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.796+0000 7efd0be70640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b38f690 0x55d81b491420 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.796+0000 7efd0be70640 1 -- 172.21.15.28:0/1841759496 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efd0400a4d0 con 0x55d81b38f690 2024-09-19T11:23:22.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.796+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b491d00 con 0x55d81b38f690 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.796+0000 7efd0be70640 1 -- 172.21.15.28:0/1841759496 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7efd0400a4d0 con 0x55d81b38f690 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.796+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4937c0 0x55d81b495c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.796+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4963c0 con 0x55d81b4937c0 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.796+0000 7efd0b66f640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4937c0 0x55d81b495c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.797+0000 7efd0b66f640 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4937c0 0x55d81b495c30 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.797+0000 7efd0b66f640 1 -- 172.21.15.28:0/1841759496 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7efcfc07a020 con 0x55d81b4937c0 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.797+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b492550 con 0x55d81b38f690 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.797+0000 7efd0be70640 1 -- 172.21.15.28:0/1841759496 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7efd0400a4d0 con 0x55d81b38f690 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.797+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4928f0 con 0x55d81b4937c0 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0b66f640 1 -- 172.21.15.28:0/1841759496 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7efcfc07a020 con 0x55d81b4937c0 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0f928800 1 Processor -- start 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0f928800 1 -- start start 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0f928800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4a4d50 0x55d81b4a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0f928800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d81b2874a0 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0be70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4a4d50 0x55d81b4a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0be70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4a4d50 0x55d81b4a5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59302/0 (socket says 172.21.15.28:59302) 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.798+0000 7efd0be70640 1 -- 172.21.15.28:0/1819369796 learned_addr learned my addr 172.21.15.28:0/1819369796 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.799+0000 7efd0be70640 1 -- 172.21.15.28:0/1819369796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b4a57b0 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.799+0000 7efd0be70640 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4a4d50 0x55d81b4a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55d81b48add0 tx=0x7efd040167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.799+0000 7efbf5602640 1 -- 172.21.15.28:0/1819369796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd04013640 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.799+0000 7efbf5602640 1 -- 172.21.15.28:0/1819369796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd040137e0 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.799+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d81b4a5fe0 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.799+0000 7efbf5602640 1 -- 172.21.15.28:0/1819369796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd04013ae0 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.799+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81b4a6580 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.800+0000 7efbf5602640 1 -- 172.21.15.28:0/1819369796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7efd04011040 con 0x55d81b4a4d50 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.800+0000 7efbf5602640 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbe403d070 0x7efbe403f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.800+0000 7efbf5602640 1 -- 172.21.15.28:0/1819369796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7efd0400e040 con 0x55d81b4a4d50 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.800+0000 7efd0b66f640 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbe403d070 0x7efbe403f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.800+0000 7efd0f928800 1 Processor -- start 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.800+0000 7efd0b66f640 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbe403d070 0x7efbe403f530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55d81b48bea0 tx=0x7efcfc073520 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.800+0000 7efd0f928800 1 -- start start 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efd0f928800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efd0f928800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d81b482e10 con 0x55d81b4b1830 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efd0be70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efd0be70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59308/0 (socket says 172.21.15.28:59308) 2024-09-19T11:23:22.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efd0be70640 1 -- 172.21.15.28:0/735381421 learned_addr learned my addr 172.21.15.28:0/735381421 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efd0be70640 1 -- 172.21.15.28:0/735381421 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b4b2220 con 0x55d81b4b1830 2024-09-19T11:23:22.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efd0be70640 1 --2- 172.21.15.28:0/735381421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b1c30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efd04054860 tx=0x7efd04020ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbdfa91a6249a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efbeeffd640 1 -- 172.21.15.28:0/735381421 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd04044030 con 0x55d81b4b1830 2024-09-19T11:23:22.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efbeeffd640 1 -- 172.21.15.28:0/735381421 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd04021960 con 0x55d81b4b1830 2024-09-19T11:23:22.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.801+0000 7efbeeffd640 1 -- 172.21.15.28:0/735381421 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd0404de60 con 0x55d81b4b1830 2024-09-19T11:23:22.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 -- 172.21.15.28:0/735381421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 msgr2=0x55d81b4b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 --2- 172.21.15.28:0/735381421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b1c30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efd04054860 tx=0x7efd04020ad0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 -- 172.21.15.28:0/735381421 shutdown_connections 2024-09-19T11:23:22.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 --2- 172.21.15.28:0/735381421 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b1c30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 -- 172.21.15.28:0/735381421 >> 172.21.15.28:0/735381421 conn(0x55d81b4adc30 msgr2=0x55d81b4abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 -- 172.21.15.28:0/735381421 shutdown_connections 2024-09-19T11:23:22.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 -- 172.21.15.28:0/735381421 wait complete. 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 Processor -- start 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 -- start start 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0f928800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d81b4b2840 con 0x55d81b4b1830 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0be70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0be70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59318/0 (socket says 172.21.15.28:59318) 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.802+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 learned_addr learned my addr 172.21.15.28:0/3546921520 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.803+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d81b4b9380 con 0x55d81b4b1830 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.803+0000 7efd0be70640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b8e20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efd04020510 tx=0x7efd040547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.803+0000 7efbed7fa640 1 -- 172.21.15.28:0/3546921520 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd04054a30 con 0x55d81b4b1830 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.803+0000 7efbed7fa640 1 -- 172.21.15.28:0/3546921520 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd040143c0 con 0x55d81b4b1830 2024-09-19T11:23:22.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.803+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d81b4b9610 con 0x55d81b4b1830 2024-09-19T11:23:22.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.803+0000 7efbed7fa640 1 -- 172.21.15.28:0/3546921520 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd04020bd0 con 0x55d81b4b1830 2024-09-19T11:23:22.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.803+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d81b4ba1a0 con 0x55d81b4b1830 2024-09-19T11:23:22.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.804+0000 7efbed7fa640 1 -- 172.21.15.28:0/3546921520 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7efd04020d70 con 0x55d81b4b1830 2024-09-19T11:23:22.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.804+0000 7efbed7fa640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbd403d000 0x7efbd403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.804+0000 7efbed7fa640 1 -- 172.21.15.28:0/3546921520 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7efd0404b070 con 0x55d81b4b1830 2024-09-19T11:23:22.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.804+0000 7efd0b66f640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbd403d000 0x7efbd403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.804+0000 7efd0b66f640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbd403d000 0x7efbd403f4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efcfc05c800 tx=0x7efcfc08c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b4c4770 0x55d81b4c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4c7380 con 0x55d81b4c4770 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0ae6e640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b4c4770 0x55d81b4c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0ae6e640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b4c4770 0x55d81b4c6c30 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efd0000f5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0f928800 20 realm 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.805+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4c3ac0 con 0x55d81b4c4770 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efd0000f5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4c7e70 con 0x55d81b4c4770 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efd0000f5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4c8740 con 0x55d81b4c4770 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7efd0000f5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4c9160 0x55d81b4cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cbcc0 con 0x55d81b4c9160 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0be70640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4c9160 0x55d81b4cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.806+0000 7efd0be70640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4c9160 0x55d81b4cb550 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0f928800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0f928800 20 searching for the correct realm 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0be70640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7efd040123d0 0x7efd0403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efd040123d0 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0b66f640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7efd040123d0 0x7efd0403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.807+0000 7efd0b66f640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7efd040123d0 0x7efd0403b350 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.808+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.808+0000 7efd0b66f640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7efcfc090040 0x7efcfc0538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.808+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.808+0000 7efd0ae6e640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7efcfc090040 0x7efcfc0538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.809+0000 7efd0ae6e640 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7efcfc090040 0x7efcfc0538d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.809+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0000f5f0 con 0x7efcfc090040 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.809+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.809+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0000f5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.809+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.809+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x55d81b4c4770 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.809+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x7efcfc090040 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x7efcfc090040 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efd040123d0 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.810+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x55d81b4c4770 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x7efcfc090040 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x55d81b4c4770 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efd040123d0 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.811+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x7efcfc090040 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.812+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x55d81b4c4770 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7efd0002b040 con 0x55d81b4c4770 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x7efcfc090040 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.813+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efd040123d0 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.814+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.814+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.814+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0002b040 con 0x55d81b4c4770 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.814+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.814+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.814+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efd040123d0 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c9160 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.815+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efcfc090040 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x55d81b4c4770 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cd5f0 con 0x7efd040123d0 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cc680 con 0x55d81b4c4770 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.816+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4ccf20 con 0x55d81b4c4770 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cda70 con 0x55d81b4c9160 2024-09-19T11:23:22.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 zone default found 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 4 Realm: () 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 zonegroup default 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cde30 con 0x7efcfc090040 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cfc60 con 0x55d81b4c9160 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.817+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.818+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.818+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d0090 con 0x55d81b4c9160 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.818+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.818+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.818+0000 7efd0f928800 20 started sync module instance, tier type = 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.818+0000 7efd0f928800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.818+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4cf100 con 0x7efd040123d0 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.819+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.819+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94386659325168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4ced60 con 0x7efd040123d0 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.819+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d0ff0 con 0x55d81b4c4770 2024-09-19T11:23:22.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.820+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94386659325168] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.820+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.820+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94386659330880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d1410 con 0x55d81b4c4770 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.820+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d1dd0 con 0x55d81b4c9160 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.821+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94386659330880] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.821+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.821+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94386659332416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d1dd0 con 0x55d81b4c9160 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.821+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d2810 con 0x7efd040123d0 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.822+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94386659332416] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.822+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.822+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94386659334432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d1d40 con 0x7efd040123d0 2024-09-19T11:23:22.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.822+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d36d0 con 0x55d81b4c4770 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.823+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.823+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94386659340832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d3680 con 0x55d81b4c4770 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.823+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d4650 con 0x7efcfc090040 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.824+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94386659334432] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.824+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.824+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94386659344704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d4610 con 0x7efcfc090040 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.824+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d50d0 con 0x7efcfc090040 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.825+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94386659340832] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.825+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94386659344704] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.826+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.826+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94386659339264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d4be0 con 0x7efcfc090040 2024-09-19T11:23:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.826+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d55a0 con 0x7efcfc090040 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94386659339264] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94386659348720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d5510 con 0x7efcfc090040 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 20 add_watcher() i=0 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 20 add_watcher() i=1 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 20 add_watcher() i=2 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 20 add_watcher() i=3 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 20 add_watcher() i=4 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 20 add_watcher() i=5 2024-09-19T11:23:22.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.827+0000 7efd0f928800 20 add_watcher() i=6 2024-09-19T11:23:22.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.828+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94386659348720] v31'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.828+0000 7efd0f928800 20 add_watcher() i=7 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.828+0000 7efd0f928800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.828+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d89a0 con 0x55d81b4c4770 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.828+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.828+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94386659343040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b4d5ef0 con 0x55d81b4c4770 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.829+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94386659343040] v31'64 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.829+0000 7efbc1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.829+0000 7efd0f928800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.829+0000 7efd0f928800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.829+0000 7efd0f928800 5 note: GC not initialized 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.829+0000 7efbc0fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.829+0000 7efd0f928800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 20 Filter name: none 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efb3a7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b50b8e0 con 0x55d81b4c4770 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.830+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b50c1f0 con 0x7efcfc090040 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.831+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.831+0000 7efd0f928800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.831+0000 7efd0f928800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.831+0000 7efd0f928800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.831+0000 7efd0f928800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.831+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~502 in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b511b70 con 0x55d81b4c4770 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~502] v31'4 uv4 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0f928800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0f928800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0f928800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94386659595664 in=690b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b50d2c0 con 0x55d81b4c4770 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94386659340832 notify 133143986179 ret 0) ==== 720+0+0 (crc 0 0 0) 0x7efd00018040 con 0x55d81b4c4770 2024-09-19T11:23:22.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efbeeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986179 cookie 94386659340832 notifier 4341 bl.length()=678 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efbeeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efbeeffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efbeeffd640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7efbd0003ea0 con 0x55d81b4c4770 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(65 notify.4 [notify cookie 94386659595664 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.832+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94386659595664 notify 133143986179 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7efd0001dbf0 con 0x55d81b4c4770 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.833+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b511590 con 0x7efcfc090040 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efd0f928800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:23:22.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efd0f928800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efd0f928800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94386659595664 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d81b50ce10 con 0x55d81b4c9160 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== watch-notify(notify (1) cookie 94386659332416 notify 133143986176 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55d81b4b9610 con 0x55d81b4c9160 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efbee7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986176 cookie 94386659332416 notifier 4341 bl.length()=177 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efbee7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efbee7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efbee7fc640 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7efb40003400 con 0x55d81b4c9160 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.834+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 notify.2 [notify cookie 94386659595664 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.835+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.835+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== watch-notify(notify_complete (2) cookie 94386659595664 notify 133143986176 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d81b4b9380 con 0x55d81b4c9160 2024-09-19T11:23:22.852 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-09-19T11:23:22.831926Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.836+0000 7efbc1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.836+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94386659343040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b50d770 con 0x55d81b4c4770 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.837+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94386659343040] v31'65 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.837+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94386659325168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x7efd040123d0 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.838+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94386659325168] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.838+0000 7efd0f928800 20 remove_watcher() i=0 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.838+0000 7efd0f928800 2 removed watcher, disabling cache 2024-09-19T11:23:22.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.838+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94386659330880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x55d81b4c4770 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.839+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94386659330880] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.839+0000 7efd0f928800 20 remove_watcher() i=1 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.839+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94386659332416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x55d81b4c9160 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.840+0000 7efd0be70640 1 -- 172.21.15.28:0/3546921520 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94386659332416] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0403a070 con 0x55d81b4c9160 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.840+0000 7efd0f928800 20 remove_watcher() i=2 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.840+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94386659334432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x7efd040123d0 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.841+0000 7efd0b66f640 1 -- 172.21.15.28:0/3546921520 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94386659334432] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efcfc07a020 con 0x7efd040123d0 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.841+0000 7efd0f928800 20 remove_watcher() i=3 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.841+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94386659340832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x55d81b4c4770 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.842+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94386659340832] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x55d81b4c4770 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.842+0000 7efd0f928800 20 remove_watcher() i=4 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.842+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94386659344704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x7efcfc090040 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.843+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94386659344704] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.843+0000 7efd0f928800 20 remove_watcher() i=5 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.843+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94386659339264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x7efcfc090040 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.844+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94386659339264] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.844+0000 7efd0f928800 20 remove_watcher() i=6 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.844+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94386659348720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d81b509710 con 0x7efcfc090040 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94386659348720] v31'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efd0003b040 con 0x7efcfc090040 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 20 remove_watcher() i=7 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7efcfc090040 msgr2=0x7efcfc0538d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7efcfc090040 0x7efcfc0538d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b4c4770 msgr2=0x55d81b4c6c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b4c4770 0x55d81b4c6c30 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7efd040123d0 msgr2=0x7efd0403b350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7efd040123d0 0x7efd0403b350 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4c9160 msgr2=0x55d81b4cb550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4c9160 0x55d81b4cb550 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbd403d000 msgr2=0x7efbd403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbd403d000 0x7efbd403f4c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efcfc05c800 tx=0x7efcfc08c000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 msgr2=0x55d81b4b8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 --2- 172.21.15.28:0/3546921520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4b1830 0x55d81b4b8e20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efd04020510 tx=0x7efd040547c0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 reap_dead start 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0ae6e640 1 -- 172.21.15.28:0/3546921520 reap_dead start 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 shutdown_connections 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 >> 172.21.15.28:0/3546921520 conn(0x55d81b4adc30 msgr2=0x55d81b4abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 shutdown_connections 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.845+0000 7efd0f928800 1 -- 172.21.15.28:0/3546921520 wait complete. 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbe403d070 msgr2=0x7efbe403f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbe403d070 0x7efbe403f530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55d81b48bea0 tx=0x7efcfc073520 comp rx=0 tx=0).stop 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4a4d50 msgr2=0x55d81b4a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4a4d50 0x55d81b4a5170 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55d81b48add0 tx=0x7efd040167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 shutdown_connections 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbe403d070 0x7efbe403f530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1819369796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b4a4d50 0x55d81b4a5170 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 >> 172.21.15.28:0/1819369796 conn(0x55d81b499100 msgr2=0x55d81b4994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 shutdown_connections 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1819369796 wait complete. 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b38f690 msgr2=0x55d81b491420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b38f690 0x55d81b491420 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4937c0 msgr2=0x55d81b495c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4937c0 0x55d81b495c30 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbf003cfb0 msgr2=0x7efbf003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbf003cfb0 0x7efbf003f470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efd00000c00 tx=0x7efd000013b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 msgr2=0x55d81b489dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b489dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efcfc025c20 tx=0x7efcfc002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 shutdown_connections 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d81b4937c0 0x55d81b495c30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efbf003cfb0 0x7efbf003f470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d81b38f690 0x55d81b491420 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 --2- 172.21.15.28:0/1841759496 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d81b38ed50 0x55d81b489dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 >> 172.21.15.28:0/1841759496 conn(0x55d81b38aa80 msgr2=0x55d81b47c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 shutdown_connections 2024-09-19T11:23:22.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.846+0000 7efd0f928800 1 -- 172.21.15.28:0/1841759496 wait complete. 2024-09-19T11:23:22.863 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-09-19T11:23:22.831926Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:22.864 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.864 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:22.864 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:22.909 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:22.910 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:22.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f8771e98800 1 Processor -- start 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f8771e98800 1 -- start start 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f8771e98800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563288079aa0 0x56328807f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f8771e98800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563288078d40 con 0x563288078690 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f8771e98800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x563287f047d0 con 0x563288079aa0 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f876f181640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563288079aa0 0x56328807f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f876f982640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x563288078690 0x563288079390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33068/0 (socket says 172.21.15.28:33068) 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f876f181640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563288079aa0 0x56328807f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59324/0 (socket says 172.21.15.28:59324) 2024-09-19T11:23:22.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.909+0000 7f876f982640 1 -- 172.21.15.28:0/2668534813 learned_addr learned my addr 172.21.15.28:0/2668534813 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.910+0000 7f876f181640 1 -- 172.21.15.28:0/2668534813 >> v1:172.21.15.22:6789/0 conn(0x563288078690 legacy=0x563288079390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.910+0000 7f876e980640 1 -- 172.21.15.28:0/2668534813 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3120723048 0 0) 0x563288078d40 con 0x563288078690 2024-09-19T11:23:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.910+0000 7f876f181640 1 -- 172.21.15.28:0/2668534813 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563288079f50 con 0x563288079aa0 2024-09-19T11:23:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f876f181640 1 --2- 172.21.15.28:0/2668534813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288079aa0 0x56328807f640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f876400a470 tx=0x7f8764045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976a7fb6c425f6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f876e980640 1 -- 172.21.15.28:0/2668534813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8764053570 con 0x563288079aa0 2024-09-19T11:23:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f876e980640 1 -- 172.21.15.28:0/2668534813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8764058070 con 0x563288079aa0 2024-09-19T11:23:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f876e980640 1 -- 172.21.15.28:0/2668534813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8764053880 con 0x563288079aa0 2024-09-19T11:23:22.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f8771e98800 1 -- 172.21.15.28:0/2668534813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288079aa0 msgr2=0x56328807f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f8771e98800 1 --2- 172.21.15.28:0/2668534813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288079aa0 0x56328807f640 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f876400a470 tx=0x7f8764045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:22.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f8771e98800 1 -- 172.21.15.28:0/2668534813 shutdown_connections 2024-09-19T11:23:22.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f8771e98800 1 --2- 172.21.15.28:0/2668534813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288079aa0 0x56328807f640 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f8771e98800 1 -- 172.21.15.28:0/2668534813 >> 172.21.15.28:0/2668534813 conn(0x563288073a60 msgr2=0x563288075ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f8771e98800 1 -- 172.21.15.28:0/2668534813 shutdown_connections 2024-09-19T11:23:22.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.912+0000 7f8771e98800 1 -- 172.21.15.28:0/2668534813 wait complete. 2024-09-19T11:23:22.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.927+0000 7f8771e98800 1 Processor -- start 2024-09-19T11:23:22.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.927+0000 7f8771e98800 1 -- start start 2024-09-19T11:23:22.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.928+0000 7f8771e98800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563287fec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.928+0000 7f8771e98800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563287f11dd0 con 0x563288078690 2024-09-19T11:23:22.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.928+0000 7f876f982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563287fec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.928+0000 7f876f982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563287fec7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59334/0 (socket says 172.21.15.28:59334) 2024-09-19T11:23:22.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.928+0000 7f876f982640 1 -- 172.21.15.28:0/4029303146 learned_addr learned my addr 172.21.15.28:0/4029303146 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.928+0000 7f876f982640 1 -- 172.21.15.28:0/4029303146 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563287fecdd0 con 0x563288078690 2024-09-19T11:23:22.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.928+0000 7f876f982640 1 --2- 172.21.15.28:0/4029303146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563287fec7e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8760001ea0 tx=0x7f876000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8564a0650d63830 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f865de0b640 1 -- 172.21.15.28:0/4029303146 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8760008070 con 0x563288078690 2024-09-19T11:23:22.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f865de0b640 1 -- 172.21.15.28:0/4029303146 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876000d500 con 0x563288078690 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f865de0b640 1 -- 172.21.15.28:0/4029303146 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f876000d880 con 0x563288078690 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f8771e98800 1 -- 172.21.15.28:0/4029303146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 msgr2=0x563287fec7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f8771e98800 1 --2- 172.21.15.28:0/4029303146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563287fec7e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8760001ea0 tx=0x7f876000b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f8771e98800 1 -- 172.21.15.28:0/4029303146 shutdown_connections 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f8771e98800 1 --2- 172.21.15.28:0/4029303146 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563287fec7e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f8771e98800 1 -- 172.21.15.28:0/4029303146 >> 172.21.15.28:0/4029303146 conn(0x563288073a60 msgr2=0x5632881639e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f8771e98800 1 -- 172.21.15.28:0/4029303146 shutdown_connections 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.929+0000 7f8771e98800 1 -- 172.21.15.28:0/4029303146 wait complete. 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f8771e98800 1 Processor -- start 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f8771e98800 1 -- start start 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f8771e98800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563288170ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f8771e98800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56328807fd90 con 0x563288078690 2024-09-19T11:23:22.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f876f982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563288170ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f876f982640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563288170ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59344/0 (socket says 172.21.15.28:59344) 2024-09-19T11:23:22.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f876f982640 1 -- 172.21.15.28:0/3886834667 learned_addr learned my addr 172.21.15.28:0/3886834667 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.930+0000 7f876f982640 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563288171200 con 0x563288078690 2024-09-19T11:23:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.931+0000 7f876f982640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563288170ca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87600012f0 tx=0x7f8760000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.931+0000 7f8657fff640 1 -- 172.21.15.28:0/3886834667 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f876000f9d0 con 0x563288078690 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.931+0000 7f8657fff640 1 -- 172.21.15.28:0/3886834667 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f876000fb70 con 0x563288078690 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.931+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563288171490 con 0x563288078690 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.931+0000 7f8657fff640 1 -- 172.21.15.28:0/3886834667 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f876000fe50 con 0x563288078690 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.931+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563288171e90 con 0x563288078690 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.932+0000 7f8657fff640 1 -- 172.21.15.28:0/3886834667 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8760017040 con 0x563288078690 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.932+0000 7f8657fff640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f865803d1f0 0x7f865803f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.932+0000 7f8657fff640 1 -- 172.21.15.28:0/3886834667 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f8760015020 con 0x563288078690 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.932+0000 7f876f181640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f865803d1f0 0x7f865803f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.932+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563288079aa0 0x5632881782f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.932+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563288178830 con 0x563288079aa0 2024-09-19T11:23:22.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.932+0000 7f8770183640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563288079aa0 0x5632881782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f876f181640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f865803d1f0 0x7f865803f6b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8764057000 tx=0x7f8764052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f8770183640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563288079aa0 0x5632881782f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f8770183640 1 -- 172.21.15.28:0/3886834667 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f876800a4d0 con 0x563288079aa0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563288178bd0 con 0x563288079aa0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f8770183640 1 -- 172.21.15.28:0/3886834667 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f876800a4d0 con 0x563288079aa0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56328817a6d0 0x56328817cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56328817d2e0 con 0x56328817a6d0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.933+0000 7f876f982640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56328817a6d0 0x56328817cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.934+0000 7f876f982640 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56328817a6d0 0x56328817cb50 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.934+0000 7f876f982640 1 -- 172.21.15.28:0/3886834667 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8760054040 con 0x56328817a6d0 2024-09-19T11:23:22.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.934+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563288179420 con 0x563288079aa0 2024-09-19T11:23:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.934+0000 7f8770183640 1 -- 172.21.15.28:0/3886834667 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f876800a4d0 con 0x563288079aa0 2024-09-19T11:23:22.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.934+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563288179870 con 0x56328817a6d0 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f876f982640 1 -- 172.21.15.28:0/3886834667 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8760054040 con 0x56328817a6d0 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f8771e98800 1 Processor -- start 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f8771e98800 1 -- start start 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f8771e98800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288169530 0x563288167da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f8771e98800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563287f704a0 con 0x563288169530 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f8770183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288169530 0x563288167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f8770183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288169530 0x563288167da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59356/0 (socket says 172.21.15.28:59356) 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.935+0000 7f8770183640 1 -- 172.21.15.28:0/1214833957 learned_addr learned my addr 172.21.15.28:0/1214833957 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.936+0000 7f8770183640 1 -- 172.21.15.28:0/1214833957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563288169a50 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.936+0000 7f8770183640 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288169530 0x563288167da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x563288171ca0 tx=0x7f87680167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.936+0000 7f86557fa640 1 -- 172.21.15.28:0/1214833957 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768013640 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.936+0000 7f86557fa640 1 -- 172.21.15.28:0/1214833957 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87680137e0 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.936+0000 7f86557fa640 1 -- 172.21.15.28:0/1214833957 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768013ac0 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.936+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632881682e0 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.936+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563288168830 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.937+0000 7f86557fa640 1 -- 172.21.15.28:0/1214833957 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8768011040 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.937+0000 7f86557fa640 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f864803d000 0x7f864803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.937+0000 7f86557fa640 1 -- 172.21.15.28:0/1214833957 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f876800e040 con 0x563288169530 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.937+0000 7f876f982640 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f864803d000 0x7f864803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.937+0000 7f8771e98800 1 Processor -- start 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f876f982640 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f864803d000 0x7f864803f4c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f876000ecb0 tx=0x7f8760001460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8771e98800 1 -- start start 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8771e98800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x56328819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8771e98800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56328816c000 con 0x56328819a8d0 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8770183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x56328819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8770183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x56328819acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59358/0 (socket says 172.21.15.28:59358) 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8770183640 1 -- 172.21.15.28:0/1437918447 learned_addr learned my addr 172.21.15.28:0/1437918447 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8770183640 1 -- 172.21.15.28:0/1437918447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328819b2c0 con 0x56328819a8d0 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f8770183640 1 --2- 172.21.15.28:0/1437918447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x56328819acd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f876800afe0 tx=0x7f87680204b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0aa667b83d6ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f864effd640 1 -- 172.21.15.28:0/1437918447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768044030 con 0x56328819a8d0 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f864effd640 1 -- 172.21.15.28:0/1437918447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768021a90 con 0x56328819a8d0 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.938+0000 7f864effd640 1 -- 172.21.15.28:0/1437918447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768021570 con 0x56328819a8d0 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 -- 172.21.15.28:0/1437918447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 msgr2=0x56328819acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 --2- 172.21.15.28:0/1437918447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x56328819acd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f876800afe0 tx=0x7f87680204b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 -- 172.21.15.28:0/1437918447 shutdown_connections 2024-09-19T11:23:22.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 --2- 172.21.15.28:0/1437918447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x56328819acd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 -- 172.21.15.28:0/1437918447 >> 172.21.15.28:0/1437918447 conn(0x563288196cd0 msgr2=0x563288194f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 -- 172.21.15.28:0/1437918447 shutdown_connections 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 -- 172.21.15.28:0/1437918447 wait complete. 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 Processor -- start 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 -- start start 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x5632881a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.939+0000 7f8771e98800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56328819b8e0 con 0x56328819a8d0 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f8770183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x5632881a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f8770183640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x5632881a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59372/0 (socket says 172.21.15.28:59372) 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 learned_addr learned my addr 172.21.15.28:0/4084481429 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632881a2420 con 0x56328819a8d0 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f8770183640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x5632881a1ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8768020b60 tx=0x7f87680547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f864d7fa640 1 -- 172.21.15.28:0/4084481429 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8768020bd0 con 0x56328819a8d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f864d7fa640 1 -- 172.21.15.28:0/4084481429 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768020d70 con 0x56328819a8d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f864d7fa640 1 -- 172.21.15.28:0/4084481429 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f876801fa90 con 0x56328819a8d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632881a26b0 con 0x56328819a8d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.940+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632881a3240 con 0x56328819a8d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.941+0000 7f864d7fa640 1 -- 172.21.15.28:0/4084481429 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f876801fc30 con 0x56328819a8d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.941+0000 7f864d7fa640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863803d050 0x7f863803f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.941+0000 7f864d7fa640 1 -- 172.21.15.28:0/4084481429 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f876803d060 con 0x56328819a8d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.941+0000 7f876f982640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863803d050 0x7f863803f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.941+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.941+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5632881ad7d0 0x5632881afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.941+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b03e0 con 0x5632881ad7d0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f876f982640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863803d050 0x7f863803f510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f876000bbf0 tx=0x7f876004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f876f181640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5632881ad7d0 0x5632881afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f876f181640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5632881ad7d0 0x5632881afc90 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 20 realm 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b0810 con 0x5632881ad7d0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.942+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b10e0 con 0x5632881ad7d0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8771e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b1480 con 0x5632881ad7d0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8771e98800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5632881b21e0 0x5632881b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b4b10 con 0x5632881b21e0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8770183640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5632881b21e0 0x5632881b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.943+0000 7f8770183640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5632881b21e0 0x5632881b45d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f8771e98800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f8771e98800 20 searching for the correct realm 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x5632881b21e0 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f8770183640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f876803c3f0 0x7f8768019000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x7f876803c3f0 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f876f982640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f876803c3f0 0x7f8768019000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.944+0000 7f876f982640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f876803c3f0 0x7f8768019000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.945+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.945+0000 7f876f982640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f876003c070 0x7f87600542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:22.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.945+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x7f876003c070 2024-09-19T11:23:22.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.945+0000 7f876f181640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f876003c070 0x7f87600542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:22.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.945+0000 7f876f181640 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f876003c070 0x7f87600542d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:22.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.945+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.945+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x5632881ad7d0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x5632881ad7d0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x7f876003c070 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x7f876003c070 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.946+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x7f876803c3f0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x5632881b21e0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x5632881ad7d0 2024-09-19T11:23:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x7f876003c070 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.947+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x5632881ad7d0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6480 con 0x7f876803c3f0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881b21e0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881b21e0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.948+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876003c070 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881b21e0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881b21e0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881ad7d0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881ad7d0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.949+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876003c070 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881b21e0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876803c3f0 2024-09-19T11:23:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881ad7d0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.950+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876003c070 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881b21e0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876003c070 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876803c3f0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.951+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881b21e0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876003c070 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x5632881ad7d0 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f8760061060 con 0x7f876803c3f0 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.952+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b5580 con 0x5632881ad7d0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b5e20 con 0x5632881ad7d0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b51c0 con 0x5632881b21e0 2024-09-19T11:23:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 zone default found 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 4 Realm: () 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 zonegroup default 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b6e00 con 0x7f876003c070 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.953+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b88f0 con 0x5632881b21e0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881b8dd0 con 0x5632881b21e0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 20 started sync module instance, tier type = 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.954+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b7c30 con 0x7f876803c3f0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.955+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.955+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94775031854976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b9230 con 0x7f876803c3f0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.955+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b9a80 con 0x5632881ad7d0 2024-09-19T11:23:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.956+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94775031854976] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.957+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.957+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94775031862736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b9ea0 con 0x5632881ad7d0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.957+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881ba6f0 con 0x5632881b21e0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.958+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94775031862736] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.958+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.958+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94775031864272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881ba600 con 0x5632881b21e0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.958+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bb180 con 0x7f876803c3f0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.959+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94775031864272] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.959+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.959+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94775031868496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bb570 con 0x7f876803c3f0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.959+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bbfc0 con 0x5632881ad7d0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.960+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94775031868496] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.960+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.960+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94775031872176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bbf80 con 0x5632881ad7d0 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.960+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bca70 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.961+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.961+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94775031874944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bca70 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.961+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94775031872176] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.961+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bd9f0 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.963+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94775031874944] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.963+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.963+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94775031878880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bd9b0 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.963+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bde90 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94775031878880] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94775031865424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bde50 con 0x7f876003c070 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 20 add_watcher() i=0 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 20 add_watcher() i=1 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 20 add_watcher() i=2 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 20 add_watcher() i=3 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 20 add_watcher() i=4 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 20 add_watcher() i=5 2024-09-19T11:23:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.964+0000 7f8771e98800 20 add_watcher() i=6 2024-09-19T11:23:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.965+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94775031865424] v31'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8764051070 con 0x7f876003c070 2024-09-19T11:23:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.965+0000 7f8771e98800 20 add_watcher() i=7 2024-09-19T11:23:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.965+0000 7f8771e98800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.965+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881c12b0 con 0x5632881ad7d0 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.965+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.965+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94775031877136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881be7a0 con 0x5632881ad7d0 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.966+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94775031877136] v31'66 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8764051070 con 0x5632881ad7d0 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.966+0000 7f8625fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.966+0000 7f8771e98800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.966+0000 7f8771e98800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.966+0000 7f8771e98800 5 note: GC not initialized 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f8624fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f8771e98800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f8771e98800 20 Filter name: none 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f859e7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f8771e98800 20 rados->read ofs=0 len=0 2024-09-19T11:23:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5632881f3cd0 con 0x5632881ad7d0 2024-09-19T11:23:22.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7f876407f040 con 0x5632881ad7d0 2024-09-19T11:23:22.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.967+0000 7f8771e98800 20 rados_obj.operate() r=0 bl.length=502 2024-09-19T11:23:22.980 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-09-19T11:23:22.831926Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.968+0000 7f8625fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.968+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94775031877136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881f27e0 con 0x5632881ad7d0 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.969+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94775031877136] v31'67 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f876407f040 con 0x5632881ad7d0 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.969+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94775031854976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bce10 con 0x7f876803c3f0 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.970+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94775031854976] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.971+0000 7f8771e98800 20 remove_watcher() i=0 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.971+0000 7f8771e98800 2 removed watcher, disabling cache 2024-09-19T11:23:22.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.971+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94775031862736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b7780 con 0x5632881ad7d0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.972+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94775031862736] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f876407f040 con 0x5632881ad7d0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.972+0000 7f8771e98800 20 remove_watcher() i=1 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.972+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94775031864272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b7780 con 0x5632881b21e0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.973+0000 7f8770183640 1 -- 172.21.15.28:0/4084481429 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94775031864272] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8768053020 con 0x5632881b21e0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.973+0000 7f8771e98800 20 remove_watcher() i=2 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.973+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94775031868496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b9bd0 con 0x7f876803c3f0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.974+0000 7f876f982640 1 -- 172.21.15.28:0/4084481429 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94775031868496] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8760054040 con 0x7f876803c3f0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.974+0000 7f8771e98800 20 remove_watcher() i=3 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.974+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94775031872176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b7780 con 0x5632881ad7d0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.975+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94775031872176] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f876407f040 con 0x5632881ad7d0 2024-09-19T11:23:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.975+0000 7f8771e98800 20 remove_watcher() i=4 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.975+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94775031874944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bac50 con 0x7f876003c070 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.976+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94775031874944] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f876407f040 con 0x7f876003c070 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.976+0000 7f8771e98800 20 remove_watcher() i=5 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.976+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94775031878880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881b9bd0 con 0x7f876003c070 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.977+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94775031878880] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f876407f040 con 0x7f876003c070 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.977+0000 7f8771e98800 20 remove_watcher() i=6 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.977+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94775031865424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5632881bbab0 con 0x7f876003c070 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94775031865424] v31'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f876407f040 con 0x7f876003c070 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 20 remove_watcher() i=7 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f876003c070 msgr2=0x7f87600542d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f876003c070 0x7f87600542d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5632881ad7d0 msgr2=0x5632881afc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5632881ad7d0 0x5632881afc90 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f876803c3f0 msgr2=0x7f8768019000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f876803c3f0 0x7f8768019000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5632881b21e0 msgr2=0x5632881b45d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5632881b21e0 0x5632881b45d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863803d050 msgr2=0x7f863803f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f863803d050 0x7f863803f510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f876000bbf0 tx=0x7f876004f000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 msgr2=0x5632881a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/4084481429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56328819a8d0 0x5632881a1ec0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8768020b60 tx=0x7f87680547f0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 reap_dead start 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f876f181640 1 -- 172.21.15.28:0/4084481429 reap_dead start 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 shutdown_connections 2024-09-19T11:23:22.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 >> 172.21.15.28:0/4084481429 conn(0x563288196cd0 msgr2=0x563288194ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 shutdown_connections 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/4084481429 wait complete. 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f864803d000 msgr2=0x7f864803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f864803d000 0x7f864803f4c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f876000ecb0 tx=0x7f8760001460 comp rx=0 tx=0).stop 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288169530 msgr2=0x563288167da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288169530 0x563288167da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x563288171ca0 tx=0x7f87680167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 shutdown_connections 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f864803d000 0x7f864803f4c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 --2- 172.21.15.28:0/1214833957 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288169530 0x563288167da0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 >> 172.21.15.28:0/1214833957 conn(0x563288180020 msgr2=0x563288180400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 shutdown_connections 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.979+0000 7f8771e98800 1 -- 172.21.15.28:0/1214833957 wait complete. 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563288079aa0 msgr2=0x5632881782f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563288079aa0 0x5632881782f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56328817a6d0 msgr2=0x56328817cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56328817a6d0 0x56328817cb50 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f865803d1f0 msgr2=0x7f865803f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f865803d1f0 0x7f865803f6b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8764057000 tx=0x7f8764052000 comp rx=0 tx=0).stop 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 msgr2=0x563288170ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563288170ca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87600012f0 tx=0x7f8760000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 shutdown_connections 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56328817a6d0 0x56328817cb50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f865803d1f0 0x7f865803f6b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563288079aa0 0x5632881782f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 --2- 172.21.15.28:0/3886834667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563288078690 0x563288170ca0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 >> 172.21.15.28:0/3886834667 conn(0x563288073a60 msgr2=0x5632881621d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 shutdown_connections 2024-09-19T11:23:22.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:22.980+0000 7f8771e98800 1 -- 172.21.15.28:0/3886834667 wait complete. 2024-09-19T11:23:22.992 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-09-19T11:23:22.831926Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:22.992 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-09-19T11:23:22.992 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-09-19T11:23:22.992 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-09-19T11:23:23.040 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.040 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d23013800 1 Processor -- start 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d23013800 1 -- start start 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d23013800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560004cc5bb0 0x560004cc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d23013800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560004cc72c0 con 0x560004cc65e0 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d23013800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x560004b517d0 con 0x560004cc5bb0 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d1e748640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560004cc5bb0 0x560004cc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d1df47640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x560004cc65e0 0x560004cc6a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33082/0 (socket says 172.21.15.28:33082) 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d1e748640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560004cc5bb0 0x560004cc5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59380/0 (socket says 172.21.15.28:59380) 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d1df47640 1 -- 172.21.15.28:0/2641326051 learned_addr learned my addr 172.21.15.28:0/2641326051 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:23.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.040+0000 7f0d1e748640 1 -- 172.21.15.28:0/2641326051 >> v1:172.21.15.22:6789/0 conn(0x560004cc65e0 legacy=0x560004cc6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.041+0000 7f0d1e748640 1 -- 172.21.15.28:0/2641326051 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560004ccd760 con 0x560004cc5bb0 2024-09-19T11:23:23.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.042+0000 7f0d1e748640 1 --2- 172.21.15.28:0/2641326051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc5fb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000a470 tx=0x7f0d10045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970a5c45f63c3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.042+0000 7f0d1d746640 1 -- 172.21.15.28:0/2641326051 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d10053570 con 0x560004cc5bb0 2024-09-19T11:23:23.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.042+0000 7f0d1d746640 1 -- 172.21.15.28:0/2641326051 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d10058070 con 0x560004cc5bb0 2024-09-19T11:23:23.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.042+0000 7f0d1d746640 1 -- 172.21.15.28:0/2641326051 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d10053880 con 0x560004cc5bb0 2024-09-19T11:23:23.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.043+0000 7f0d23013800 1 -- 172.21.15.28:0/2641326051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 msgr2=0x560004cc5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.043+0000 7f0d23013800 1 --2- 172.21.15.28:0/2641326051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc5fb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000a470 tx=0x7f0d10045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.043+0000 7f0d23013800 1 -- 172.21.15.28:0/2641326051 shutdown_connections 2024-09-19T11:23:23.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.043+0000 7f0d23013800 1 --2- 172.21.15.28:0/2641326051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc5fb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.043+0000 7f0d23013800 1 -- 172.21.15.28:0/2641326051 >> 172.21.15.28:0/2641326051 conn(0x560004cc01d0 msgr2=0x560004cc2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.043+0000 7f0d23013800 1 -- 172.21.15.28:0/2641326051 shutdown_connections 2024-09-19T11:23:23.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.043+0000 7f0d23013800 1 -- 172.21.15.28:0/2641326051 wait complete. 2024-09-19T11:23:23.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.058+0000 7f0d23013800 1 Processor -- start 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.058+0000 7f0d23013800 1 -- start start 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.058+0000 7f0d23013800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.058+0000 7f0d23013800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560004b5edd0 con 0x560004cc5bb0 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0d1e748640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0d1e748640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59384/0 (socket says 172.21.15.28:59384) 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0d1e748640 1 -- 172.21.15.28:0/3695230742 learned_addr learned my addr 172.21.15.28:0/3695230742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0d1e748640 1 -- 172.21.15.28:0/3695230742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560004db8f80 con 0x560004cc5bb0 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0d1e748640 1 --2- 172.21.15.28:0/3695230742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d10025c20 tx=0x7f0d10051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdda698fa52a038f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0c0c608640 1 -- 172.21.15.28:0/3695230742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d10053ba0 con 0x560004cc5bb0 2024-09-19T11:23:23.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0c0c608640 1 -- 172.21.15.28:0/3695230742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d10063030 con 0x560004cc5bb0 2024-09-19T11:23:23.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.059+0000 7f0c0c608640 1 -- 172.21.15.28:0/3695230742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d10053440 con 0x560004cc5bb0 2024-09-19T11:23:23.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 -- 172.21.15.28:0/3695230742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 msgr2=0x560004cc4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 --2- 172.21.15.28:0/3695230742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc4d90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0d10025c20 tx=0x7f0d10051000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 -- 172.21.15.28:0/3695230742 shutdown_connections 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 --2- 172.21.15.28:0/3695230742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004cc4d90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 -- 172.21.15.28:0/3695230742 >> 172.21.15.28:0/3695230742 conn(0x560004cc01d0 msgr2=0x560004db0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 -- 172.21.15.28:0/3695230742 shutdown_connections 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 -- 172.21.15.28:0/3695230742 wait complete. 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 Processor -- start 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.060+0000 7f0d23013800 1 -- start start 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.061+0000 7f0d23013800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004db6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.061+0000 7f0d23013800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560004ccca50 con 0x560004cc5bb0 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.061+0000 7f0d1e748640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004db6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.061+0000 7f0d1e748640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004db6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59388/0 (socket says 172.21.15.28:59388) 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.061+0000 7f0d1e748640 1 -- 172.21.15.28:0/1337137954 learned_addr learned my addr 172.21.15.28:0/1337137954 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.061+0000 7f0d1e748640 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560004db6bd0 con 0x560004cc5bb0 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.061+0000 7f0d1e748640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004db6670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0d10045f50 tx=0x7f0d10002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.062+0000 7f0c0ae05640 1 -- 172.21.15.28:0/1337137954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d10078070 con 0x560004cc5bb0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.062+0000 7f0c0ae05640 1 -- 172.21.15.28:0/1337137954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d10053570 con 0x560004cc5bb0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.062+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560004db4f80 con 0x560004cc5bb0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.062+0000 7f0c0ae05640 1 -- 172.21.15.28:0/1337137954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d10053850 con 0x560004cc5bb0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.062+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560004db55a0 con 0x560004cc5bb0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.062+0000 7f0c0ae05640 1 -- 172.21.15.28:0/1337137954 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0d1007b070 con 0x560004cc5bb0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0c0ae05640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c0403d000 0x7f0c0403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0c0ae05640 1 -- 172.21.15.28:0/1337137954 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0d10070070 con 0x560004cc5bb0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0d1df47640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c0403d000 0x7f0c0403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004cc65e0 0x560004dc7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dc7890 con 0x560004cc65e0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004cc65e0 0x560004dc7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0d1df47640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c0403d000 0x7f0c0403f4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d08001d70 tx=0x7f0d080012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.063+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004cc65e0 0x560004dc7350 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.064+0000 7f0d1ef49640 1 -- 172.21.15.28:0/1337137954 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d1800a4d0 con 0x560004cc65e0 2024-09-19T11:23:23.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.064+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dc8150 con 0x560004cc65e0 2024-09-19T11:23:23.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.064+0000 7f0d1ef49640 1 -- 172.21.15.28:0/1337137954 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d1800a4d0 con 0x560004cc65e0 2024-09-19T11:23:23.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.064+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dc9a60 0x560004dcbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.064+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dcc690 con 0x560004dc9a60 2024-09-19T11:23:23.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.064+0000 7f0d1e748640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dc9a60 0x560004dcbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.064+0000 7f0d1e748640 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dc9a60 0x560004dcbf00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.065+0000 7f0d1e748640 1 -- 172.21.15.28:0/1337137954 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0d1007a030 con 0x560004dc9a60 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.065+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dc89a0 con 0x560004cc65e0 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.065+0000 7f0d1ef49640 1 -- 172.21.15.28:0/1337137954 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d1800a4d0 con 0x560004cc65e0 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.065+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dc8d40 con 0x560004dc9a60 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.065+0000 7f0d1e748640 1 -- 172.21.15.28:0/1337137954 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0d1007a030 con 0x560004dc9a60 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d23013800 1 Processor -- start 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d23013800 1 -- start start 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d23013800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004ddaf40 0x560004ddb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d23013800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560004bbd4a0 con 0x560004ddaf40 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d1ef49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004ddaf40 0x560004ddb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d1ef49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004ddaf40 0x560004ddb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59402/0 (socket says 172.21.15.28:59402) 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d1ef49640 1 -- 172.21.15.28:0/2562460159 learned_addr learned my addr 172.21.15.28:0/2562460159 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.066+0000 7f0d1ef49640 1 -- 172.21.15.28:0/2562460159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560004ddb9a0 con 0x560004ddaf40 2024-09-19T11:23:23.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.067+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004ddaf40 0x560004ddb360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x560004db57c0 tx=0x7f0d180167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.067+0000 7f0c03fff640 1 -- 172.21.15.28:0/2562460159 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d18013640 con 0x560004ddaf40 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.067+0000 7f0c03fff640 1 -- 172.21.15.28:0/2562460159 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180137e0 con 0x560004ddaf40 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.067+0000 7f0c03fff640 1 -- 172.21.15.28:0/2562460159 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d18013ae0 con 0x560004ddaf40 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.067+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560004ddc1d0 con 0x560004ddaf40 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.067+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560004ddc770 con 0x560004ddaf40 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0c03fff640 1 -- 172.21.15.28:0/2562460159 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0d18011040 con 0x560004ddaf40 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0c03fff640 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0bf803d000 0x7f0bf803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0c03fff640 1 -- 172.21.15.28:0/2562460159 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0d1800e040 con 0x560004ddaf40 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0d1e748640 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0bf803d000 0x7f0bf803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0d23013800 1 Processor -- start 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0d1e748640 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0bf803d000 0x7f0bf803f4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x560004db5c30 tx=0x7f0d1005e4a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0d23013800 1 -- start start 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0d23013800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004de7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.068+0000 7f0d23013800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560004db9200 con 0x560004de79c0 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0d1ef49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004de7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0d1ef49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004de7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59418/0 (socket says 172.21.15.28:59418) 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0d1ef49640 1 -- 172.21.15.28:0/1115615909 learned_addr learned my addr 172.21.15.28:0/1115615909 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0d1ef49640 1 -- 172.21.15.28:0/1115615909 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560004de83b0 con 0x560004de79c0 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/1115615909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004de7dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d18016cd0 tx=0x7f0d18020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3effd3681f2259a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0c01ffb640 1 -- 172.21.15.28:0/1115615909 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d18044030 con 0x560004de79c0 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0c01ffb640 1 -- 172.21.15.28:0/1115615909 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1801fdb0 con 0x560004de79c0 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0c01ffb640 1 -- 172.21.15.28:0/1115615909 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d180561a0 con 0x560004de79c0 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0d23013800 1 -- 172.21.15.28:0/1115615909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 msgr2=0x560004de7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.069+0000 7f0d23013800 1 --2- 172.21.15.28:0/1115615909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004de7dc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d18016cd0 tx=0x7f0d18020460 comp rx=0 tx=0).stop 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 -- 172.21.15.28:0/1115615909 shutdown_connections 2024-09-19T11:23:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 --2- 172.21.15.28:0/1115615909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004de7dc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 -- 172.21.15.28:0/1115615909 >> 172.21.15.28:0/1115615909 conn(0x560004de3dc0 msgr2=0x560004de2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 -- 172.21.15.28:0/1115615909 shutdown_connections 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 -- 172.21.15.28:0/1115615909 wait complete. 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 Processor -- start 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 -- start start 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004deef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d23013800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560004de89d0 con 0x560004de79c0 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d1ef49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004deef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d1ef49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004deef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59422/0 (socket says 172.21.15.28:59422) 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 learned_addr learned my addr 172.21.15.28:0/212673691 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.070+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560004def4d0 con 0x560004de79c0 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.071+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004deef70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0d18016cd0 tx=0x7f0d18054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.071+0000 7f0beffff640 1 -- 172.21.15.28:0/212673691 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d1801f340 con 0x560004de79c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.071+0000 7f0beffff640 1 -- 172.21.15.28:0/212673691 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180532b0 con 0x560004de79c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.071+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560004def760 con 0x560004de79c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.071+0000 7f0beffff640 1 -- 172.21.15.28:0/212673691 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d1801f4e0 con 0x560004de79c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.071+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560004def760 con 0x560004de79c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.072+0000 7f0beffff640 1 -- 172.21.15.28:0/212673691 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0d18054940 con 0x560004de79c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.072+0000 7f0beffff640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0be803d000 0x7f0be803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.072+0000 7f0beffff640 1 -- 172.21.15.28:0/212673691 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0d18044410 con 0x560004de79c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.072+0000 7f0d1e748640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0be803d000 0x7f0be803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.072+0000 7f0d1e748640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0be803d000 0x7f0be803f4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0d10054e60 tx=0x7f0d10087000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.072+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004dfa6c0 0x560004dfcb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dfd2d0 con 0x560004dfa6c0 2024-09-19T11:23:23.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d1df47640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004dfa6c0 0x560004dfcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d1df47640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004dfa6c0 0x560004dfcb80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d08009050 con 0x560004dfa6c0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 20 realm 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004df9a10 con 0x560004dfa6c0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d08009050 con 0x560004dfa6c0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.073+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dfddc0 con 0x560004dfa6c0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d08009050 con 0x560004dfa6c0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d23013800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004dfe690 con 0x560004dfa6c0 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d08009050 con 0x560004dfa6c0 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d23013800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dff0b0 0x560004e014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e01be0 con 0x560004dff0b0 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dff0b0 0x560004e014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.074+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dff0b0 0x560004e014a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d23013800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d23013800 20 searching for the correct realm 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dff0b0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d1ef49640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0d18035ba0 0x7f0d180446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x7f0d18035ba0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d1e748640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0d18035ba0 0x7f0d180446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.075+0000 7f0d1e748640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0d18035ba0 0x7f0d180446f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.076+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.076+0000 7f0d1e748640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0d1007d060 0x7f0d100966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.076+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x7f0d1007d060 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.076+0000 7f0d1df47640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0d1007d060 0x7f0d100966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.076+0000 7f0d1df47640 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0d1007d060 0x7f0d100966d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.076+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08009050 con 0x7f0d1007d060 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.076+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dfa6c0 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08009050 con 0x560004dfa6c0 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dfa6c0 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x7f0d1007d060 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x7f0d1007d060 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.077+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x7f0d18035ba0 2024-09-19T11:23:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dff0b0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dfa6c0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x7f0d1007d060 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.078+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dfa6c0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x7f0d18035ba0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dff0b0 2024-09-19T11:23:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x560004e03510 con 0x560004dff0b0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.079+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d1007d060 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dff0b0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dff0b0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dfa6c0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.080+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dfa6c0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d1007d060 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dff0b0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d18035ba0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.081+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dfa6c0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.082+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.082+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d1007d060 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.082+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.082+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dff0b0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.082+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.082+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d1007d060 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d18035ba0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dff0b0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d1007d060 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.083+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x560004dfa6c0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f0d180630c0 con 0x7f0d18035ba0 2024-09-19T11:23:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e025a0 con 0x560004dfa6c0 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e02df0 con 0x560004dfa6c0 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.084+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e03990 con 0x560004dff0b0 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 zone default found 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 4 Realm: () 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 zonegroup default 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e03d50 con 0x7f0d1007d060 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e06030 con 0x560004dff0b0 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.085+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e063d0 con 0x560004dff0b0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.086+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.086+0000 7f0d23013800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.086+0000 7f0d23013800 20 started sync module instance, tier type = 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.086+0000 7f0d23013800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.086+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e05470 con 0x7f0d18035ba0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.087+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.087+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94558081798080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e067c0 con 0x7f0d18035ba0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.087+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e07010 con 0x560004dfa6c0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.088+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94558081798080] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.088+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.088+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94558081805152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e07430 con 0x560004dfa6c0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.088+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e07c80 con 0x560004dff0b0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.089+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94558081805152] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.090+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.090+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94558081806688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e07b70 con 0x560004dff0b0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.090+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e086f0 con 0x7f0d18035ba0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.091+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94558081806688] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.091+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.091+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94558081810912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e08ae0 con 0x7f0d18035ba0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.091+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e094e0 con 0x560004dfa6c0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.092+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.092+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94558081814512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e094b0 con 0x560004dfa6c0 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.092+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e0a4d0 con 0x7f0d1007d060 2024-09-19T11:23:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.092+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94558081810912] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.093+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94558081814512] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.093+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.093+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94558081818528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e09690 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.093+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e0ad70 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.094+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94558081818528] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.094+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.094+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94558081807808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e0abc0 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.094+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e0b410 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.095+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94558081807808] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94558081820912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e0b200 con 0x7f0d1007d060 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 20 add_watcher() i=0 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 20 add_watcher() i=1 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 20 add_watcher() i=2 2024-09-19T11:23:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 20 add_watcher() i=3 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 20 add_watcher() i=4 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 20 add_watcher() i=5 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.096+0000 7f0d23013800 20 add_watcher() i=6 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.097+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94558081820912] v31'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.097+0000 7f0d23013800 20 add_watcher() i=7 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.097+0000 7f0d23013800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.097+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e0e770 con 0x560004dfa6c0 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.097+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.097+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94558081821968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e0bc30 con 0x560004dfa6c0 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.098+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94558081821968] v31'68 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.098+0000 7f0bd4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.098+0000 7f0d23013800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.098+0000 7f0d23013800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.098+0000 7f0d23013800 5 note: GC not initialized 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.098+0000 7f0b4f7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.098+0000 7f0d23013800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 20 Filter name: none 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0b4d7fa640 20 reqs_thread_entry: start 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e416e0 con 0x560004dfa6c0 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 20 rados_obj.operate() r=0 bl.length=502 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:23.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.099+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e47bd0 con 0x560004dfa6c0 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.100+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v31'5 uv5 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.100+0000 7f0d23013800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.100+0000 7f0d23013800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.100+0000 7f0d23013800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.100+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94558082070032 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e43290 con 0x560004dfa6c0 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94558081814512 notify 133143986180 ret 0) ==== 609+0+0 (crc 0 0 0) 0x560004cc72c0 con 0x560004dfa6c0 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0c01ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986180 cookie 94558081814512 notifier 4355 bl.length()=567 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0c01ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0c01ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(64 notify.4 [notify cookie 94558082070032 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0c01ffb640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f0be4003ea0 con 0x560004dfa6c0 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94558082070032 notify 133143986180 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f0d0802d8a0 con 0x560004dfa6c0 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.101+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e47610 con 0x7f0d1007d060 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.102+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(66 p5YnriCv1nAtykxBrupQ [delete] v31'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.102+0000 7f0d23013800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.102+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94558082070032 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560004e42de0 con 0x560004dff0b0 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.103+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== watch-notify(notify (1) cookie 94558081806688 notify 133143986177 ret 0) ==== 212+0+0 (crc 0 0 0) 0x560004def4d0 con 0x560004dff0b0 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.103+0000 7f0c017fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986177 cookie 94558081806688 notifier 4355 bl.length()=170 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.103+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 notify.2 [notify cookie 94558082070032 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.103+0000 7f0c017fa640 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f0b540032c0 con 0x560004dff0b0 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.103+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== watch-notify(notify_complete (2) cookie 94558082070032 notify 133143986177 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f0d1806a020 con 0x560004dff0b0 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.103+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.115 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:23.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.104+0000 7f0bd4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.104+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94558081821968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x560004dfa6c0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.105+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94558081821968] v31'69 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.105+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94558081798080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x7f0d18035ba0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.106+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94558081798080] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.106+0000 7f0d23013800 20 remove_watcher() i=0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.106+0000 7f0d23013800 2 removed watcher, disabling cache 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.106+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94558081805152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x560004dfa6c0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.107+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94558081805152] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.107+0000 7f0d23013800 20 remove_watcher() i=1 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.107+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94558081806688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x560004dff0b0 2024-09-19T11:23:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.108+0000 7f0d1ef49640 1 -- 172.21.15.28:0/212673691 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94558081806688] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d18053020 con 0x560004dff0b0 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.108+0000 7f0d23013800 20 remove_watcher() i=2 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.108+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94558081810912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x7f0d18035ba0 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.109+0000 7f0d1e748640 1 -- 172.21.15.28:0/212673691 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94558081810912] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d1008b040 con 0x7f0d18035ba0 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.109+0000 7f0d23013800 20 remove_watcher() i=3 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.109+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94558081814512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x560004dfa6c0 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.110+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94558081814512] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x560004dfa6c0 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.110+0000 7f0d23013800 20 remove_watcher() i=4 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.110+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94558081818528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x7f0d1007d060 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.111+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94558081818528] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.111+0000 7f0d23013800 20 remove_watcher() i=5 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.111+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94558081807808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x7f0d1007d060 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.112+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94558081807808] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.112+0000 7f0d23013800 20 remove_watcher() i=6 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.112+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94558081820912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560004e3f530 con 0x7f0d1007d060 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.113+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94558081820912] v31'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08029040 con 0x7f0d1007d060 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.113+0000 7f0d23013800 20 remove_watcher() i=7 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0d1007d060 msgr2=0x7f0d100966d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0d1007d060 0x7f0d100966d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004dfa6c0 msgr2=0x560004dfcb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004dfa6c0 0x560004dfcb80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0d18035ba0 msgr2=0x7f0d180446f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0d18035ba0 0x7f0d180446f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dff0b0 msgr2=0x560004e014a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dff0b0 0x560004e014a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0be803d000 msgr2=0x7f0be803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0be803d000 0x7f0be803f4c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0d10054e60 tx=0x7f0d10087000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 msgr2=0x560004deef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d1df47640 1 -- 172.21.15.28:0/212673691 reap_dead start 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004deef70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0d18016cd0 tx=0x7f0d18054790 comp rx=0 tx=0).stop 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 shutdown_connections 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/212673691 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004de79c0 0x560004deef70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 >> 172.21.15.28:0/212673691 conn(0x560004de3dc0 msgr2=0x560004de1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 shutdown_connections 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/212673691 wait complete. 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0bf803d000 msgr2=0x7f0bf803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0bf803d000 0x7f0bf803f4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x560004db5c30 tx=0x7f0d1005e4a0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004ddaf40 msgr2=0x560004ddb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004ddaf40 0x560004ddb360 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x560004db57c0 tx=0x7f0d180167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 shutdown_connections 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0bf803d000 0x7f0bf803f4c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 --2- 172.21.15.28:0/2562460159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004ddaf40 0x560004ddb360 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 >> 172.21.15.28:0/2562460159 conn(0x560004dcf2f0 msgr2=0x560004dcf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 shutdown_connections 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.114+0000 7f0d23013800 1 -- 172.21.15.28:0/2562460159 wait complete. 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004cc65e0 msgr2=0x560004dc7350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004cc65e0 0x560004dc7350 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dc9a60 msgr2=0x560004dcbf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dc9a60 0x560004dcbf00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c0403d000 msgr2=0x7f0c0403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c0403d000 0x7f0c0403f4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d08001d70 tx=0x7f0d080012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 msgr2=0x560004db6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004db6670 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0d10045f50 tx=0x7f0d10002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 shutdown_connections 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560004dc9a60 0x560004dcbf00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c0403d000 0x7f0c0403f4c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560004cc65e0 0x560004dc7350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 --2- 172.21.15.28:0/1337137954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560004cc5bb0 0x560004db6670 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 >> 172.21.15.28:0/1337137954 conn(0x560004cc01d0 msgr2=0x560004daf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 shutdown_connections 2024-09-19T11:23:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.115+0000 7f0d23013800 1 -- 172.21.15.28:0/1337137954 wait complete. 2024-09-19T11:23:23.127 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:23.127 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-09-19T11:23:23.127 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-09-19T11:23:23.127 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-09-19T11:23:23.175 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.175 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb801b800 1 Processor -- start 2024-09-19T11:23:23.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb801b800 1 -- start start 2024-09-19T11:23:23.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb801b800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5611ad187ab0 0x5611ad18d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb801b800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5611ad186d50 con 0x5611ad1866a0 2024-09-19T11:23:23.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb801b800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5611ad0127d0 con 0x5611ad187ab0 2024-09-19T11:23:23.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb2fea640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5611ad187ab0 0x5611ad18d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb37eb640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5611ad1866a0 0x5611ad1873a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33096/0 (socket says 172.21.15.28:33096) 2024-09-19T11:23:23.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb37eb640 1 -- 172.21.15.28:0/2229744708 learned_addr learned my addr 172.21.15.28:0/2229744708 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:23.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb2fea640 1 -- 172.21.15.28:0/2229744708 >> v1:172.21.15.22:6789/0 conn(0x5611ad1866a0 legacy=0x5611ad1873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb27e9640 1 -- 172.21.15.28:0/2229744708 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3225912545 0 0) 0x5611ad186d50 con 0x5611ad1866a0 2024-09-19T11:23:23.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.175+0000 7fcbb2fea640 1 -- 172.21.15.28:0/2229744708 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad187f60 con 0x5611ad187ab0 2024-09-19T11:23:23.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.177+0000 7fcbb2fea640 1 --2- 172.21.15.28:0/2229744708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad187ab0 0x5611ad18d650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcba800a470 tx=0x7fcba8045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b07972ed29034f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.177+0000 7fcbb27e9640 1 -- 172.21.15.28:0/2229744708 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba8053570 con 0x5611ad187ab0 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.177+0000 7fcbb27e9640 1 -- 172.21.15.28:0/2229744708 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba8058070 con 0x5611ad187ab0 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb27e9640 1 -- 172.21.15.28:0/2229744708 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba8053880 con 0x5611ad187ab0 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb801b800 1 -- 172.21.15.28:0/2229744708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad187ab0 msgr2=0x5611ad18d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2229744708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad187ab0 0x5611ad18d650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcba800a470 tx=0x7fcba8045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb801b800 1 -- 172.21.15.28:0/2229744708 shutdown_connections 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2229744708 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad187ab0 0x5611ad18d650 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb801b800 1 -- 172.21.15.28:0/2229744708 >> 172.21.15.28:0/2229744708 conn(0x5611ad181a70 msgr2=0x5611ad183ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb801b800 1 -- 172.21.15.28:0/2229744708 shutdown_connections 2024-09-19T11:23:23.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.178+0000 7fcbb801b800 1 -- 172.21.15.28:0/2229744708 wait complete. 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.193+0000 7fcbb801b800 1 Processor -- start 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.193+0000 7fcbb801b800 1 -- start start 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.193+0000 7fcbb801b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad276070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.193+0000 7fcbb801b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5611ad01fdd0 con 0x5611ad1866a0 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcbb37eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad276070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcbb37eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad276070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59452/0 (socket says 172.21.15.28:59452) 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcbb37eb640 1 -- 172.21.15.28:0/2029279840 learned_addr learned my addr 172.21.15.28:0/2029279840 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcbb37eb640 1 -- 172.21.15.28:0/2029279840 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad276660 con 0x5611ad1866a0 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/2029279840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad276070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006fa0 tx=0x7fcba4001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adf6ee1fa560d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcaa160a640 1 -- 172.21.15.28:0/2029279840 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba4003450 con 0x5611ad1866a0 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcaa160a640 1 -- 172.21.15.28:0/2029279840 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba4003610 con 0x5611ad1866a0 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.194+0000 7fcaa160a640 1 -- 172.21.15.28:0/2029279840 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba4003970 con 0x5611ad1866a0 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 -- 172.21.15.28:0/2029279840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 msgr2=0x5611ad276070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2029279840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad276070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006fa0 tx=0x7fcba4001b00 comp rx=0 tx=0).stop 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 -- 172.21.15.28:0/2029279840 shutdown_connections 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2029279840 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad276070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 -- 172.21.15.28:0/2029279840 >> 172.21.15.28:0/2029279840 conn(0x5611ad181a70 msgr2=0x5611ad2719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 -- 172.21.15.28:0/2029279840 shutdown_connections 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 -- 172.21.15.28:0/2029279840 wait complete. 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 Processor -- start 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.195+0000 7fcbb801b800 1 -- start start 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fcbb801b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad0fa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fcbb801b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5611ad18dda0 con 0x5611ad1866a0 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fcbb37eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad0fa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fcbb37eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad0fa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59460/0 (socket says 172.21.15.28:59460) 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fcbb37eb640 1 -- 172.21.15.28:0/3851412162 learned_addr learned my addr 172.21.15.28:0/3851412162 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fcbb37eb640 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad0fae00 con 0x5611ad1866a0 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad0fa8a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcba4001330 tx=0x7fcba4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.196+0000 7fca9fe07640 1 -- 172.21.15.28:0/3851412162 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba4014a10 con 0x5611ad1866a0 2024-09-19T11:23:23.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.197+0000 7fca9fe07640 1 -- 172.21.15.28:0/3851412162 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba4014bb0 con 0x5611ad1866a0 2024-09-19T11:23:23.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.197+0000 7fca9fe07640 1 -- 172.21.15.28:0/3851412162 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba401e3f0 con 0x5611ad1866a0 2024-09-19T11:23:23.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.197+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5611ad281910 con 0x5611ad1866a0 2024-09-19T11:23:23.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.197+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ad281ed0 con 0x5611ad1866a0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fca9fe07640 1 -- 172.21.15.28:0/3851412162 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fcba4010070 con 0x5611ad1866a0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fca9fe07640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca9803d000 0x7fca9803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fca9fe07640 1 -- 172.21.15.28:0/3851412162 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fcba4018070 con 0x5611ad1866a0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fcbb2fea640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca9803d000 0x7fca9803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad187ab0 0x5611ad285ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad286550 con 0x5611ad187ab0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad187ab0 0x5611ad285ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.198+0000 7fcbb2fea640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca9803d000 0x7fca9803f4c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcba8057000 tx=0x7fcba8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad187ab0 0x5611ad285ee0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb3fec640 1 -- 172.21.15.28:0/3851412162 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcbac0083e0 con 0x5611ad187ab0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad283310 con 0x5611ad187ab0 2024-09-19T11:23:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb3fec640 1 -- 172.21.15.28:0/3851412162 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcbac0083e0 con 0x5611ad187ab0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2888d0 0x5611ad28acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad28b3b0 con 0x5611ad2888d0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2888d0 0x5611ad28acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.199+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2888d0 0x5611ad28acc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.200+0000 7fcbb37eb640 1 -- 172.21.15.28:0/3851412162 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fcba401b070 con 0x5611ad2888d0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.200+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad287770 con 0x5611ad187ab0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.200+0000 7fcbb3fec640 1 -- 172.21.15.28:0/3851412162 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcbac0083e0 con 0x5611ad187ab0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.200+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad287c10 con 0x5611ad2888d0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb37eb640 1 -- 172.21.15.28:0/3851412162 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fcba401b070 con 0x5611ad2888d0 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb801b800 1 Processor -- start 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb801b800 1 -- start start 2024-09-19T11:23:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb801b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad299d10 0x5611ad29a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb801b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5611ad07e4a0 con 0x5611ad299d10 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb3fec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad299d10 0x5611ad29a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb3fec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad299d10 0x5611ad29a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59472/0 (socket says 172.21.15.28:59472) 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.201+0000 7fcbb3fec640 1 -- 172.21.15.28:0/2758748235 learned_addr learned my addr 172.21.15.28:0/2758748235 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.202+0000 7fcbb3fec640 1 -- 172.21.15.28:0/2758748235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad29a770 con 0x5611ad299d10 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.202+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad299d10 0x5611ad29a130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5611ad0fa440 tx=0x7fcbac00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.202+0000 7fca9d602640 1 -- 172.21.15.28:0/2758748235 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbac01c020 con 0x5611ad299d10 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.202+0000 7fca9d602640 1 -- 172.21.15.28:0/2758748235 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac007c90 con 0x5611ad299d10 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.202+0000 7fca9d602640 1 -- 172.21.15.28:0/2758748235 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbac018440 con 0x5611ad299d10 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.202+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5611ad29afa0 con 0x5611ad299d10 2024-09-19T11:23:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.202+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ad29b540 con 0x5611ad299d10 2024-09-19T11:23:23.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.203+0000 7fca9d602640 1 -- 172.21.15.28:0/2758748235 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fcbac01f020 con 0x5611ad299d10 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.203+0000 7fca9d602640 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca8c03d000 0x7fca8c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.203+0000 7fca9d602640 1 -- 172.21.15.28:0/2758748235 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fcbac055a00 con 0x5611ad299d10 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.203+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca8c03d000 0x7fca8c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.203+0000 7fcbb801b800 1 Processor -- start 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.203+0000 7fcbb801b800 1 -- start start 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.203+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca8c03d000 0x7fca8c03f4c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5611ad286a50 tx=0x7fcba4014fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fcbb801b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fcbb801b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5611ad276a90 con 0x5611ad2a67d0 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fcbb3fec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fcbb3fec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2a6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59476/0 (socket says 172.21.15.28:59476) 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fcbb3fec640 1 -- 172.21.15.28:0/1445078229 learned_addr learned my addr 172.21.15.28:0/1445078229 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fcbb3fec640 1 -- 172.21.15.28:0/1445078229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad2a71c0 con 0x5611ad2a67d0 2024-09-19T11:23:23.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/1445078229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2a6bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbac053d80 tx=0x7fcbac053e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31591d025d9b6024 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fca96ffd640 1 -- 172.21.15.28:0/1445078229 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbac052020 con 0x5611ad2a67d0 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fca96ffd640 1 -- 172.21.15.28:0/1445078229 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac0174d0 con 0x5611ad2a67d0 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.204+0000 7fca96ffd640 1 -- 172.21.15.28:0/1445078229 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbac015840 con 0x5611ad2a67d0 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 -- 172.21.15.28:0/1445078229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 msgr2=0x5611ad2a6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 --2- 172.21.15.28:0/1445078229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2a6bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fcbac053d80 tx=0x7fcbac053e90 comp rx=0 tx=0).stop 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 -- 172.21.15.28:0/1445078229 shutdown_connections 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 --2- 172.21.15.28:0/1445078229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2a6bd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 -- 172.21.15.28:0/1445078229 >> 172.21.15.28:0/1445078229 conn(0x5611ad2a2bd0 msgr2=0x5611ad2a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 -- 172.21.15.28:0/1445078229 shutdown_connections 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 -- 172.21.15.28:0/1445078229 wait complete. 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 Processor -- start 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 -- start start 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb801b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5611ad2a77b0 con 0x5611ad2a67d0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb3fec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb3fec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59490/0 (socket says 172.21.15.28:59490) 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.205+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 learned_addr learned my addr 172.21.15.28:0/616201273 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.206+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ad2ae240 con 0x5611ad2a67d0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.206+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2adce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcbac008fd0 tx=0x7fcbac053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.206+0000 7fca957fa640 1 -- 172.21.15.28:0/616201273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbac013450 con 0x5611ad2a67d0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.206+0000 7fca957fa640 1 -- 172.21.15.28:0/616201273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbac0135f0 con 0x5611ad2a67d0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.206+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5611ad2ae4d0 con 0x5611ad2a67d0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.206+0000 7fca957fa640 1 -- 172.21.15.28:0/616201273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbac0139e0 con 0x5611ad2a67d0 2024-09-19T11:23:23.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.206+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ad2af060 con 0x5611ad2a67d0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.207+0000 7fca957fa640 1 -- 172.21.15.28:0/616201273 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fcbac016040 con 0x5611ad2a67d0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.207+0000 7fca957fa640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca7c03d000 0x7fca7c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.207+0000 7fca957fa640 1 -- 172.21.15.28:0/616201273 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fcbac034cb0 con 0x5611ad2a67d0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.207+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca7c03d000 0x7fca7c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.207+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca7c03d000 0x7fca7c03f4c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcba401d700 tx=0x7fcba4035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad2b9690 0x5611ad2bbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2bc2a0 con 0x5611ad2b9690 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb2fea640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad2b9690 0x5611ad2bbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb2fea640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad2b9690 0x5611ad2bbb50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb801b800 20 realm 2024-09-19T11:23:23.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.208+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2bc6d0 con 0x5611ad2b9690 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2bcfa0 con 0x5611ad2b9690 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2bd340 con 0x5611ad2b9690 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2be0a0 0x5611ad2c0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c0aa0 con 0x5611ad2be0a0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.209+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2be0a0 0x5611ad2c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2be0a0 0x5611ad2c0490 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb801b800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb801b800 20 searching for the correct realm 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb3fec640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcbac021020 0x7fcbac04e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcbac021020 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.210+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcbac021020 0x7fcbac04e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.211+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcbac021020 0x7fcbac04e860 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.211+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.211+0000 7fcbb37eb640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcba4055850 0x7fcba4053f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.211+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.211+0000 7fcbb2fea640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcba4055850 0x7fcba4053f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.211+0000 7fcbb2fea640 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcba4055850 0x7fcba4053f10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.212+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcbac021020 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.213+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcbac021020 2024-09-19T11:23:23.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.214+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.215+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcbac021020 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.216+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.217+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.218+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcbac021020 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.218+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.218+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.218+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.218+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcba4055850 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.218+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.218+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x5611ad2b9690 2024-09-19T11:23:23.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c23d0 con 0x7fcbac021020 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c1460 con 0x5611ad2b9690 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c1cb0 con 0x5611ad2b9690 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.219+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c2850 con 0x5611ad2be0a0 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 zone default found 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 4 Realm: () 2024-09-19T11:23:23.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 zonegroup default 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c2bf0 con 0x7fcba4055850 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c4a20 con 0x5611ad2be0a0 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c4eb0 con 0x5611ad2be0a0 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.220+0000 7fcbb801b800 20 started sync module instance, tier type = 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.221+0000 7fcbb801b800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.221+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c3ec0 con 0x7fcbac021020 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.222+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.222+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94633919789584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c4eb0 con 0x7fcbac021020 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.222+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c5700 con 0x5611ad2b9690 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.223+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94633919789584] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.223+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.223+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94633919795792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c5b20 con 0x5611ad2b9690 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.223+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c64e0 con 0x5611ad2be0a0 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.224+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94633919795792] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.224+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.224+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94633919797328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c64e0 con 0x5611ad2be0a0 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.224+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c6ee0 con 0x7fcbac021020 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.225+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94633919797328] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.225+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.225+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94633919801840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c64e0 con 0x7fcbac021020 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.225+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c7960 con 0x5611ad2b9690 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.227+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.227+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94633919804528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c7960 con 0x5611ad2b9690 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.227+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94633919801840] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.227+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c8760 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.228+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94633919804528] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.228+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.228+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94633919805792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c82b0 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.228+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c8d80 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.229+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.229+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94633919809648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c91f0 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.229+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c9d10 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.229+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94633919805792] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94633919809648] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94633919813600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c9cd0 con 0x7fcba4055850 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 20 add_watcher() i=0 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 20 add_watcher() i=1 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 20 add_watcher() i=2 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 20 add_watcher() i=3 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 20 add_watcher() i=4 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 20 add_watcher() i=5 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.230+0000 7fcbb801b800 20 add_watcher() i=6 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.231+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94633919813600] v31'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x7fcba4055850 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.231+0000 7fcbb801b800 20 add_watcher() i=7 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.231+0000 7fcbb801b800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.231+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c7d80 con 0x5611ad2b9690 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.231+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.231+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94633919810880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2c7d80 con 0x5611ad2b9690 2024-09-19T11:23:23.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.232+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94633919810880] v31'70 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcba8051070 con 0x5611ad2b9690 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.232+0000 7fca69fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.232+0000 7fcbb801b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.232+0000 7fcbb801b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.232+0000 7fcbb801b800 5 note: GC not initialized 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fca68fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 20 Filter name: none 2024-09-19T11:23:23.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fc9e27fc640 20 reqs_thread_entry: start 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2ffa30 con 0x5611ad2b9690 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.233+0000 7fcbb801b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad300340 con 0x5611ad2b9690 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.234+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad305ce0 con 0x5611ad2b9690 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~556] v31'6 uv6 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fcbb801b800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fcbb801b800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fcbb801b800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94633920059168 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad301350 con 0x5611ad2b9690 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify (1) cookie 94633919804528 notify 133143986181 ret 0) ==== 774+0+0 (crc 0 0 0) 0x5611ad187f60 con 0x5611ad2b9690 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fca96ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986181 cookie 94633919804528 notifier 4362 bl.length()=732 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fca96ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fca96ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fca96ffd640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fca78003ea0 con 0x5611ad2b9690 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.235+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(65 notify.4 [notify cookie 94633920059168 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.236+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.236+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== watch-notify(notify_complete (2) cookie 94633920059168 notify 133143986181 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fcba805fd80 con 0x5611ad2b9690 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.236+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad305720 con 0x5611ad2b9690 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fcbb801b800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fcbb801b800 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fcbb801b800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94633920059168 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5611ad300ea0 con 0x7fcbac021020 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== watch-notify(notify (1) cookie 94633919789584 notify 133143986178 ret 0) ==== 208+0+0 (crc 0 0 0) 0x7fcba405c040 con 0x7fcbac021020 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fca967fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986178 cookie 94633919789584 notifier 4362 bl.length()=166 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fca967fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fca967fc640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-09-19T11:23:23.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(68 notify.0 [notify cookie 94633920059168 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.237+0000 7fca967fc640 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fc9e8003400 con 0x7fcbac021020 2024-09-19T11:23:23.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.238+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== watch-notify(notify_complete (2) cookie 94633920059168 notify 133143986178 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fcba404f040 con 0x7fcbac021020 2024-09-19T11:23:23.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.238+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.251 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"jsivcge33DHXD4BPA0goVnBWhLZi9KWWiS3urfS0","active":true,"create_date":"2024-09-19T11:23:23.235068Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:23.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.239+0000 7fca69fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.239+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94633919810880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x5611ad2b9690 2024-09-19T11:23:23.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.240+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94633919810880] v31'71 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.240+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94633919789584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x7fcbac021020 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.241+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94633919789584] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.241+0000 7fcbb801b800 20 remove_watcher() i=0 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.241+0000 7fcbb801b800 2 removed watcher, disabling cache 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.241+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94633919795792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x5611ad2b9690 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.242+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94633919795792] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.242+0000 7fcbb801b800 20 remove_watcher() i=1 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.242+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94633919797328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x5611ad2be0a0 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.243+0000 7fcbb3fec640 1 -- 172.21.15.28:0/616201273 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94633919797328] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcbac039020 con 0x5611ad2be0a0 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.243+0000 7fcbb801b800 20 remove_watcher() i=2 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.243+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94633919801840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x7fcbac021020 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.244+0000 7fcbb37eb640 1 -- 172.21.15.28:0/616201273 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94633919801840] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba401b070 con 0x7fcbac021020 2024-09-19T11:23:23.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.244+0000 7fcbb801b800 20 remove_watcher() i=3 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.244+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94633919804528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x5611ad2b9690 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.245+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94633919804528] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x5611ad2b9690 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.245+0000 7fcbb801b800 20 remove_watcher() i=4 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.245+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94633919805792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x7fcba4055850 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.246+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94633919805792] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x7fcba4055850 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.246+0000 7fcbb801b800 20 remove_watcher() i=5 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.246+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94633919809648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x7fcba4055850 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.247+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94633919809648] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x7fcba4055850 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.247+0000 7fcbb801b800 20 remove_watcher() i=6 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.247+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94633919813600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5611ad2fd8d0 con 0x7fcba4055850 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94633919813600] v31'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcba807f040 con 0x7fcba4055850 2024-09-19T11:23:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 20 remove_watcher() i=7 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcba4055850 msgr2=0x7fcba4053f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcba4055850 0x7fcba4053f10 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad2b9690 msgr2=0x5611ad2bbb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad2b9690 0x5611ad2bbb50 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcbac021020 msgr2=0x7fcbac04e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcbac021020 0x7fcbac04e860 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2be0a0 msgr2=0x5611ad2c0490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.248+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2be0a0 0x5611ad2c0490 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca7c03d000 msgr2=0x7fca7c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca7c03d000 0x7fca7c03f4c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcba401d700 tx=0x7fcba4035000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 msgr2=0x5611ad2adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2adce0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcbac008fd0 tx=0x7fcbac053ce0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb2fea640 1 -- 172.21.15.28:0/616201273 reap_dead start 2024-09-19T11:23:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 shutdown_connections 2024-09-19T11:23:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 --2- 172.21.15.28:0/616201273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad2a67d0 0x5611ad2adce0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 >> 172.21.15.28:0/616201273 conn(0x5611ad2a2bd0 msgr2=0x5611ad2a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 shutdown_connections 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/616201273 wait complete. 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca8c03d000 msgr2=0x7fca8c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca8c03d000 0x7fca8c03f4c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5611ad286a50 tx=0x7fcba4014fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad299d10 msgr2=0x5611ad29a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad299d10 0x5611ad29a130 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5611ad0fa440 tx=0x7fcbac00bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 shutdown_connections 2024-09-19T11:23:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca8c03d000 0x7fca8c03f4c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 --2- 172.21.15.28:0/2758748235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad299d10 0x5611ad29a130 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 >> 172.21.15.28:0/2758748235 conn(0x5611ad28e0c0 msgr2=0x5611ad28e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 shutdown_connections 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.249+0000 7fcbb801b800 1 -- 172.21.15.28:0/2758748235 wait complete. 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad187ab0 msgr2=0x5611ad285ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad187ab0 0x5611ad285ee0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2888d0 msgr2=0x5611ad28acc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2888d0 0x5611ad28acc0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca9803d000 msgr2=0x7fca9803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca9803d000 0x7fca9803f4c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcba8057000 tx=0x7fcba8052000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 msgr2=0x5611ad0fa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad0fa8a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcba4001330 tx=0x7fcba4001250 comp rx=0 tx=0).stop 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 shutdown_connections 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5611ad2888d0 0x5611ad28acc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fca9803d000 0x7fca9803f4c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5611ad187ab0 0x5611ad285ee0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 --2- 172.21.15.28:0/3851412162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5611ad1866a0 0x5611ad0fa8a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 >> 172.21.15.28:0/3851412162 conn(0x5611ad181a70 msgr2=0x5611ad277560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 shutdown_connections 2024-09-19T11:23:23.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.250+0000 7fcbb801b800 1 -- 172.21.15.28:0/3851412162 wait complete. 2024-09-19T11:23:23.265 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'jsivcge33DHXD4BPA0goVnBWhLZi9KWWiS3urfS0', 'active': True, 'create_date': '2024-09-19T11:23:23.235068Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:23.265 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-09-19T11:23:23.265 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-09-19T11:23:23.266 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-09-19T11:23:23.316 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.317 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42e152e800 1 Processor -- start 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42e152e800 1 -- start start 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42e152e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55629d7cef60 0x55629d7d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42e152e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55629d7d55f0 con 0x55629d7d18f0 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42e152e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55629d6597d0 con 0x55629d7cef60 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42dd16b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55629d7cef60 0x55629d7d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42dc96a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55629d7d18f0 0x55629d7d3d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33112/0 (socket says 172.21.15.28:33112) 2024-09-19T11:23:23.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.316+0000 7f42dc96a640 1 -- 172.21.15.28:0/3307633590 learned_addr learned my addr 172.21.15.28:0/3307633590 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:23.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.317+0000 7f42dd16b640 1 -- 172.21.15.28:0/3307633590 >> v1:172.21.15.22:6789/0 conn(0x55629d7d18f0 legacy=0x55629d7d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.317+0000 7f42dd16b640 1 -- 172.21.15.28:0/3307633590 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d7d6820 con 0x55629d7cef60 2024-09-19T11:23:23.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.318+0000 7f42dd16b640 1 --2- 172.21.15.28:0/3307633590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d7d1350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42d000a470 tx=0x7f42d0045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f916245955042002 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42d7fff640 1 -- 172.21.15.28:0/3307633590 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0053570 con 0x55629d7cef60 2024-09-19T11:23:23.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42d7fff640 1 -- 172.21.15.28:0/3307633590 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d0058070 con 0x55629d7cef60 2024-09-19T11:23:23.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42d7fff640 1 -- 172.21.15.28:0/3307633590 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d0053880 con 0x55629d7cef60 2024-09-19T11:23:23.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42e152e800 1 -- 172.21.15.28:0/3307633590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 msgr2=0x55629d7d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42e152e800 1 --2- 172.21.15.28:0/3307633590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d7d1350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f42d000a470 tx=0x7f42d0045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42e152e800 1 -- 172.21.15.28:0/3307633590 shutdown_connections 2024-09-19T11:23:23.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42e152e800 1 --2- 172.21.15.28:0/3307633590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d7d1350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42e152e800 1 -- 172.21.15.28:0/3307633590 >> 172.21.15.28:0/3307633590 conn(0x55629d7c8a80 msgr2=0x55629d7caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42e152e800 1 -- 172.21.15.28:0/3307633590 shutdown_connections 2024-09-19T11:23:23.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.319+0000 7f42e152e800 1 -- 172.21.15.28:0/3307633590 wait complete. 2024-09-19T11:23:23.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.334+0000 7f42e152e800 1 Processor -- start 2024-09-19T11:23:23.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.334+0000 7f42e152e800 1 -- start start 2024-09-19T11:23:23.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.334+0000 7f42e152e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.334+0000 7f42e152e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55629d666dd0 con 0x55629d7cef60 2024-09-19T11:23:23.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f42dd16b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f42dd16b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8be3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59498/0 (socket says 172.21.15.28:59498) 2024-09-19T11:23:23.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f42dd16b640 1 -- 172.21.15.28:0/3981129086 learned_addr learned my addr 172.21.15.28:0/3981129086 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f42dd16b640 1 -- 172.21.15.28:0/3981129086 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d8be9d0 con 0x55629d7cef60 2024-09-19T11:23:23.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f42dd16b640 1 --2- 172.21.15.28:0/3981129086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8be3e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42d0025c20 tx=0x7f42d0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75302bb986819a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f41cb606640 1 -- 172.21.15.28:0/3981129086 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d005c580 con 0x55629d7cef60 2024-09-19T11:23:23.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f41cb606640 1 -- 172.21.15.28:0/3981129086 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d0063030 con 0x55629d7cef60 2024-09-19T11:23:23.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.335+0000 7f41cb606640 1 -- 172.21.15.28:0/3981129086 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d005c850 con 0x55629d7cef60 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 -- 172.21.15.28:0/3981129086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 msgr2=0x55629d8be3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 --2- 172.21.15.28:0/3981129086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8be3e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42d0025c20 tx=0x7f42d0051000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 -- 172.21.15.28:0/3981129086 shutdown_connections 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 --2- 172.21.15.28:0/3981129086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8be3e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 -- 172.21.15.28:0/3981129086 >> 172.21.15.28:0/3981129086 conn(0x55629d7c8a80 msgr2=0x55629d8b9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 -- 172.21.15.28:0/3981129086 shutdown_connections 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 -- 172.21.15.28:0/3981129086 wait complete. 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.336+0000 7f42e152e800 1 Processor -- start 2024-09-19T11:23:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42e152e800 1 -- start start 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42e152e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42e152e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55629d7d5ac0 con 0x55629d7cef60 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42dd16b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42dd16b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59510/0 (socket says 172.21.15.28:59510) 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42dd16b640 1 -- 172.21.15.28:0/2466543109 learned_addr learned my addr 172.21.15.28:0/2466543109 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42dd16b640 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d8c6300 con 0x55629d7cef60 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.337+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8c5da0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f42d000a330 tx=0x7f42d0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.338+0000 7f41c9e03640 1 -- 172.21.15.28:0/2466543109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d006c030 con 0x55629d7cef60 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.338+0000 7f41c9e03640 1 -- 172.21.15.28:0/2466543109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d005c580 con 0x55629d7cef60 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.338+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55629d8c6590 con 0x55629d7cef60 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.338+0000 7f41c9e03640 1 -- 172.21.15.28:0/2466543109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d005c860 con 0x55629d7cef60 2024-09-19T11:23:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.338+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629d8c71a0 con 0x55629d7cef60 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f41c9e03640 1 -- 172.21.15.28:0/2466543109 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f42d0053440 con 0x55629d7cef60 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f41c9e03640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41c403cf60 0x7f41c403f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f41c9e03640 1 -- 172.21.15.28:0/2466543109 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f42d0070020 con 0x55629d7cef60 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f42dc96a640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41c403cf60 0x7f41c403f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d7d18f0 0x55629d8cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d8cd930 con 0x55629d7d18f0 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f42dd96c640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d7d18f0 0x55629d8cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f42dc96a640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41c403cf60 0x7f41c403f420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42c8004170 tx=0x7f42c8001330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.339+0000 7f42dd96c640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d7d18f0 0x55629d8cd3f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.340+0000 7f42dd96c640 1 -- 172.21.15.28:0/2466543109 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d7d18f0 2024-09-19T11:23:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.340+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d8cdcd0 con 0x55629d7d18f0 2024-09-19T11:23:23.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.340+0000 7f42dd96c640 1 -- 172.21.15.28:0/2466543109 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d7d18f0 2024-09-19T11:23:23.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.340+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d8cf7d0 0x55629d8d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.340+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d8d23d0 con 0x55629d8cf7d0 2024-09-19T11:23:23.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.340+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d8cf7d0 0x55629d8d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.340+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d8cf7d0 0x55629d8d1c40 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.341+0000 7f42dd16b640 1 -- 172.21.15.28:0/2466543109 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f42d007b070 con 0x55629d8cf7d0 2024-09-19T11:23:23.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.341+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d8ce520 con 0x55629d7d18f0 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.341+0000 7f42dd96c640 1 -- 172.21.15.28:0/2466543109 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d7d18f0 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.341+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d8ce970 con 0x55629d8cf7d0 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.341+0000 7f42dd16b640 1 -- 172.21.15.28:0/2466543109 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f42d007b070 con 0x55629d8cf7d0 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42e152e800 1 Processor -- start 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42e152e800 1 -- start start 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42e152e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8e0d80 0x55629d8e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42e152e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55629d6c54a0 con 0x55629d8e0d80 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42dd96c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8e0d80 0x55629d8e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42dd96c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8e0d80 0x55629d8e11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59524/0 (socket says 172.21.15.28:59524) 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42dd96c640 1 -- 172.21.15.28:0/2732083283 learned_addr learned my addr 172.21.15.28:0/2732083283 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.342+0000 7f42dd96c640 1 -- 172.21.15.28:0/2732083283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d8e17e0 con 0x55629d8e0d80 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.343+0000 7f42dd96c640 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8e0d80 0x55629d8e11a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55629d8c6da0 tx=0x7f42d80167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.343+0000 7f41c2ffd640 1 -- 172.21.15.28:0/2732083283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8013640 con 0x55629d8e0d80 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.343+0000 7f41c2ffd640 1 -- 172.21.15.28:0/2732083283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d80137e0 con 0x55629d8e0d80 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.343+0000 7f41c2ffd640 1 -- 172.21.15.28:0/2732083283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8013ac0 con 0x55629d8e0d80 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.343+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55629d8e2010 con 0x55629d8e0d80 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.343+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629d8e25b0 con 0x55629d8e0d80 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f41c2ffd640 1 -- 172.21.15.28:0/2732083283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f42d8011040 con 0x55629d8e0d80 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f41c2ffd640 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41b403cfb0 0x7f41b403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f41c2ffd640 1 -- 172.21.15.28:0/2732083283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f42d800e040 con 0x55629d8e0d80 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41b403cfb0 0x7f41b403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f42e152e800 1 Processor -- start 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f42e152e800 1 -- start start 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41b403cfb0 0x7f41b403f470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f42d0053fd0 tx=0x7f42d00687b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f42e152e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d8edba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.344+0000 7f42e152e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55629d8beda0 con 0x55629d8ed7c0 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f42dd96c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d8edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f42dd96c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d8edba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59536/0 (socket says 172.21.15.28:59536) 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f42dd96c640 1 -- 172.21.15.28:0/3825199271 learned_addr learned my addr 172.21.15.28:0/3825199271 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f42dd96c640 1 -- 172.21.15.28:0/3825199271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d8ee190 con 0x55629d8ed7c0 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f42dd96c640 1 --2- 172.21.15.28:0/3825199271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d8edba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f42d8054840 tx=0x7f42d801fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2094d8be15f085a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f41c0ff9640 1 -- 172.21.15.28:0/3825199271 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8051070 con 0x55629d8ed7c0 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f41c0ff9640 1 -- 172.21.15.28:0/3825199271 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d8021910 con 0x55629d8ed7c0 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.345+0000 7f41c0ff9640 1 -- 172.21.15.28:0/3825199271 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8021440 con 0x55629d8ed7c0 2024-09-19T11:23:23.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 -- 172.21.15.28:0/3825199271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 msgr2=0x55629d8edba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 --2- 172.21.15.28:0/3825199271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d8edba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f42d8054840 tx=0x7f42d801fc90 comp rx=0 tx=0).stop 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 -- 172.21.15.28:0/3825199271 shutdown_connections 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 --2- 172.21.15.28:0/3825199271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d8edba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 -- 172.21.15.28:0/3825199271 >> 172.21.15.28:0/3825199271 conn(0x55629d8e9bc0 msgr2=0x55629d8e7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 -- 172.21.15.28:0/3825199271 shutdown_connections 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 -- 172.21.15.28:0/3825199271 wait complete. 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 Processor -- start 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 -- start start 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d7ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42e152e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55629d8ee7d0 con 0x55629d8ed7c0 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42dd96c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d7ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42dd96c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d7ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59550/0 (socket says 172.21.15.28:59550) 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.346+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 learned_addr learned my addr 172.21.15.28:0/2818578046 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.347+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55629d7cebc0 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.347+0000 7f42dd96c640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d7ce660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f42d8020790 tx=0x7f42d80547d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.347+0000 7f41b2ffd640 1 -- 172.21.15.28:0/2818578046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d8054a40 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.347+0000 7f41b2ffd640 1 -- 172.21.15.28:0/2818578046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d801f6c0 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.347+0000 7f41b2ffd640 1 -- 172.21.15.28:0/2818578046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42d801f9a0 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.347+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55629d7ccf70 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.347+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55629d7cd4b0 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.348+0000 7f41b2ffd640 1 -- 172.21.15.28:0/2818578046 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f42d8045a10 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.348+0000 7f41b2ffd640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41a803cfb0 0x7f41a803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.348+0000 7f41b2ffd640 1 -- 172.21.15.28:0/2818578046 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f42d8034c80 con 0x55629d8ed7c0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.348+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41a803cfb0 0x7f41a803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.348+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41a803cfb0 0x7f41a803f470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f42d005d790 tx=0x7f42d008a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d9028e0 0x55629d904da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d9054f0 con 0x55629d9028e0 2024-09-19T11:23:23.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42dc96a640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d9028e0 0x55629d904da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42dc96a640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d9028e0 0x55629d904da0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f42c8009050 con 0x55629d9028e0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42e152e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42e152e800 20 realm 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.349+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d901c30 con 0x55629d9028e0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f42c8009050 con 0x55629d9028e0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d905fe0 con 0x55629d9028e0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f42c8009050 con 0x55629d9028e0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d9068b0 con 0x55629d9028e0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f42c8009050 con 0x55629d9028e0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d9072d0 0x55629d9096c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d909d90 con 0x55629d9072d0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42dd96c640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d9072d0 0x55629d9096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.350+0000 7f42dd96c640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d9072d0 0x55629d9096c0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42e152e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42e152e800 20 searching for the correct realm 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42dd96c640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f42d8031b20 0x7f42d804d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d8031b20 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.351+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f42d8031b20 0x7f42d804d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.352+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f42d8031b20 0x7f42d804d750 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.352+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.352+0000 7f42dd16b640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f42d008c070 0x7f42d0053730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.352+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.352+0000 7f42dc96a640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f42d008c070 0x7f42d0053730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.352+0000 7f42dc96a640 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f42d008c070 0x7f42d0053730 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8009050 con 0x7f42d008c070 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8009050 con 0x55629d9028e0 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.353+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d8031b20 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.354+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d8031b20 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.355+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.356+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.357+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.357+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.357+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.357+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.357+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.357+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.358+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.358+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d8031b20 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.358+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.358+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.358+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.358+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d8031b20 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.359+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9072d0 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d008c070 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x55629d9028e0 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55629d90b6e0 con 0x7f42d8031b20 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.360+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d90a750 con 0x55629d9028e0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d90aff0 con 0x55629d9028e0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d90bb60 con 0x55629d9072d0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 zone default found 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 4 Realm: () 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:23.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 zonegroup default 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d90bf00 con 0x7f42d008c070 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.361+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d90e1b0 con 0x55629d9072d0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42e152e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d90e550 con 0x55629d9072d0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42e152e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42e152e800 20 started sync module instance, tier type = 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42e152e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.362+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d90d5f0 con 0x7f42d8031b20 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.363+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.363+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93882038669632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d90e980 con 0x7f42d8031b20 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.363+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d90f1d0 con 0x55629d9028e0 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.364+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93882038669632] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.364+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.364+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93882038676768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d90f5f0 con 0x55629d9028e0 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.364+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d90fe80 con 0x55629d9072d0 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.366+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.366+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93882038679024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d90fa90 con 0x55629d9072d0 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.366+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d910e50 con 0x7f42d8031b20 2024-09-19T11:23:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.366+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93882038676768] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.367+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93882038679024] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.367+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.367+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93882038682224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d90f140 con 0x7f42d8031b20 2024-09-19T11:23:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.367+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d911730 con 0x55629d9028e0 2024-09-19T11:23:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.368+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93882038682224] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.368+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.368+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93882038683968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d9116f0 con 0x55629d9028e0 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.368+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d912180 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.369+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.369+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93882038688912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d9120f0 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.369+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d913110 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.369+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93882038683968] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.370+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.370+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93882038686272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d913110 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.370+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d913b10 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.370+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93882038688912] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93882038686272] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93882038691152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d913620 con 0x7f42d008c070 2024-09-19T11:23:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 20 add_watcher() i=0 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 20 add_watcher() i=1 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 20 add_watcher() i=2 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 20 add_watcher() i=3 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 20 add_watcher() i=4 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 20 add_watcher() i=5 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.371+0000 7f42e152e800 20 add_watcher() i=6 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.372+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93882038691152] v31'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.372+0000 7f42e152e800 20 add_watcher() i=7 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.372+0000 7f42e152e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.372+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d916b00 con 0x55629d9028e0 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.373+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.373+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93882038695456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d913ef0 con 0x55629d9028e0 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93882038695456] v31'72 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f41937e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 5 note: GC not initialized 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f41927e4640 20 reqs_thread_entry: start 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 20 Filter name: none 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f4103fff640 20 reqs_thread_entry: start 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.374+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d9498b0 con 0x55629d9028e0 2024-09-19T11:23:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.375+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv6 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.375+0000 7f42e152e800 20 rados_obj.operate() r=0 bl.length=556 2024-09-19T11:23:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.375+0000 7f42e152e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.375+0000 7f42e152e800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.375+0000 7f42e152e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.375+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d94fea0 con 0x55629d9028e0 2024-09-19T11:23:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v31'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42e152e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42e152e800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42e152e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93882038941856 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55629d94b560 con 0x55629d9028e0 2024-09-19T11:23:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 93882038683968 notify 133143986182 ret 0) ==== 774+0+0 (crc 0 0 0) 0x55629d7d55f0 con 0x55629d9028e0 2024-09-19T11:23:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f41c0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986182 cookie 93882038683968 notifier 4369 bl.length()=732 2024-09-19T11:23:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(64 notify.4 [notify cookie 93882038941856 out=8b] v0'0 uv57 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f41c0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f41c0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f41c0ff9640 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f41a0003ea0 con 0x55629d9028e0 2024-09-19T11:23:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.376+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.377+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 93882038941856 notify 133143986182 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f42c802d6a0 con 0x55629d9028e0 2024-09-19T11:23:23.389 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:23.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.378+0000 7f41937e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.378+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93882038695456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d94b990 con 0x55629d9028e0 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.378+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93882038695456] v31'73 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.379+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93882038669632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x7f42d8031b20 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.380+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93882038669632] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.380+0000 7f42e152e800 20 remove_watcher() i=0 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.380+0000 7f42e152e800 2 removed watcher, disabling cache 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.380+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93882038676768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x55629d9028e0 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.381+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93882038676768] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.381+0000 7f42e152e800 20 remove_watcher() i=1 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.381+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93882038679024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x55629d9072d0 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.382+0000 7f42dd96c640 1 -- 172.21.15.28:0/2818578046 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93882038679024] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d800a4d0 con 0x55629d9072d0 2024-09-19T11:23:23.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.382+0000 7f42e152e800 20 remove_watcher() i=2 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.382+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93882038682224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x7f42d8031b20 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.383+0000 7f42dd16b640 1 -- 172.21.15.28:0/2818578046 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93882038682224] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42d0095070 con 0x7f42d8031b20 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.383+0000 7f42e152e800 20 remove_watcher() i=3 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.383+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93882038683968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x55629d9028e0 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.384+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93882038683968] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x55629d9028e0 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.384+0000 7f42e152e800 20 remove_watcher() i=4 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.384+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93882038688912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x7f42d008c070 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.385+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93882038688912] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.385+0000 7f42e152e800 20 remove_watcher() i=5 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.385+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93882038686272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x7f42d008c070 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.386+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93882038686272] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.386+0000 7f42e152e800 20 remove_watcher() i=6 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.386+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93882038691152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55629d947710 con 0x7f42d008c070 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93882038691152] v31'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f42c8029040 con 0x7f42d008c070 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 20 remove_watcher() i=7 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f42d008c070 msgr2=0x7f42d0053730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f42d008c070 0x7f42d0053730 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d9028e0 msgr2=0x55629d904da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d9028e0 0x55629d904da0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f42d8031b20 msgr2=0x7f42d804d750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f42d8031b20 0x7f42d804d750 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d9072d0 msgr2=0x55629d9096c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d9072d0 0x55629d9096c0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41a803cfb0 msgr2=0x7f41a803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41a803cfb0 0x7f41a803f470 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f42d005d790 tx=0x7f42d008a000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 msgr2=0x55629d7ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 --2- 172.21.15.28:0/2818578046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8ed7c0 0x55629d7ce660 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f42d8020790 tx=0x7f42d80547d0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 reap_dead start 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42dc96a640 1 -- 172.21.15.28:0/2818578046 reap_dead start 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 shutdown_connections 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 >> 172.21.15.28:0/2818578046 conn(0x55629d8e9bc0 msgr2=0x55629d8e7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.387+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 shutdown_connections 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2818578046 wait complete. 2024-09-19T11:23:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41b403cfb0 msgr2=0x7f41b403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41b403cfb0 0x7f41b403f470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f42d0053fd0 tx=0x7f42d00687b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8e0d80 msgr2=0x55629d8e11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8e0d80 0x55629d8e11a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55629d8c6da0 tx=0x7f42d80167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 shutdown_connections 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41b403cfb0 0x7f41b403f470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2732083283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d8e0d80 0x55629d8e11a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 >> 172.21.15.28:0/2732083283 conn(0x55629d8d5130 msgr2=0x55629d8d5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 shutdown_connections 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2732083283 wait complete. 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d7d18f0 msgr2=0x55629d8cd3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d7d18f0 0x55629d8cd3f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d8cf7d0 msgr2=0x55629d8d1c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d8cf7d0 0x55629d8d1c40 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41c403cf60 msgr2=0x7f41c403f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41c403cf60 0x7f41c403f420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f42c8004170 tx=0x7f42c8001330 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 msgr2=0x55629d8c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8c5da0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f42d000a330 tx=0x7f42d0002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 shutdown_connections 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55629d8cf7d0 0x55629d8d1c40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f41c403cf60 0x7f41c403f420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55629d7d18f0 0x55629d8cd3f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 --2- 172.21.15.28:0/2466543109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55629d7cef60 0x55629d8c5da0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.388+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 >> 172.21.15.28:0/2466543109 conn(0x55629d7c8a80 msgr2=0x55629d8b8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.389+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 shutdown_connections 2024-09-19T11:23:23.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.389+0000 7f42e152e800 1 -- 172.21.15.28:0/2466543109 wait complete. 2024-09-19T11:23:23.403 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:23.404 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:23.404 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:23.404 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:23.450 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.450 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.449+0000 7f1b86137800 1 Processor -- start 2024-09-19T11:23:23.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.449+0000 7f1b86137800 1 -- start start 2024-09-19T11:23:23.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b86137800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x565335a967a0 0x565335a96c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b86137800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x565335a972b0 con 0x565335a95ba0 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b86137800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5653359217d0 con 0x565335a967a0 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b8167c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x565335a967a0 0x565335a96c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b8167c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x565335a967a0 0x565335a96c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59562/0 (socket says 172.21.15.28:59562) 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b8167c640 1 -- 172.21.15.28:0/3793888863 learned_addr learned my addr 172.21.15.28:0/3793888863 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b81e7d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x565335a95ba0 0x565335a95fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33120/0 (socket says 172.21.15.28:33120) 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b8167c640 1 -- 172.21.15.28:0/3793888863 >> v1:172.21.15.22:6789/0 conn(0x565335a95ba0 legacy=0x565335a95fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b80e7b640 1 -- 172.21.15.28:0/3793888863 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980871854 0 0) 0x565335a972b0 con 0x565335a95ba0 2024-09-19T11:23:23.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.450+0000 7f1b8167c640 1 -- 172.21.15.28:0/3793888863 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x565335a9d750 con 0x565335a967a0 2024-09-19T11:23:23.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.452+0000 7f1b8167c640 1 --2- 172.21.15.28:0/3793888863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a967a0 0x565335a96c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800a520 tx=0x7f1b78045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ff196683bd88e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.452+0000 7f1b80e7b640 1 -- 172.21.15.28:0/3793888863 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b78055070 con 0x565335a967a0 2024-09-19T11:23:23.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.452+0000 7f1b80e7b640 1 -- 172.21.15.28:0/3793888863 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b780515b0 con 0x565335a967a0 2024-09-19T11:23:23.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.452+0000 7f1b80e7b640 1 -- 172.21.15.28:0/3793888863 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b780518d0 con 0x565335a967a0 2024-09-19T11:23:23.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.453+0000 7f1b86137800 1 -- 172.21.15.28:0/3793888863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a967a0 msgr2=0x565335a96c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.453+0000 7f1b86137800 1 --2- 172.21.15.28:0/3793888863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a967a0 0x565335a96c50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800a520 tx=0x7f1b78045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.453+0000 7f1b86137800 1 -- 172.21.15.28:0/3793888863 shutdown_connections 2024-09-19T11:23:23.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.453+0000 7f1b86137800 1 --2- 172.21.15.28:0/3793888863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a967a0 0x565335a96c50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.453+0000 7f1b86137800 1 -- 172.21.15.28:0/3793888863 >> 172.21.15.28:0/3793888863 conn(0x565335a901c0 msgr2=0x565335a92640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.453+0000 7f1b86137800 1 -- 172.21.15.28:0/3793888863 shutdown_connections 2024-09-19T11:23:23.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.453+0000 7f1b86137800 1 -- 172.21.15.28:0/3793888863 wait complete. 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.468+0000 7f1b86137800 1 Processor -- start 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.468+0000 7f1b86137800 1 -- start start 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.468+0000 7f1b86137800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.468+0000 7f1b86137800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56533592edd0 con 0x565335a95ba0 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1b81e7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1b81e7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b850c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59578/0 (socket says 172.21.15.28:59578) 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1b81e7d640 1 -- 172.21.15.28:0/2979858482 learned_addr learned my addr 172.21.15.28:0/2979858482 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1b81e7d640 1 -- 172.21.15.28:0/2979858482 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565335b856b0 con 0x565335a95ba0 2024-09-19T11:23:23.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/2979858482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b850c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b74001ea0 tx=0x7f1b7400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42aad5b430be2d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1a6fe07640 1 -- 172.21.15.28:0/2979858482 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b74008070 con 0x565335a95ba0 2024-09-19T11:23:23.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1a6fe07640 1 -- 172.21.15.28:0/2979858482 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7400d500 con 0x565335a95ba0 2024-09-19T11:23:23.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.469+0000 7f1a6fe07640 1 -- 172.21.15.28:0/2979858482 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7400d880 con 0x565335a95ba0 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 -- 172.21.15.28:0/2979858482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 msgr2=0x565335b850c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 --2- 172.21.15.28:0/2979858482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b850c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b74001ea0 tx=0x7f1b7400b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 -- 172.21.15.28:0/2979858482 shutdown_connections 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 --2- 172.21.15.28:0/2979858482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b850c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 -- 172.21.15.28:0/2979858482 >> 172.21.15.28:0/2979858482 conn(0x565335a901c0 msgr2=0x565335b80a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 -- 172.21.15.28:0/2979858482 shutdown_connections 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 -- 172.21.15.28:0/2979858482 wait complete. 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 Processor -- start 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 -- start start 2024-09-19T11:23:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b8ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.470+0000 7f1b86137800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565335a9ca40 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1b81e7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b8ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1b81e7d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b8ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59594/0 (socket says 172.21.15.28:59594) 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1b81e7d640 1 -- 172.21.15.28:0/2261585446 learned_addr learned my addr 172.21.15.28:0/2261585446 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1b81e7d640 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565335b8b0d0 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b8ab70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b740012f0 tx=0x7f1b74000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1a6e604640 1 -- 172.21.15.28:0/2261585446 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7400f9d0 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1a6e604640 1 -- 172.21.15.28:0/2261585446 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7400fb70 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565335b89480 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1a6e604640 1 -- 172.21.15.28:0/2261585446 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7400fe50 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.471+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565335b89a40 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.472+0000 7f1a6e604640 1 -- 172.21.15.28:0/2261585446 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1b74017040 con 0x565335a95ba0 2024-09-19T11:23:23.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.472+0000 7f1a6e604640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a6803d1f0 0x7f1a6803f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.472+0000 7f1a6e604640 1 -- 172.21.15.28:0/2261585446 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f1b74015020 con 0x565335a95ba0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b8167c640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a6803d1f0 0x7f1a6803f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335a967a0 0x565335b97130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335b97750 con 0x565335a967a0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b8267e640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335a967a0 0x565335b97130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b8167c640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a6803d1f0 0x7f1a6803f6b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b78057000 tx=0x7f1b78052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b8267e640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335a967a0 0x565335b97130 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b8267e640 1 -- 172.21.15.28:0/2261585446 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b7c00a4d0 con 0x565335a967a0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.473+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335b8a000 con 0x565335a967a0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.474+0000 7f1b8267e640 1 -- 172.21.15.28:0/2261585446 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b7c00a4d0 con 0x565335a967a0 2024-09-19T11:23:23.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.474+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335b99aa0 0x565335b9be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.474+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335b9c560 con 0x565335b99aa0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.474+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335b99aa0 0x565335b9be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.474+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335b99aa0 0x565335b9be90 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.474+0000 7f1b81e7d640 1 -- 172.21.15.28:0/2261585446 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f1b74054040 con 0x565335b99aa0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.474+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335b98940 con 0x565335a967a0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.475+0000 7f1b8267e640 1 -- 172.21.15.28:0/2261585446 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b7c00a4d0 con 0x565335a967a0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.475+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335b98da0 con 0x565335b99aa0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.475+0000 7f1b81e7d640 1 -- 172.21.15.28:0/2261585446 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f1b74054040 con 0x565335b99aa0 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.475+0000 7f1b86137800 1 Processor -- start 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.475+0000 7f1b86137800 1 -- start start 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1b86137800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335baaef0 0x565335bab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1b86137800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56533598d4a0 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1b8267e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335baaef0 0x565335bab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1b8267e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335baaef0 0x565335bab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59600/0 (socket says 172.21.15.28:59600) 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1b8267e640 1 -- 172.21.15.28:0/2807170579 learned_addr learned my addr 172.21.15.28:0/2807170579 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1b8267e640 1 -- 172.21.15.28:0/2807170579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565335bab950 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1b8267e640 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335baaef0 0x565335bab310 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x565335b8a710 tx=0x7f1b7c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1a677fe640 1 -- 172.21.15.28:0/2807170579 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c013640 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.476+0000 7f1a677fe640 1 -- 172.21.15.28:0/2807170579 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7c0137e0 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.477+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565335bac180 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.477+0000 7f1a677fe640 1 -- 172.21.15.28:0/2807170579 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c013ac0 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.477+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565335bac720 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.477+0000 7f1a677fe640 1 -- 172.21.15.28:0/2807170579 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1b7c011040 con 0x565335baaef0 2024-09-19T11:23:23.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1a677fe640 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5c03d000 0x7f1a5c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1a677fe640 1 -- 172.21.15.28:0/2807170579 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f1b7c00e040 con 0x565335baaef0 2024-09-19T11:23:23.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5c03d000 0x7f1a5c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b86137800 1 Processor -- start 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5c03d000 0x7f1a5c03f4c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x565335b97c20 tx=0x7f1b74001460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b86137800 1 -- start start 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b86137800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bb7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b86137800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565335b85ae0 con 0x565335bb7970 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b8267e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bb7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b8267e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bb7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59608/0 (socket says 172.21.15.28:59608) 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.478+0000 7f1b8267e640 1 -- 172.21.15.28:0/3296166230 learned_addr learned my addr 172.21.15.28:0/3296166230 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b8267e640 1 -- 172.21.15.28:0/3296166230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565335bb8360 con 0x565335bb7970 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b8267e640 1 --2- 172.21.15.28:0/3296166230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bb7d70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00afe0 tx=0x7f1b7c0204b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90deb382d63e4d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1a657fa640 1 -- 172.21.15.28:0/3296166230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c044030 con 0x565335bb7970 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1a657fa640 1 -- 172.21.15.28:0/3296166230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7c021a90 con 0x565335bb7970 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1a657fa640 1 -- 172.21.15.28:0/3296166230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c021570 con 0x565335bb7970 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b86137800 1 -- 172.21.15.28:0/3296166230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 msgr2=0x565335bb7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b86137800 1 --2- 172.21.15.28:0/3296166230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bb7d70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00afe0 tx=0x7f1b7c0204b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b86137800 1 -- 172.21.15.28:0/3296166230 shutdown_connections 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b86137800 1 --2- 172.21.15.28:0/3296166230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bb7d70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b86137800 1 -- 172.21.15.28:0/3296166230 >> 172.21.15.28:0/3296166230 conn(0x565335bb3d70 msgr2=0x565335bb1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b86137800 1 -- 172.21.15.28:0/3296166230 shutdown_connections 2024-09-19T11:23:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.479+0000 7f1b86137800 1 -- 172.21.15.28:0/3296166230 wait complete. 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b86137800 1 Processor -- start 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b86137800 1 -- start start 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b86137800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bbef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b86137800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565335bb8980 con 0x565335bb7970 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b8267e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bbef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b8267e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bbef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59618/0 (socket says 172.21.15.28:59618) 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 learned_addr learned my addr 172.21.15.28:0/440484792 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565335bbf4b0 con 0x565335bb7970 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.480+0000 7f1b8267e640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bbef50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c020b60 tx=0x7f1b7c0547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.481+0000 7f1a537fe640 1 -- 172.21.15.28:0/440484792 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c020bd0 con 0x565335bb7970 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.481+0000 7f1a537fe640 1 -- 172.21.15.28:0/440484792 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b7c020d70 con 0x565335bb7970 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.481+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565335bbf740 con 0x565335bb7970 2024-09-19T11:23:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.481+0000 7f1a537fe640 1 -- 172.21.15.28:0/440484792 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c01fa90 con 0x565335bb7970 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.481+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565335bc02d0 con 0x565335bb7970 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.481+0000 7f1a537fe640 1 -- 172.21.15.28:0/440484792 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1b7c01fc30 con 0x565335bb7970 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1a537fe640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4c03d000 0x7f1a4c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1a537fe640 1 -- 172.21.15.28:0/440484792 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f1b7c03d060 con 0x565335bb7970 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4c03d000 0x7f1a4c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335bca8b0 0x565335bccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bcd470 con 0x565335bca8b0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4c03d000 0x7f1a4c03f4c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400efd0 tx=0x7f1b7404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1b8167c640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335bca8b0 0x565335bccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.482+0000 7f1b8167c640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335bca8b0 0x565335bccd70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 realm 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bcd8a0 con 0x565335bca8b0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bce170 con 0x565335bca8b0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bce510 con 0x565335bca8b0 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335bcf270 0x565335bd1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bd1ba0 con 0x565335bcf270 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.483+0000 7f1b8267e640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335bcf270 0x565335bd1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b8267e640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335bcf270 0x565335bd1660 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b86137800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b86137800 20 searching for the correct realm 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b8267e640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b7c03c3f0 0x7f1b7c008970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.484+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b7c03c3f0 0x7f1b7c008970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.485+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b7c03c3f0 0x7f1b7c008970 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.485+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.485+0000 7f1b81e7d640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b7403c070 0x7f1b740542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.485+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.485+0000 7f1b8167c640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b7403c070 0x7f1b740542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b7403c070 0x7f1b740542d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.486+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.487+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.488+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.489+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.490+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.491+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.492+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.492+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bcf270 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.492+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.492+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7403c070 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.492+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.492+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x565335bca8b0 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x565335bd3510 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:23.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bd25e0 con 0x565335bca8b0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bd2e80 con 0x565335bca8b0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.493+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bd2220 con 0x565335bcf270 2024-09-19T11:23:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 zone default found 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 4 Realm: () 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 zonegroup default 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bd2ab0 con 0x7f1b7403c070 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bd5db0 con 0x565335bcf270 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.494+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bd6150 con 0x565335bcf270 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.495+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.495+0000 7f1b86137800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.495+0000 7f1b86137800 20 started sync module instance, tier type = 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.495+0000 7f1b86137800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.495+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd5240 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.496+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.496+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94915383875168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd6a30 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.496+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7280 con 0x565335bca8b0 2024-09-19T11:23:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.497+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94915383875168] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.497+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.497+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94915383881168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd76a0 con 0x565335bca8b0 2024-09-19T11:23:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.497+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7ef0 con 0x565335bcf270 2024-09-19T11:23:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.498+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94915383881168] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.498+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.498+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94915383883504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7fc0 con 0x565335bcf270 2024-09-19T11:23:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.498+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd8980 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.499+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94915383883504] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.499+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.499+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94915383884560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7f30 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.499+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd97f0 con 0x565335bca8b0 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.500+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94915383884560] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.500+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.500+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94915383890752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd9760 con 0x565335bca8b0 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.500+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bda6b0 con 0x7f1b7403c070 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.501+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.501+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94915383894528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bda620 con 0x7f1b7403c070 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.501+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94915383890752] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.501+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bdb5f0 con 0x7f1b7403c070 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.502+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.502+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94915383898304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bdb5b0 con 0x7f1b7403c070 2024-09-19T11:23:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.502+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bdb5b0 con 0x7f1b7403c070 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.502+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94915383894528] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94915383898304] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94915383896736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bdbb80 con 0x7f1b7403c070 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 20 add_watcher() i=0 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 20 add_watcher() i=1 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 20 add_watcher() i=2 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 20 add_watcher() i=3 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 20 add_watcher() i=4 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 20 add_watcher() i=5 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.503+0000 7f1b86137800 20 add_watcher() i=6 2024-09-19T11:23:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.504+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94915383896736] v31'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x7f1b7403c070 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.504+0000 7f1b86137800 20 add_watcher() i=7 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.504+0000 7f1b86137800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.504+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335bdf060 con 0x565335bca8b0 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.505+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.505+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94915383901056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bdc450 con 0x565335bca8b0 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94915383901056] v31'74 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1a337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1b86137800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1b86137800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1b86137800 5 note: GC not initialized 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1a327fc640 20 reqs_thread_entry: start 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1b86137800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f1b86137800 20 Filter name: none 2024-09-19T11:23:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.506+0000 7f19a3fff640 20 reqs_thread_entry: start 2024-09-19T11:23:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.507+0000 7f1b86137800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.507+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x565335c11860 con 0x565335bca8b0 2024-09-19T11:23:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.507+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.507+0000 7f1b86137800 20 rados_obj.operate() r=0 bl.length=556 2024-09-19T11:23:23.522 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:23.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.508+0000 7f1a337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.508+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94915383901056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335c10310 con 0x565335bca8b0 2024-09-19T11:23:23.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.509+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94915383901056] v31'75 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b78057030 con 0x565335bca8b0 2024-09-19T11:23:23.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.509+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94915383875168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335c107c0 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.510+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94915383875168] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.510+0000 7f1b86137800 20 remove_watcher() i=0 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.510+0000 7f1b86137800 2 removed watcher, disabling cache 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.510+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94915383881168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd5660 con 0x565335bca8b0 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.511+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94915383881168] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7807f040 con 0x565335bca8b0 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.511+0000 7f1b86137800 20 remove_watcher() i=1 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.511+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94915383883504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd5660 con 0x565335bcf270 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.512+0000 7f1b8267e640 1 -- 172.21.15.28:0/440484792 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94915383883504] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7c053020 con 0x565335bcf270 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.512+0000 7f1b86137800 20 remove_watcher() i=2 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.512+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94915383884560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd5660 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.513+0000 7f1b81e7d640 1 -- 172.21.15.28:0/440484792 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94915383884560] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b74054040 con 0x7f1b7c03c3f0 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.513+0000 7f1b86137800 20 remove_watcher() i=3 2024-09-19T11:23:23.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.513+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94915383890752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7b50 con 0x565335bca8b0 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.514+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94915383890752] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7807f040 con 0x565335bca8b0 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.514+0000 7f1b86137800 20 remove_watcher() i=4 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.514+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94915383894528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7b50 con 0x7f1b7403c070 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.515+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94915383894528] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7807f040 con 0x7f1b7403c070 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.515+0000 7f1b86137800 20 remove_watcher() i=5 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.515+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94915383898304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7b50 con 0x7f1b7403c070 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.516+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94915383898304] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7807f040 con 0x7f1b7403c070 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.516+0000 7f1b86137800 20 remove_watcher() i=6 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.516+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94915383896736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x565335bd7b50 con 0x7f1b7403c070 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94915383896736] v31'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b7807f040 con 0x7f1b7403c070 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 20 remove_watcher() i=7 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b7403c070 msgr2=0x7f1b740542d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b7403c070 0x7f1b740542d0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335bca8b0 msgr2=0x565335bccd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335bca8b0 0x565335bccd70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b7c03c3f0 msgr2=0x7f1b7c008970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b7c03c3f0 0x7f1b7c008970 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335bcf270 msgr2=0x565335bd1660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335bcf270 0x565335bd1660 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4c03d000 msgr2=0x7f1a4c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4c03d000 0x7f1a4c03f4c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400efd0 tx=0x7f1b7404f000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 msgr2=0x565335bbef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b8167c640 1 -- 172.21.15.28:0/440484792 reap_dead start 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bbef50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c020b60 tx=0x7f1b7c0547f0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 shutdown_connections 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 --2- 172.21.15.28:0/440484792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335bb7970 0x565335bbef50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 >> 172.21.15.28:0/440484792 conn(0x565335bb3d70 msgr2=0x565335bb1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 shutdown_connections 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.517+0000 7f1b86137800 1 -- 172.21.15.28:0/440484792 wait complete. 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5c03d000 msgr2=0x7f1a5c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5c03d000 0x7f1a5c03f4c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x565335b97c20 tx=0x7f1b74001460 comp rx=0 tx=0).stop 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335baaef0 msgr2=0x565335bab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335baaef0 0x565335bab310 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x565335b8a710 tx=0x7f1b7c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 shutdown_connections 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5c03d000 0x7f1a5c03f4c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2807170579 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335baaef0 0x565335bab310 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 >> 172.21.15.28:0/2807170579 conn(0x565335b9f2a0 msgr2=0x565335b9f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 shutdown_connections 2024-09-19T11:23:23.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2807170579 wait complete. 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335a967a0 msgr2=0x565335b97130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335a967a0 0x565335b97130 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335b99aa0 msgr2=0x565335b9be90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335b99aa0 0x565335b9be90 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a6803d1f0 msgr2=0x7f1a6803f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a6803d1f0 0x7f1a6803f6b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1b78057000 tx=0x7f1b78052000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 msgr2=0x565335b8ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b8ab70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1b740012f0 tx=0x7f1b74000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 shutdown_connections 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565335b99aa0 0x565335b9be90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a6803d1f0 0x7f1a6803f6b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565335a967a0 0x565335b97130 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 --2- 172.21.15.28:0/2261585446 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565335a95ba0 0x565335b8ab70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 >> 172.21.15.28:0/2261585446 conn(0x565335a901c0 msgr2=0x565335b865b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 shutdown_connections 2024-09-19T11:23:23.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.518+0000 7f1b86137800 1 -- 172.21.15.28:0/2261585446 wait complete. 2024-09-19T11:23:23.533 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:23.534 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:23.534 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:23.534 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:23.580 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.580 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.579+0000 7fd6a7b40800 1 Processor -- start 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.579+0000 7fd6a7b40800 1 -- start start 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a7b40800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5607608b1820 0x5607608ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a7b40800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5607608ba880 con 0x5607608afd00 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a7b40800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x56076073c7d0 con 0x5607608b1820 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a3085640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5607608b1820 0x5607608ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a3886640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5607608afd00 0x5607608b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33122/0 (socket says 172.21.15.28:33122) 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a3886640 1 -- 172.21.15.28:0/3487690864 learned_addr learned my addr 172.21.15.28:0/3487690864 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a3085640 1 --2- 172.21.15.28:0/3487690864 >> v2:172.21.15.22:3300/0 conn(0x5607608b1820 0x5607608ba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59628/0 (socket says 172.21.15.28:59628) 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a3085640 1 -- 172.21.15.28:0/3487690864 >> v1:172.21.15.22:6789/0 conn(0x5607608afd00 legacy=0x5607608b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.580+0000 7fd6a3085640 1 -- 172.21.15.28:0/3487690864 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607608bba60 con 0x5607608b1820 2024-09-19T11:23:23.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.582+0000 7fd6a3085640 1 --2- 172.21.15.28:0/3487690864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608b1820 0x5607608ba340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd69800a470 tx=0x7fd698045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e158ce0f9778c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a2884640 1 -- 172.21.15.28:0/3487690864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd698053570 con 0x5607608b1820 2024-09-19T11:23:23.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a2884640 1 -- 172.21.15.28:0/3487690864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd698058070 con 0x5607608b1820 2024-09-19T11:23:23.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a2884640 1 -- 172.21.15.28:0/3487690864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd698053880 con 0x5607608b1820 2024-09-19T11:23:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3487690864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608b1820 msgr2=0x5607608ba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3487690864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608b1820 0x5607608ba340 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd69800a470 tx=0x7fd698045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3487690864 shutdown_connections 2024-09-19T11:23:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3487690864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608b1820 0x5607608ba340 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3487690864 >> 172.21.15.28:0/3487690864 conn(0x5607608aba50 msgr2=0x5607608adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3487690864 shutdown_connections 2024-09-19T11:23:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.583+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3487690864 wait complete. 2024-09-19T11:23:23.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.598+0000 7fd6a7b40800 1 Processor -- start 2024-09-19T11:23:23.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.598+0000 7fd6a7b40800 1 -- start start 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.598+0000 7fd6a7b40800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609a34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.598+0000 7fd6a7b40800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560760749dd0 con 0x5607608afd00 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.598+0000 7fd6a3886640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609a34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.599+0000 7fd6a3886640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609a34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59640/0 (socket says 172.21.15.28:59640) 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.599+0000 7fd6a3886640 1 -- 172.21.15.28:0/3597709382 learned_addr learned my addr 172.21.15.28:0/3597709382 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.599+0000 7fd6a3886640 1 -- 172.21.15.28:0/3597709382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607609a3ab0 con 0x5607608afd00 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.599+0000 7fd6a3886640 1 --2- 172.21.15.28:0/3597709382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609a34c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd694001800 tx=0x7fd694001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f5493ab1621f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.599+0000 7fd591e0b640 1 -- 172.21.15.28:0/3597709382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd694004520 con 0x5607608afd00 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.599+0000 7fd591e0b640 1 -- 172.21.15.28:0/3597709382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6940046c0 con 0x5607608afd00 2024-09-19T11:23:23.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.599+0000 7fd591e0b640 1 -- 172.21.15.28:0/3597709382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6940049e0 con 0x5607608afd00 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3597709382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 msgr2=0x5607609a34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3597709382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609a34c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd694001800 tx=0x7fd694001cc0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3597709382 shutdown_connections 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3597709382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609a34c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3597709382 >> 172.21.15.28:0/3597709382 conn(0x5607608aba50 msgr2=0x56076099ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3597709382 shutdown_connections 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3597709382 wait complete. 2024-09-19T11:23:23.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 Processor -- start 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 -- start start 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609ab030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.600+0000 7fd6a7b40800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607608bad50 con 0x5607608afd00 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd6a3886640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd6a3886640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609ab030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59652/0 (socket says 172.21.15.28:59652) 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd6a3886640 1 -- 172.21.15.28:0/3137401936 learned_addr learned my addr 172.21.15.28:0/3137401936 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd6a3886640 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607609ab590 con 0x5607608afd00 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd6a3886640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609ab030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd694000f80 tx=0x7fd694001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd58bfff640 1 -- 172.21.15.28:0/3137401936 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd694022020 con 0x5607608afd00 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd58bfff640 1 -- 172.21.15.28:0/3137401936 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd694014890 con 0x5607608afd00 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607609ab820 con 0x5607608afd00 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd58bfff640 1 -- 172.21.15.28:0/3137401936 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd694014bb0 con 0x5607608afd00 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.601+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607609ac220 con 0x5607608afd00 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.602+0000 7fd58bfff640 1 -- 172.21.15.28:0/3137401936 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd694025020 con 0x5607608afd00 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.602+0000 7fd58bfff640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd58c03d000 0x7fd58c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.602+0000 7fd58bfff640 1 -- 172.21.15.28:0/3137401936 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd694018030 con 0x5607608afd00 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.602+0000 7fd6a3085640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd58c03d000 0x7fd58c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.602+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607608b1820 0x5607609b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.602+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609b2bc0 con 0x5607608b1820 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.603+0000 7fd6a4087640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607608b1820 0x5607609b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.603+0000 7fd6a3085640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd58c03d000 0x7fd58c03f4c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd698057000 tx=0x7fd698052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.603+0000 7fd6a4087640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607608b1820 0x5607609b2680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.603+0000 7fd6a4087640 1 -- 172.21.15.28:0/3137401936 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd69c0083e0 con 0x5607608b1820 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.603+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609b2f60 con 0x5607608b1820 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.603+0000 7fd6a4087640 1 -- 172.21.15.28:0/3137401936 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd69c0083e0 con 0x5607608b1820 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.604+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609b4a60 0x5607609b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.604+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609b7670 con 0x5607609b4a60 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.604+0000 7fd6a3886640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609b4a60 0x5607609b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.604+0000 7fd6a3886640 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609b4a60 0x5607609b6ee0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.604+0000 7fd6a3886640 1 -- 172.21.15.28:0/3137401936 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd69401b070 con 0x5607609b4a60 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.604+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609b37b0 con 0x5607608b1820 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.605+0000 7fd6a4087640 1 -- 172.21.15.28:0/3137401936 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd69c0083e0 con 0x5607608b1820 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.605+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609b3ba0 con 0x5607609b4a60 2024-09-19T11:23:23.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.605+0000 7fd6a3886640 1 -- 172.21.15.28:0/3137401936 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd69401b070 con 0x5607609b4a60 2024-09-19T11:23:23.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.605+0000 7fd6a7b40800 1 Processor -- start 2024-09-19T11:23:23.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.605+0000 7fd6a7b40800 1 -- start start 2024-09-19T11:23:23.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.605+0000 7fd6a7b40800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609c6030 0x5607609c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.605+0000 7fd6a7b40800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607607a84a0 con 0x5607609c6030 2024-09-19T11:23:23.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd6a4087640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609c6030 0x5607609c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd6a4087640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609c6030 0x5607609c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59660/0 (socket says 172.21.15.28:59660) 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd6a4087640 1 -- 172.21.15.28:0/848132506 learned_addr learned my addr 172.21.15.28:0/848132506 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd6a4087640 1 -- 172.21.15.28:0/848132506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607609c6a90 con 0x5607609c6030 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd6a4087640 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609c6030 0x5607609c6450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5607609ac030 tx=0x7fd69c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd5897fa640 1 -- 172.21.15.28:0/848132506 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c01c020 con 0x5607609c6030 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd5897fa640 1 -- 172.21.15.28:0/848132506 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c007c90 con 0x5607609c6030 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607609c72c0 con 0x5607609c6030 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd5897fa640 1 -- 172.21.15.28:0/848132506 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c018440 con 0x5607609c6030 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.606+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607609c7860 con 0x5607609c6030 2024-09-19T11:23:23.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.607+0000 7fd5897fa640 1 -- 172.21.15.28:0/848132506 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd69c01f020 con 0x5607609c6030 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.607+0000 7fd5897fa640 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd57c03d050 0x7fd57c03f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.607+0000 7fd5897fa640 1 -- 172.21.15.28:0/848132506 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd69c055a00 con 0x5607609c6030 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.607+0000 7fd6a3886640 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd57c03d050 0x7fd57c03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a7b40800 1 Processor -- start 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a7b40800 1 -- start start 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a3886640 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd57c03d050 0x7fd57c03f510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5607609b7a40 tx=0x7fd694006820 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a7b40800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a7b40800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607609a3ee0 con 0x5607609d2af0 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a4087640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a4087640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609d2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59672/0 (socket says 172.21.15.28:59672) 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a4087640 1 -- 172.21.15.28:0/1464979731 learned_addr learned my addr 172.21.15.28:0/1464979731 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a4087640 1 -- 172.21.15.28:0/1464979731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607609d34e0 con 0x5607609d2af0 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd6a4087640 1 --2- 172.21.15.28:0/1464979731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609d2ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd69c053d80 tx=0x7fd69c015de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd07703d12990b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.608+0000 7fd582ffd640 1 -- 172.21.15.28:0/1464979731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c015820 con 0x5607609d2af0 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd582ffd640 1 -- 172.21.15.28:0/1464979731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c016c50 con 0x5607609d2af0 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd582ffd640 1 -- 172.21.15.28:0/1464979731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c013320 con 0x5607609d2af0 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 -- 172.21.15.28:0/1464979731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 msgr2=0x5607609d2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/1464979731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609d2ef0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd69c053d80 tx=0x7fd69c015de0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 -- 172.21.15.28:0/1464979731 shutdown_connections 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/1464979731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609d2ef0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 -- 172.21.15.28:0/1464979731 >> 172.21.15.28:0/1464979731 conn(0x5607609ceef0 msgr2=0x5607609cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 -- 172.21.15.28:0/1464979731 shutdown_connections 2024-09-19T11:23:23.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 -- 172.21.15.28:0/1464979731 wait complete. 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 Processor -- start 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 -- start start 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.609+0000 7fd6a7b40800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609da0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a7b40800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607609d3b00 con 0x5607609d2af0 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a4087640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609da0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a4087640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609da0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59674/0 (socket says 172.21.15.28:59674) 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 learned_addr learned my addr 172.21.15.28:0/681543188 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607609da650 con 0x5607609d2af0 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a4087640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609da0f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd69c016010 tx=0x7fd69c053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd5817fa640 1 -- 172.21.15.28:0/681543188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c02e020 con 0x5607609d2af0 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd5817fa640 1 -- 172.21.15.28:0/681543188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd69c013d20 con 0x5607609d2af0 2024-09-19T11:23:23.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607609da8e0 con 0x5607609d2af0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd5817fa640 1 -- 172.21.15.28:0/681543188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd69c0187c0 con 0x5607609d2af0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.610+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607609db260 con 0x5607609d2af0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.611+0000 7fd5817fa640 1 -- 172.21.15.28:0/681543188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd69c031070 con 0x5607609d2af0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd5817fa640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd56c03d000 0x7fd56c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd5817fa640 1 -- 172.21.15.28:0/681543188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd69c03b700 con 0x5607609d2af0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd6a3886640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd56c03d000 0x7fd56c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607609e59c0 0x5607609e7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609e85d0 con 0x5607609e59c0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd6a3085640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607609e59c0 0x5607609e7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd6a3886640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd56c03d000 0x7fd56c03f4c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd694014fd0 tx=0x7fd694052d00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.612+0000 7fd6a3085640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607609e59c0 0x5607609e7e80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 realm 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609e8a00 con 0x5607609e59c0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609e92d0 con 0x5607609e59c0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609e9670 con 0x5607609e59c0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609ea3d0 0x5607609ec7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.613+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609ecd00 con 0x5607609ea3d0 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a4087640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609ea3d0 0x5607609ec7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a4087640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609ea3d0 0x5607609ec7c0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a7b40800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a7b40800 20 searching for the correct realm 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a4087640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd69c021020 0x7fd69c03ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd69c021020 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.614+0000 7fd6a3886640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd69c021020 0x7fd69c03ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.615+0000 7fd6a3886640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd69c021020 0x7fd69c03ca70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.615+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.615+0000 7fd6a3886640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd694027020 0x7fd69401e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.615+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.615+0000 7fd6a3085640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd694027020 0x7fd69401e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.616+0000 7fd6a3085640 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd694027020 0x7fd69401e3f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.616+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.616+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.616+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.616+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.616+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.616+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd69c021020 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.617+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd69c021020 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.618+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.619+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.620+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd69c021020 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.621+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.622+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.622+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd69c021020 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.622+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.622+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609ea3d0 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.622+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.622+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd694027020 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x5607609e59c0 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee670 con 0x7fd69c021020 2024-09-19T11:23:23.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609ed740 con 0x5607609e59c0 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.623+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609ed380 con 0x5607609e59c0 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609ee0a0 con 0x5607609ea3d0 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 zone default found 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 4 Realm: () 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:23.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 zonegroup default 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609edc30 con 0x7fd694027020 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609f0f10 con 0x5607609ea3d0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.624+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609f1320 con 0x5607609ea3d0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.625+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.625+0000 7fd6a7b40800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.625+0000 7fd6a7b40800 20 started sync module instance, tier type = 2024-09-19T11:23:23.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.625+0000 7fd6a7b40800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.625+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f0350 con 0x7fd69c021020 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.626+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.626+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94589685792416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f1780 con 0x7fd69c021020 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.626+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f1fd0 con 0x5607609e59c0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.627+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94589685792416] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.627+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.627+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94589685799712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f23f0 con 0x5607609e59c0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.627+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f2cc0 con 0x5607609ea3d0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.628+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.628+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94589685801968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f2870 con 0x5607609ea3d0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.628+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f3cb0 con 0x7fd69c021020 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.628+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94589685799712] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.629+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94589685801968] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.629+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.629+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94589685805232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f1f40 con 0x7fd69c021020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.629+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f46c0 con 0x5607609e59c0 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.630+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.630+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94589685809680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f4ae0 con 0x5607609e59c0 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.630+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f5530 con 0x7fd694027020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.631+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94589685805232] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.631+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94589685809680] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.631+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.631+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94589685813280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f4630 con 0x7fd694027020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.631+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f5e00 con 0x7fd694027020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.632+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94589685813280] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.632+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.632+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94589685806976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f5c20 con 0x7fd694027020 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.633+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f6470 con 0x7fd694027020 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94589685816624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f67c0 con 0x7fd694027020 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 20 add_watcher() i=0 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 20 add_watcher() i=1 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 20 add_watcher() i=2 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 20 add_watcher() i=3 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 20 add_watcher() i=4 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 20 add_watcher() i=5 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94589685806976] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.634+0000 7fd6a7b40800 20 add_watcher() i=6 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.635+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94589685816624] v31'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd698051070 con 0x7fd694027020 2024-09-19T11:23:23.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.635+0000 7fd6a7b40800 20 add_watcher() i=7 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.635+0000 7fd6a7b40800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.635+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5607609f57a0 con 0x5607609e59c0 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.635+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd698051070 con 0x5607609e59c0 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.635+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94589685817936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f57a0 con 0x5607609e59c0 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.636+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94589685817936] v31'76 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd69807f040 con 0x5607609e59c0 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.636+0000 7fd559fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.636+0000 7fd6a7b40800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.636+0000 7fd6a7b40800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.636+0000 7fd6a7b40800 5 note: GC not initialized 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.636+0000 7fd558fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.636+0000 7fd6a7b40800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.637+0000 7fd6a7b40800 20 Filter name: none 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.637+0000 7fd4d27fc640 20 reqs_thread_entry: start 2024-09-19T11:23:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.637+0000 7fd6a7b40800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.637+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x560760a2c2e0 con 0x5607609e59c0 2024-09-19T11:23:23.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.637+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7fd69807f040 con 0x5607609e59c0 2024-09-19T11:23:23.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.637+0000 7fd6a7b40800 20 rados_obj.operate() r=0 bl.length=556 2024-09-19T11:23:23.651 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.638+0000 7fd559fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.638+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94589685817936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x560760a2add0 con 0x5607609e59c0 2024-09-19T11:23:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.639+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94589685817936] v31'77 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd69807f040 con 0x5607609e59c0 2024-09-19T11:23:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.639+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94589685792416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f6210 con 0x7fd69c021020 2024-09-19T11:23:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.640+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94589685792416] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.640+0000 7fd6a7b40800 20 remove_watcher() i=0 2024-09-19T11:23:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.640+0000 7fd6a7b40800 2 removed watcher, disabling cache 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.640+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94589685799712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609efea0 con 0x5607609e59c0 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.641+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94589685799712] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69807f040 con 0x5607609e59c0 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.641+0000 7fd6a7b40800 20 remove_watcher() i=1 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.641+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94589685801968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609efea0 con 0x5607609ea3d0 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.642+0000 7fd6a4087640 1 -- 172.21.15.28:0/681543188 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94589685801968] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69c030030 con 0x5607609ea3d0 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.642+0000 7fd6a7b40800 20 remove_watcher() i=2 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.642+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94589685805232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f23f0 con 0x7fd69c021020 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.643+0000 7fd6a3886640 1 -- 172.21.15.28:0/681543188 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94589685805232] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69401b070 con 0x7fd69c021020 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.644+0000 7fd6a7b40800 20 remove_watcher() i=3 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.644+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94589685809680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609efea0 con 0x5607609e59c0 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.644+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94589685809680] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69807f040 con 0x5607609e59c0 2024-09-19T11:23:23.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.644+0000 7fd6a7b40800 20 remove_watcher() i=4 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.644+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94589685813280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f30b0 con 0x7fd694027020 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.645+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94589685813280] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69807f040 con 0x7fd694027020 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.645+0000 7fd6a7b40800 20 remove_watcher() i=5 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.645+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94589685806976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f23f0 con 0x7fd694027020 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.646+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94589685806976] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69807f040 con 0x7fd694027020 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.646+0000 7fd6a7b40800 20 remove_watcher() i=6 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.646+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94589685816624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5607609f23f0 con 0x7fd694027020 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.647+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94589685816624] v31'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd69807f040 con 0x7fd694027020 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.647+0000 7fd6a7b40800 20 remove_watcher() i=7 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd694027020 msgr2=0x7fd69401e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd694027020 0x7fd69401e3f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607609e59c0 msgr2=0x5607609e7e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607609e59c0 0x5607609e7e80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd69c021020 msgr2=0x7fd69c03ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd69c021020 0x7fd69c03ca70 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609ea3d0 msgr2=0x5607609ec7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609ea3d0 0x5607609ec7c0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd56c03d000 msgr2=0x7fd56c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd56c03d000 0x7fd56c03f4c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd694014fd0 tx=0x7fd694052d00 comp rx=0 tx=0).stop 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 msgr2=0x5607609da0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/681543188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609d2af0 0x5607609da0f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd69c016010 tx=0x7fd69c053ce0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 reap_dead start 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a3085640 1 -- 172.21.15.28:0/681543188 reap_dead start 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 shutdown_connections 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 >> 172.21.15.28:0/681543188 conn(0x5607609ceef0 msgr2=0x5607609cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 shutdown_connections 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/681543188 wait complete. 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd57c03d050 msgr2=0x7fd57c03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd57c03d050 0x7fd57c03f510 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5607609b7a40 tx=0x7fd694006820 comp rx=0 tx=0).stop 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609c6030 msgr2=0x5607609c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609c6030 0x5607609c6450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5607609ac030 tx=0x7fd69c00bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 shutdown_connections 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd57c03d050 0x7fd57c03f510 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/848132506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607609c6030 0x5607609c6450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 >> 172.21.15.28:0/848132506 conn(0x5607609ba3e0 msgr2=0x5607609ba7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 shutdown_connections 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.648+0000 7fd6a7b40800 1 -- 172.21.15.28:0/848132506 wait complete. 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607608b1820 msgr2=0x5607609b2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607608b1820 0x5607609b2680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609b4a60 msgr2=0x5607609b6ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609b4a60 0x5607609b6ee0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd58c03d000 msgr2=0x7fd58c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd58c03d000 0x7fd58c03f4c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd698057000 tx=0x7fd698052000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 msgr2=0x5607609ab030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609ab030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd694000f80 tx=0x7fd694001250 comp rx=0 tx=0).stop 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 shutdown_connections 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607609b4a60 0x5607609b6ee0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd58c03d000 0x7fd58c03f4c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607608b1820 0x5607609b2680 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 --2- 172.21.15.28:0/3137401936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607608afd00 0x5607609ab030 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 >> 172.21.15.28:0/3137401936 conn(0x5607608aba50 msgr2=0x5607609a49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 shutdown_connections 2024-09-19T11:23:23.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.649+0000 7fd6a7b40800 1 -- 172.21.15.28:0/3137401936 wait complete. 2024-09-19T11:23:23.663 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:23.663 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-09-19T11:23:23.664 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-09-19T11:23:23.664 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-09-19T11:23:23.714 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.714 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.713+0000 7f26bba11800 1 Processor -- start 2024-09-19T11:23:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.713+0000 7f26bba11800 1 -- start start 2024-09-19T11:23:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.713+0000 7f26bba11800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561f19c74d00 0x561f19c77710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.713+0000 7f26bba11800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561f19c78100 con 0x561f19c77c50 2024-09-19T11:23:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.713+0000 7f26bba11800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x561f19b017d0 con 0x561f19c74d00 2024-09-19T11:23:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.714+0000 7f26b7192640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561f19c74d00 0x561f19c77710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.714+0000 7f26b6991640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x561f19c77c50 0x561f19c7c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33136/0 (socket says 172.21.15.28:33136) 2024-09-19T11:23:23.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.714+0000 7f26b6991640 1 -- 172.21.15.28:0/569877227 learned_addr learned my addr 172.21.15.28:0/569877227 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:23.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.714+0000 7f26b7192640 1 -- 172.21.15.28:0/569877227 >> v1:172.21.15.22:6789/0 conn(0x561f19c77c50 legacy=0x561f19c7c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.714+0000 7f26b7192640 1 -- 172.21.15.28:0/569877227 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f19c782d0 con 0x561f19c74d00 2024-09-19T11:23:23.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26b7192640 1 --2- 172.21.15.28:0/569877227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c77710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f26a800a470 tx=0x7f26a8045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197226c944d8ae80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26b6190640 1 -- 172.21.15.28:0/569877227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8053570 con 0x561f19c74d00 2024-09-19T11:23:23.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26b6190640 1 -- 172.21.15.28:0/569877227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a8058070 con 0x561f19c74d00 2024-09-19T11:23:23.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26b6190640 1 -- 172.21.15.28:0/569877227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8053880 con 0x561f19c74d00 2024-09-19T11:23:23.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26bba11800 1 -- 172.21.15.28:0/569877227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 msgr2=0x561f19c77710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26bba11800 1 --2- 172.21.15.28:0/569877227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c77710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f26a800a470 tx=0x7f26a8045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26bba11800 1 -- 172.21.15.28:0/569877227 shutdown_connections 2024-09-19T11:23:23.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26bba11800 1 --2- 172.21.15.28:0/569877227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c77710 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26bba11800 1 -- 172.21.15.28:0/569877227 >> 172.21.15.28:0/569877227 conn(0x561f19c701e0 msgr2=0x561f19c72660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26bba11800 1 -- 172.21.15.28:0/569877227 shutdown_connections 2024-09-19T11:23:23.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.716+0000 7f26bba11800 1 -- 172.21.15.28:0/569877227 wait complete. 2024-09-19T11:23:23.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26bba11800 1 Processor -- start 2024-09-19T11:23:23.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26bba11800 1 -- start start 2024-09-19T11:23:23.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26bba11800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c76e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26bba11800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f19b0edd0 con 0x561f19c74d00 2024-09-19T11:23:23.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26b7192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c76e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26b7192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c76e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59694/0 (socket says 172.21.15.28:59694) 2024-09-19T11:23:23.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26b7192640 1 -- 172.21.15.28:0/874645494 learned_addr learned my addr 172.21.15.28:0/874645494 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.732+0000 7f26b7192640 1 -- 172.21.15.28:0/874645494 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f19c77470 con 0x561f19c74d00 2024-09-19T11:23:23.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f26b7192640 1 --2- 172.21.15.28:0/874645494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c76e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f26a8025c20 tx=0x7f26a8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc84a71d1fdfd9b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f25a560a640 1 -- 172.21.15.28:0/874645494 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a805c580 con 0x561f19c74d00 2024-09-19T11:23:23.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f25a560a640 1 -- 172.21.15.28:0/874645494 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a8063030 con 0x561f19c74d00 2024-09-19T11:23:23.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f25a560a640 1 -- 172.21.15.28:0/874645494 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a805c850 con 0x561f19c74d00 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f26bba11800 1 -- 172.21.15.28:0/874645494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 msgr2=0x561f19c76e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f26bba11800 1 --2- 172.21.15.28:0/874645494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c76e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f26a8025c20 tx=0x7f26a8051000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f26bba11800 1 -- 172.21.15.28:0/874645494 shutdown_connections 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f26bba11800 1 --2- 172.21.15.28:0/874645494 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19c76e80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.733+0000 7f26bba11800 1 -- 172.21.15.28:0/874645494 >> 172.21.15.28:0/874645494 conn(0x561f19c701e0 msgr2=0x561f19d61f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26bba11800 1 -- 172.21.15.28:0/874645494 shutdown_connections 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26bba11800 1 -- 172.21.15.28:0/874645494 wait complete. 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26bba11800 1 Processor -- start 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26bba11800 1 -- start start 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26bba11800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19d70250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26bba11800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f19c7e090 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26b7192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19d70250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26b7192640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19d70250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59710/0 (socket says 172.21.15.28:59710) 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26b7192640 1 -- 172.21.15.28:0/1121582161 learned_addr learned my addr 172.21.15.28:0/1121582161 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.734+0000 7f26b7192640 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f19d707b0 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.735+0000 7f26b7192640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19d70250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f26a8045f50 tx=0x7f26a8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.735+0000 7f25a3e07640 1 -- 172.21.15.28:0/1121582161 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8054ad0 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.735+0000 7f25a3e07640 1 -- 172.21.15.28:0/1121582161 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a8054c70 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.735+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f19d70a40 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.735+0000 7f25a3e07640 1 -- 172.21.15.28:0/1121582161 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26a8068420 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.735+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f19d71440 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f25a3e07640 1 -- 172.21.15.28:0/1121582161 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f26a80685c0 con 0x561f19c74d00 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f25a3e07640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259c03d000 0x7f259c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f25a3e07640 1 -- 172.21.15.28:0/1121582161 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f26a8070030 con 0x561f19c74d00 2024-09-19T11:23:23.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f26b6991640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259c03d000 0x7f259c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19c77c50 0x561f19d778a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19d77de0 con 0x561f19c77c50 2024-09-19T11:23:23.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f26b7993640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19c77c50 0x561f19d778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.736+0000 7f26b6991640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259c03d000 0x7f259c03f4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26ac001d70 tx=0x7f26ac0012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.737+0000 7f26b7993640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19c77c50 0x561f19d778a0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.737+0000 7f26b7993640 1 -- 172.21.15.28:0/1121582161 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19c77c50 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.737+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19d78180 con 0x561f19c77c50 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.737+0000 7f26b7993640 1 -- 172.21.15.28:0/1121582161 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19c77c50 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.737+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19d79c40 0x561f19d7c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.737+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19d7c870 con 0x561f19d79c40 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.737+0000 7f26b7192640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19d79c40 0x561f19d7c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.738+0000 7f26b7192640 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19d79c40 0x561f19d7c0e0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.738+0000 7f26b7192640 1 -- 172.21.15.28:0/1121582161 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f26a8051070 con 0x561f19d79c40 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.738+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19d789d0 con 0x561f19c77c50 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.738+0000 7f26b7993640 1 -- 172.21.15.28:0/1121582161 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19c77c50 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.738+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19d78d70 con 0x561f19d79c40 2024-09-19T11:23:23.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.738+0000 7f26b7192640 1 -- 172.21.15.28:0/1121582161 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f26a8051070 con 0x561f19d79c40 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.739+0000 7f26bba11800 1 Processor -- start 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.739+0000 7f26bba11800 1 -- start start 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.739+0000 7f26bba11800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d8b200 0x561f19d8b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.739+0000 7f26bba11800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f19b6d4a0 con 0x561f19d8b200 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.739+0000 7f26b7993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d8b200 0x561f19d8b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.739+0000 7f26b7993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d8b200 0x561f19d8b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59718/0 (socket says 172.21.15.28:59718) 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.739+0000 7f26b7993640 1 -- 172.21.15.28:0/3429477061 learned_addr learned my addr 172.21.15.28:0/3429477061 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.740+0000 7f26b7993640 1 -- 172.21.15.28:0/3429477061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f19d8bc60 con 0x561f19d8b200 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.740+0000 7f26b7993640 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d8b200 0x561f19d8b620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x561f19d71610 tx=0x7f26b00167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.740+0000 7f25a1602640 1 -- 172.21.15.28:0/3429477061 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b0013640 con 0x561f19d8b200 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.740+0000 7f25a1602640 1 -- 172.21.15.28:0/3429477061 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b00137e0 con 0x561f19d8b200 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.740+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f19d8c490 con 0x561f19d8b200 2024-09-19T11:23:23.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.740+0000 7f25a1602640 1 -- 172.21.15.28:0/3429477061 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b0013b00 con 0x561f19d8b200 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.740+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f19d8ca30 con 0x561f19d8b200 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.741+0000 7f25a1602640 1 -- 172.21.15.28:0/3429477061 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f26b0011040 con 0x561f19d8b200 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.741+0000 7f25a1602640 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259003d0c0 0x7f259003f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.741+0000 7f25a1602640 1 -- 172.21.15.28:0/3429477061 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f26b000e040 con 0x561f19d8b200 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.741+0000 7f26b7192640 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259003d0c0 0x7f259003f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26bba11800 1 Processor -- start 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26bba11800 1 -- start start 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26b7192640 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259003d0c0 0x7f259003f580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x561f19d72350 tx=0x7f26a8096a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26bba11800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d98080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26bba11800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f19c758c0 con 0x561f19d97c80 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26b7993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d98080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26b7993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d98080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59730/0 (socket says 172.21.15.28:59730) 2024-09-19T11:23:23.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26b7993640 1 -- 172.21.15.28:0/211814955 learned_addr learned my addr 172.21.15.28:0/211814955 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26b7993640 1 -- 172.21.15.28:0/211814955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f19d98670 con 0x561f19d97c80 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.742+0000 7f26b7993640 1 --2- 172.21.15.28:0/211814955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d98080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f26b000afe0 tx=0x7f26b001ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398602d73431d719 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f259affd640 1 -- 172.21.15.28:0/211814955 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b0013ca0 con 0x561f19d97c80 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f259affd640 1 -- 172.21.15.28:0/211814955 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b0052070 con 0x561f19d97c80 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f259affd640 1 -- 172.21.15.28:0/211814955 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b002a3f0 con 0x561f19d97c80 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 -- 172.21.15.28:0/211814955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 msgr2=0x561f19d98080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 --2- 172.21.15.28:0/211814955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d98080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f26b000afe0 tx=0x7f26b001ff20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 -- 172.21.15.28:0/211814955 shutdown_connections 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 --2- 172.21.15.28:0/211814955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d98080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 -- 172.21.15.28:0/211814955 >> 172.21.15.28:0/211814955 conn(0x561f19d94080 msgr2=0x561f19d922c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 -- 172.21.15.28:0/211814955 shutdown_connections 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 -- 172.21.15.28:0/211814955 wait complete. 2024-09-19T11:23:23.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 Processor -- start 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.743+0000 7f26bba11800 1 -- start start 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26bba11800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d9f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26bba11800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561f19d98c90 con 0x561f19d97c80 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26b7993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26b7993640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d9f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59740/0 (socket says 172.21.15.28:59740) 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 learned_addr learned my addr 172.21.15.28:0/1112511568 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f19d9f790 con 0x561f19d97c80 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26b7993640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d9f230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f26b0054780 tx=0x7f26b00547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f25997fa640 1 -- 172.21.15.28:0/1112511568 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b0045c00 con 0x561f19d97c80 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f25997fa640 1 -- 172.21.15.28:0/1112511568 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b004de50 con 0x561f19d97c80 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f25997fa640 1 -- 172.21.15.28:0/1112511568 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26b0054960 con 0x561f19d97c80 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f19d9fa20 con 0x561f19d97c80 2024-09-19T11:23:23.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.744+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f19da05b0 con 0x561f19d97c80 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.745+0000 7f25997fa640 1 -- 172.21.15.28:0/1112511568 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f26b0054b00 con 0x561f19d97c80 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.745+0000 7f25997fa640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f258003d000 0x7f258003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.745+0000 7f25997fa640 1 -- 172.21.15.28:0/1112511568 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f26b0004600 con 0x561f19d97c80 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.745+0000 7f26b7192640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f258003d000 0x7f258003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.746+0000 7f26b7192640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f258003d000 0x7f258003f4c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f26a8098010 tx=0x7f26a80745a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.746+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.746+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19daab30 0x561f19dacff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.746+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19dad740 con 0x561f19daab30 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.746+0000 7f26b6991640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19daab30 0x561f19dacff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.746+0000 7f26b6991640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19daab30 0x561f19dacff0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26ac009050 con 0x561f19daab30 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 20 realm 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19da9e80 con 0x561f19daab30 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26ac009050 con 0x561f19daab30 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19dae230 con 0x561f19daab30 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26ac009050 con 0x561f19daab30 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.747+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19daeb00 con 0x561f19daab30 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f26ac009050 con 0x561f19daab30 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26bba11800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19daf520 0x561f19db1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19db2050 con 0x561f19daf520 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26b7993640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19daf520 0x561f19db1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26b7993640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19daf520 0x561f19db1910 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26bba11800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26bba11800 20 searching for the correct realm 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.748+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7993640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26b0021620 0x7f26b00451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26b0021620 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7192640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26b0021620 0x7f26b00451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7192640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26b0021620 0x7f26b00451d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7192640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26a805c070 0x7f26a8009850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.749+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.750+0000 7f26b6991640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26a805c070 0x7f26a8009850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.750+0000 7f26b6991640 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26a805c070 0x7f26a8009850 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.750+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac009050 con 0x7f26a805c070 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.750+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.750+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac009050 con 0x561f19daab30 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26b0021620 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.751+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.752+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.752+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.752+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.752+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.752+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.752+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26b0021620 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.753+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.754+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26b0021620 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.755+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.756+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26b0021620 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daf520 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26a805c070 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x561f19daab30 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.757+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3980 con 0x7f26b0021620 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19db2a10 con 0x561f19daab30 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3260 con 0x561f19daab30 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.758+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19db3e00 con 0x561f19daf520 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 zone default found 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 4 Realm: () 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:23.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 zonegroup default 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19db41a0 con 0x7f26a805c070 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19db63e0 con 0x561f19daf520 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19db6780 con 0x561f19daf520 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.759+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.760+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.760+0000 7f26bba11800 20 started sync module instance, tier type = 2024-09-19T11:23:23.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.760+0000 7f26bba11800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:23.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.760+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db5820 con 0x7f26b0021620 2024-09-19T11:23:23.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.761+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.761+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94691577779056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db6780 con 0x7f26b0021620 2024-09-19T11:23:23.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.761+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db7140 con 0x561f19daab30 2024-09-19T11:23:23.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.762+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94691577779056] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.762+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.762+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94691577785488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db70b0 con 0x561f19daab30 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.762+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db8000 con 0x561f19daf520 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.763+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94691577785488] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.763+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.763+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94691577787696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db8000 con 0x561f19daf520 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.763+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db8a50 con 0x7f26b0021620 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.764+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.764+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94691577787696] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.764+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94691577791808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db92f0 con 0x7f26b0021620 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.765+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db7b50 con 0x561f19daab30 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.766+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94691577791808] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.766+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.766+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94691577794112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19db9260 con 0x561f19daab30 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.766+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19dba3b0 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.767+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.767+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94691577798400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19dba7d0 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.767+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94691577794112] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.767+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19dbb2f0 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.768+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94691577798400] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.768+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.768+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94691577802176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19dba320 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.768+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19dbbbc0 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94691577802176] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94691577796832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19dba320 con 0x7f26a805c070 2024-09-19T11:23:23.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 20 add_watcher() i=0 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 20 add_watcher() i=1 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 20 add_watcher() i=2 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 20 add_watcher() i=3 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 20 add_watcher() i=4 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 20 add_watcher() i=5 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.769+0000 7f26bba11800 20 add_watcher() i=6 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.770+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94691577796832] v31'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.770+0000 7f26bba11800 20 add_watcher() i=7 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.770+0000 7f26bba11800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.770+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19dbebf0 con 0x561f19daab30 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.770+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.770+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94691577804480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19dbc0b0 con 0x561f19daab30 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.771+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94691577804480] v31'78 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.771+0000 7f256e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 5 note: GC not initialized 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f256d7e2640 20 reqs_thread_entry: start 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 20 Filter name: none 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f25667fc640 20 reqs_thread_entry: start 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.772+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19df1a70 con 0x561f19daab30 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 20 rados_obj.operate() r=0 bl.length=556 2024-09-19T11:23:23.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19df2660 con 0x7f26b0021620 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.773+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~673 in=673b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19df8380 con 0x561f19daab30 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.774+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~673] v31'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.774+0000 7f26bba11800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.774+0000 7f26bba11800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.774+0000 7f26bba11800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.774+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94691578051968 in=861b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19df4f50 con 0x561f19daab30 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94691577794112 notify 133143986183 ret 0) ==== 891+0+0 (crc 0 0 0) 0x561f19c78100 con 0x561f19daab30 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f259affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986183 cookie 94691577794112 notifier 4390 bl.length()=849 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f259affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f259affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(65 notify.4 [notify cookie 94691578051968 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f259affd640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f257c003ea0 con 0x561f19daab30 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94691578051968 notify 133143986183 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f26ac02d9d0 con 0x561f19daab30 2024-09-19T11:23:23.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.775+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19df7d80 con 0x7f26b0021620 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v31'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f26bba11800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f26bba11800 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f26bba11800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94691578051968 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561f19df2a80 con 0x7f26b0021620 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== watch-notify(notify (1) cookie 94691577779056 notify 133143986179 ret 0) ==== 208+0+0 (crc 0 0 0) 0x7f26a80b1060 con 0x7f26b0021620 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f259affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986179 cookie 94691577779056 notifier 4390 bl.length()=166 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f259affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f259affd640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.0 [notify cookie 94691578051968 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.776+0000 7f259affd640 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f257c003ea0 con 0x7f26b0021620 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.777+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.777+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== watch-notify(notify_complete (2) cookie 94691578051968 notify 133143986179 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561f19d71440 con 0x7f26b0021620 2024-09-19T11:23:23.788 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-09-19T11:23:23.774256Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.778+0000 7f256e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.778+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94691577804480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x561f19daab30 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.779+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94691577804480] v31'79 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.779+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94691577779056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x7f26b0021620 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.780+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94691577779056] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.780+0000 7f26bba11800 20 remove_watcher() i=0 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.780+0000 7f26bba11800 2 removed watcher, disabling cache 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.780+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94691577785488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x561f19daab30 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.781+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94691577785488] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.781+0000 7f26bba11800 20 remove_watcher() i=1 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.781+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94691577787696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x561f19daf520 2024-09-19T11:23:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.782+0000 7f26b7993640 1 -- 172.21.15.28:0/1112511568 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94691577787696] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26b000a4d0 con 0x561f19daf520 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.782+0000 7f26bba11800 20 remove_watcher() i=2 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.782+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94691577791808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x7f26b0021620 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.783+0000 7f26b7192640 1 -- 172.21.15.28:0/1112511568 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94691577791808] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26a80a1030 con 0x7f26b0021620 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.783+0000 7f26bba11800 20 remove_watcher() i=3 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.783+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94691577794112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x561f19daab30 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.784+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94691577794112] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x561f19daab30 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.784+0000 7f26bba11800 20 remove_watcher() i=4 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.784+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94691577798400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x7f26a805c070 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.785+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94691577798400] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.785+0000 7f26bba11800 20 remove_watcher() i=5 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.785+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94691577802176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x7f26a805c070 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.786+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94691577802176] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.786+0000 7f26bba11800 20 remove_watcher() i=6 2024-09-19T11:23:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.786+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94691577796832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561f19def890 con 0x7f26a805c070 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94691577796832] v31'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26ac029040 con 0x7f26a805c070 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 20 remove_watcher() i=7 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26a805c070 msgr2=0x7f26a8009850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f26a805c070 0x7f26a8009850 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19daab30 msgr2=0x561f19dacff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19daab30 0x561f19dacff0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26b0021620 msgr2=0x7f26b00451d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f26b0021620 0x7f26b00451d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19daf520 msgr2=0x561f19db1910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19daf520 0x561f19db1910 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f258003d000 msgr2=0x7f258003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f258003d000 0x7f258003f4c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f26a8098010 tx=0x7f26a80745a0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 msgr2=0x561f19d9f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 --2- 172.21.15.28:0/1112511568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d97c80 0x561f19d9f230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f26b0054780 tx=0x7f26b00547b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 reap_dead start 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26b6991640 1 -- 172.21.15.28:0/1112511568 reap_dead start 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 shutdown_connections 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 >> 172.21.15.28:0/1112511568 conn(0x561f19d94080 msgr2=0x561f19d92290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 shutdown_connections 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.787+0000 7f26bba11800 1 -- 172.21.15.28:0/1112511568 wait complete. 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259003d0c0 msgr2=0x7f259003f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259003d0c0 0x7f259003f580 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x561f19d72350 tx=0x7f26a8096a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d8b200 msgr2=0x561f19d8b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d8b200 0x561f19d8b620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x561f19d71610 tx=0x7f26b00167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 shutdown_connections 2024-09-19T11:23:23.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259003d0c0 0x7f259003f580 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/3429477061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19d8b200 0x561f19d8b620 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 >> 172.21.15.28:0/3429477061 conn(0x561f19d7f5b0 msgr2=0x561f19d7f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 shutdown_connections 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/3429477061 wait complete. 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19c77c50 msgr2=0x561f19d778a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19c77c50 0x561f19d778a0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19d79c40 msgr2=0x561f19d7c0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19d79c40 0x561f19d7c0e0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259c03d000 msgr2=0x7f259c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259c03d000 0x7f259c03f4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f26ac001d70 tx=0x7f26ac0012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 msgr2=0x561f19d70250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19d70250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f26a8045f50 tx=0x7f26a8002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 shutdown_connections 2024-09-19T11:23:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561f19d79c40 0x561f19d7c0e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f259c03d000 0x7f259c03f4c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561f19c77c50 0x561f19d778a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 --2- 172.21.15.28:0/1121582161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561f19c74d00 0x561f19d70250 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 >> 172.21.15.28:0/1121582161 conn(0x561f19c701e0 msgr2=0x561f19d61ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 shutdown_connections 2024-09-19T11:23:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.788+0000 7f26bba11800 1 -- 172.21.15.28:0/1121582161 wait complete. 2024-09-19T11:23:23.799 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-09-19T11:23:23.774256Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:23.799 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:23:23.799 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:23:23.799 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:23:23.845 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.845 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.844+0000 7fdc89d32800 1 Processor -- start 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.844+0000 7fdc89d32800 1 -- start start 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc89d32800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x556784c5d690 0x556784c5e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc89d32800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556784c5dd40 con 0x556784c5e8d0 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc89d32800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x556784ae97d0 con 0x556784c5d690 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc85ab0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x556784c5d690 0x556784c5e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc852af640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x556784c5e8d0 0x556784c63630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33144/0 (socket says 172.21.15.28:33144) 2024-09-19T11:23:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc852af640 1 -- 172.21.15.28:0/884331382 learned_addr learned my addr 172.21.15.28:0/884331382 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc85ab0640 1 -- 172.21.15.28:0/884331382 >> v1:172.21.15.22:6789/0 conn(0x556784c5e8d0 legacy=0x556784c63630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc84aae640 1 -- 172.21.15.28:0/884331382 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 906000368 0 0) 0x556784c5dd40 con 0x556784c5e8d0 2024-09-19T11:23:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.845+0000 7fdc85ab0640 1 -- 172.21.15.28:0/884331382 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556784c5ed80 con 0x556784c5d690 2024-09-19T11:23:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.847+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/884331382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784c5e390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800a470 tx=0x7fdc78045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16be03ebcf483c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.847+0000 7fdc84aae640 1 -- 172.21.15.28:0/884331382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78053570 con 0x556784c5d690 2024-09-19T11:23:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.847+0000 7fdc84aae640 1 -- 172.21.15.28:0/884331382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78058070 con 0x556784c5d690 2024-09-19T11:23:23.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.847+0000 7fdc84aae640 1 -- 172.21.15.28:0/884331382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78053880 con 0x556784c5d690 2024-09-19T11:23:23.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.848+0000 7fdc89d32800 1 -- 172.21.15.28:0/884331382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 msgr2=0x556784c5e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.848+0000 7fdc89d32800 1 --2- 172.21.15.28:0/884331382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784c5e390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800a470 tx=0x7fdc78045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.848+0000 7fdc89d32800 1 -- 172.21.15.28:0/884331382 shutdown_connections 2024-09-19T11:23:23.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.848+0000 7fdc89d32800 1 --2- 172.21.15.28:0/884331382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784c5e390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.848+0000 7fdc89d32800 1 -- 172.21.15.28:0/884331382 >> 172.21.15.28:0/884331382 conn(0x556784c58a60 msgr2=0x556784c5aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.848+0000 7fdc89d32800 1 -- 172.21.15.28:0/884331382 shutdown_connections 2024-09-19T11:23:23.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.848+0000 7fdc89d32800 1 -- 172.21.15.28:0/884331382 wait complete. 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.863+0000 7fdc89d32800 1 Processor -- start 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.863+0000 7fdc89d32800 1 -- start start 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.863+0000 7fdc89d32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d4e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.863+0000 7fdc89d32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556784af6dd0 con 0x556784c5d690 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.863+0000 7fdc85ab0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d4e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.863+0000 7fdc85ab0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d4e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59758/0 (socket says 172.21.15.28:59758) 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.863+0000 7fdc85ab0640 1 -- 172.21.15.28:0/2196534555 learned_addr learned my addr 172.21.15.28:0/2196534555 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.864+0000 7fdc85ab0640 1 -- 172.21.15.28:0/2196534555 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556784d4e720 con 0x556784c5d690 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.864+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/2196534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d4e130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc78045f50 tx=0x7fdc78051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296d2395d2c27781 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.864+0000 7fdb73e07640 1 -- 172.21.15.28:0/2196534555 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78053c20 con 0x556784c5d690 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.864+0000 7fdb73e07640 1 -- 172.21.15.28:0/2196534555 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78063030 con 0x556784c5d690 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.864+0000 7fdb73e07640 1 -- 172.21.15.28:0/2196534555 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78053440 con 0x556784c5d690 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 -- 172.21.15.28:0/2196534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 msgr2=0x556784d4e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 --2- 172.21.15.28:0/2196534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d4e130 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdc78045f50 tx=0x7fdc78051000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 -- 172.21.15.28:0/2196534555 shutdown_connections 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 --2- 172.21.15.28:0/2196534555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d4e130 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 -- 172.21.15.28:0/2196534555 >> 172.21.15.28:0/2196534555 conn(0x556784c58a60 msgr2=0x556784d49ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 -- 172.21.15.28:0/2196534555 shutdown_connections 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 -- 172.21.15.28:0/2196534555 wait complete. 2024-09-19T11:23:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 Processor -- start 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 -- start start 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d44df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.865+0000 7fdc89d32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556784c64d50 con 0x556784c5d690 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdc85ab0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d44df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdc85ab0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d44df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59772/0 (socket says 172.21.15.28:59772) 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdc85ab0640 1 -- 172.21.15.28:0/4029716488 learned_addr learned my addr 172.21.15.28:0/4029716488 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdc85ab0640 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556784d45350 con 0x556784c5d690 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d44df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc78045f20 tx=0x7fdc78002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdb72604640 1 -- 172.21.15.28:0/4029716488 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78053570 con 0x556784c5d690 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdb72604640 1 -- 172.21.15.28:0/4029716488 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78053710 con 0x556784c5d690 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdb72604640 1 -- 172.21.15.28:0/4029716488 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc78068420 con 0x556784c5d690 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556784d43700 con 0x556784c5d690 2024-09-19T11:23:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.866+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556784d43c40 con 0x556784c5d690 2024-09-19T11:23:23.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.867+0000 7fdb72604640 1 -- 172.21.15.28:0/4029716488 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdc780685c0 con 0x556784c5d690 2024-09-19T11:23:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.867+0000 7fdb72604640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6c03d000 0x7fdb6c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.867+0000 7fdb72604640 1 -- 172.21.15.28:0/4029716488 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fdc78070070 con 0x556784c5d690 2024-09-19T11:23:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc852af640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6c03d000 0x7fdb6c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784c5e8d0 0x556784d5efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d5f570 con 0x556784c5e8d0 2024-09-19T11:23:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc862b1640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784c5e8d0 0x556784d5efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc852af640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6c03d000 0x7fdb6c03f4c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c000c00 tx=0x7fdc7c0013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc862b1640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784c5e8d0 0x556784d5efa0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc862b1640 1 -- 172.21.15.28:0/4029716488 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc8000a4d0 con 0x556784c5e8d0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.868+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d5fe50 con 0x556784c5e8d0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.869+0000 7fdc862b1640 1 -- 172.21.15.28:0/4029716488 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdc8000a4d0 con 0x556784c5e8d0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.869+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d61880 0x556784d63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.869+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d64480 con 0x556784d61880 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.869+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d61880 0x556784d63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.869+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d61880 0x556784d63cf0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.869+0000 7fdc85ab0640 1 -- 172.21.15.28:0/4029716488 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fdc78051070 con 0x556784d61880 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.869+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d606a0 con 0x556784c5e8d0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.870+0000 7fdc862b1640 1 -- 172.21.15.28:0/4029716488 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdc8000a4d0 con 0x556784c5e8d0 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.870+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d60a40 con 0x556784d61880 2024-09-19T11:23:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.870+0000 7fdc85ab0640 1 -- 172.21.15.28:0/4029716488 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fdc78051070 con 0x556784d61880 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.870+0000 7fdc89d32800 1 Processor -- start 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.870+0000 7fdc89d32800 1 -- start start 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.870+0000 7fdc89d32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d72bf0 0x556784d73010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.870+0000 7fdc89d32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556784b554a0 con 0x556784d72bf0 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.871+0000 7fdc862b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d72bf0 0x556784d73010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.871+0000 7fdc862b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d72bf0 0x556784d73010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59774/0 (socket says 172.21.15.28:59774) 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.871+0000 7fdc862b1640 1 -- 172.21.15.28:0/2678892355 learned_addr learned my addr 172.21.15.28:0/2678892355 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.871+0000 7fdc862b1640 1 -- 172.21.15.28:0/2678892355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556784d73650 con 0x556784d72bf0 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.871+0000 7fdc862b1640 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d72bf0 0x556784d73010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x556784d5f970 tx=0x7fdc800163e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.871+0000 7fdb6b7fe640 1 -- 172.21.15.28:0/2678892355 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc80013450 con 0x556784d72bf0 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.871+0000 7fdb6b7fe640 1 -- 172.21.15.28:0/2678892355 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc800135f0 con 0x556784d72bf0 2024-09-19T11:23:23.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.872+0000 7fdb6b7fe640 1 -- 172.21.15.28:0/2678892355 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc800138d0 con 0x556784d72bf0 2024-09-19T11:23:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.872+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556784d73e80 con 0x556784d72bf0 2024-09-19T11:23:23.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.872+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556784d74420 con 0x556784d72bf0 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.872+0000 7fdb6b7fe640 1 -- 172.21.15.28:0/2678892355 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdc80011040 con 0x556784d72bf0 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdb6b7fe640 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6003d000 0x7fdb6003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdb6b7fe640 1 -- 172.21.15.28:0/2678892355 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fdc8000e040 con 0x556784d72bf0 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6003d000 0x7fdb6003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc89d32800 1 Processor -- start 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6003d000 0x7fdb6003f4c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x556784d442d0 tx=0x7fdc7805f890 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc89d32800 1 -- start start 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc89d32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d7fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc89d32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556784d4eaf0 con 0x556784d7f710 2024-09-19T11:23:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc862b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d7fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc862b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d7fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59780/0 (socket says 172.21.15.28:59780) 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.873+0000 7fdc862b1640 1 -- 172.21.15.28:0/3155798332 learned_addr learned my addr 172.21.15.28:0/3155798332 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc862b1640 1 -- 172.21.15.28:0/3155798332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556784d80100 con 0x556784d7f710 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc862b1640 1 --2- 172.21.15.28:0/3155798332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d7fb10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc800298b0 tx=0x7fdc800293a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c2ca3075c7fdd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdb697fa640 1 -- 172.21.15.28:0/3155798332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc80052070 con 0x556784d7f710 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdb697fa640 1 -- 172.21.15.28:0/3155798332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8002aa90 con 0x556784d7f710 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdb697fa640 1 -- 172.21.15.28:0/3155798332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc8002a570 con 0x556784d7f710 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc89d32800 1 -- 172.21.15.28:0/3155798332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 msgr2=0x556784d7fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc89d32800 1 --2- 172.21.15.28:0/3155798332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d7fb10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc800298b0 tx=0x7fdc800293a0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc89d32800 1 -- 172.21.15.28:0/3155798332 shutdown_connections 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc89d32800 1 --2- 172.21.15.28:0/3155798332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d7fb10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc89d32800 1 -- 172.21.15.28:0/3155798332 >> 172.21.15.28:0/3155798332 conn(0x556784d7bb10 msgr2=0x556784d79d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc89d32800 1 -- 172.21.15.28:0/3155798332 shutdown_connections 2024-09-19T11:23:23.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.874+0000 7fdc89d32800 1 -- 172.21.15.28:0/3155798332 wait complete. 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc89d32800 1 Processor -- start 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc89d32800 1 -- start start 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc89d32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d86d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc89d32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556784d80720 con 0x556784d7f710 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc862b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d86d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc862b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d86d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59796/0 (socket says 172.21.15.28:59796) 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 learned_addr learned my addr 172.21.15.28:0/261455917 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556784d87280 con 0x556784d7f710 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.875+0000 7fdc862b1640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d86d20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc8002aa60 tx=0x7fdc80054870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.876+0000 7fdb577fe640 1 -- 172.21.15.28:0/261455917 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc80052030 con 0x556784d7f710 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.876+0000 7fdb577fe640 1 -- 172.21.15.28:0/261455917 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc80053270 con 0x556784d7f710 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.876+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556784d87510 con 0x556784d7f710 2024-09-19T11:23:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.876+0000 7fdb577fe640 1 -- 172.21.15.28:0/261455917 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc80053550 con 0x556784d7f710 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.876+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556784d87e90 con 0x556784d7f710 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.876+0000 7fdb577fe640 1 -- 172.21.15.28:0/261455917 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdc8001fae0 con 0x556784d7f710 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdb577fe640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb5003d000 0x7fdb5003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdb577fe640 1 -- 172.21.15.28:0/261455917 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fdc80053070 con 0x556784d7f710 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb5003d000 0x7fdb5003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784d92670 0x556784d94b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d95280 con 0x556784d92670 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdc852af640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784d92670 0x556784d94b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.877+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb5003d000 0x7fdb5003f4c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800a330 tx=0x7fdc7808a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc852af640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784d92670 0x556784d94b30 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc7c00f5f0 con 0x556784d92670 2024-09-19T11:23:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 20 realm 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d919c0 con 0x556784d92670 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc7c00f5f0 con 0x556784d92670 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d95d60 con 0x556784d92670 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc7c00f5f0 con 0x556784d92670 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.878+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d96630 con 0x556784d92670 2024-09-19T11:23:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdc7c00f5f0 con 0x556784d92670 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc89d32800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d96f30 0x556784d99340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d99aa0 con 0x556784d96f30 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc862b1640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d96f30 0x556784d99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc862b1640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d96f30 0x556784d99340 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc89d32800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc89d32800 20 searching for the correct realm 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.879+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc862b1640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdc80013070 0x7fdc80055950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc80013070 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdc80013070 0x7fdc80055950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdc80013070 0x7fdc80055950 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc85ab0640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdc780a5040 0x7fdc7808f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.880+0000 7fdc852af640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdc780a5040 0x7fdc7808f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.881+0000 7fdc852af640 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdc780a5040 0x7fdc7808f6b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.881+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c00f5f0 con 0x7fdc780a5040 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.881+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.881+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c00f5f0 con 0x556784d92670 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.881+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.882+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x556784d92670 2024-09-19T11:23:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.882+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.882+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x7fdc780a5040 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.882+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.882+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x7fdc780a5040 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.882+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc80013070 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x556784d92670 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x7fdc780a5040 2024-09-19T11:23:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.883+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x556784d92670 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc80013070 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.884+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x7fdc780a5040 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x556784d92670 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fdc7c02b040 con 0x556784d92670 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.885+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x7fdc780a5040 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc80013070 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c02b040 con 0x556784d92670 2024-09-19T11:23:23.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.886+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.887+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.887+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.887+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.887+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.887+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.887+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc80013070 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.888+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.888+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d96f30 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.888+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.888+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc780a5040 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.888+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.888+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x556784d92670 2024-09-19T11:23:23.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x556784d9b480 con 0x7fdc80013070 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d9a550 con 0x556784d92670 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d9adf0 con 0x556784d92670 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.889+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d9a190 con 0x556784d96f30 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 zone default found 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 4 Realm: () 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:23.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 zonegroup default 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d9aa20 con 0x7fdc780a5040 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d9d870 con 0x556784d96f30 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.890+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784d9dde0 con 0x556784d96f30 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.891+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.891+0000 7fdc89d32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:23.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.891+0000 7fdc89d32800 20 started sync module instance, tier type = 2024-09-19T11:23:23.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.891+0000 7fdc89d32800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:23.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.891+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9cce0 con 0x7fdc80013070 2024-09-19T11:23:23.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.892+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.892+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93903098857520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9e220 con 0x7fdc80013070 2024-09-19T11:23:23.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.892+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9ebe0 con 0x556784d92670 2024-09-19T11:23:23.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.893+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.893+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93903098865456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9eb90 con 0x556784d92670 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.893+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9fb90 con 0x556784d96f30 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.893+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93903098857520] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.894+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93903098865456] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.894+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.894+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93903098867664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9f6a0 con 0x556784d96f30 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.894+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da0460 con 0x7fdc80013070 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.895+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93903098867664] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.895+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.895+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93903098859952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9f6a0 con 0x7fdc80013070 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.895+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da0b30 con 0x556784d92670 2024-09-19T11:23:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.896+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.896+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93903098873344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da0b30 con 0x556784d92670 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.896+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da1b30 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.896+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93903098859952] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.897+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.897+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93903098877472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da1af0 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.897+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93903098873344] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.897+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da0f40 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.898+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93903098877472] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.898+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.898+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93903098875712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da20c0 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.898+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da2a80 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93903098875712] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93903098871728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da2a80 con 0x7fdc780a5040 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 20 add_watcher() i=0 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 20 add_watcher() i=1 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 20 add_watcher() i=2 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 20 add_watcher() i=3 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 20 add_watcher() i=4 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 20 add_watcher() i=5 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.899+0000 7fdc89d32800 20 add_watcher() i=6 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.900+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93903098871728] v31'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.900+0000 7fdc89d32800 20 add_watcher() i=7 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.900+0000 7fdc89d32800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.900+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784da5d80 con 0x556784d92670 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.901+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.901+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93903098881856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da3010 con 0x556784d92670 2024-09-19T11:23:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.902+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93903098881856] v31'80 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.902+0000 7fdb377fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.902+0000 7fdc89d32800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.902+0000 7fdc89d32800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.902+0000 7fdc89d32800 5 note: GC not initialized 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.902+0000 7fdb367fc640 20 reqs_thread_entry: start 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.903+0000 7fdc89d32800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.903+0000 7fdc89d32800 20 Filter name: none 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.903+0000 7fdaa7fff640 20 reqs_thread_entry: start 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.903+0000 7fdc89d32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.903+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x556784dd88a0 con 0x556784d92670 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.903+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.903+0000 7fdc89d32800 20 rados_obj.operate() r=0 bl.length=673 2024-09-19T11:23:23.917 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-09-19T11:23:23.774256Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.904+0000 7fdb377fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.904+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93903098881856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784dd8cb0 con 0x556784d92670 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.905+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93903098881856] v31'81 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.905+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93903098857520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da2740 con 0x7fdc80013070 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.906+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93903098857520] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.906+0000 7fdc89d32800 20 remove_watcher() i=0 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.906+0000 7fdc89d32800 2 removed watcher, disabling cache 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.906+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93903098865456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da2740 con 0x556784d92670 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.907+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93903098865456] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.907+0000 7fdc89d32800 20 remove_watcher() i=1 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.907+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93903098867664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9e730 con 0x556784d96f30 2024-09-19T11:23:23.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.908+0000 7fdc862b1640 1 -- 172.21.15.28:0/261455917 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93903098867664] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc80027020 con 0x556784d96f30 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.909+0000 7fdc89d32800 20 remove_watcher() i=2 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.909+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93903098859952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da2740 con 0x7fdc80013070 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.909+0000 7fdc85ab0640 1 -- 172.21.15.28:0/261455917 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93903098859952] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc78051070 con 0x7fdc80013070 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.909+0000 7fdc89d32800 20 remove_watcher() i=3 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.909+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93903098873344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9e730 con 0x556784d92670 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.910+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93903098873344] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x556784d92670 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.910+0000 7fdc89d32800 20 remove_watcher() i=4 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.910+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93903098877472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da2740 con 0x7fdc780a5040 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.911+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93903098877472] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.911+0000 7fdc89d32800 20 remove_watcher() i=5 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.911+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93903098875712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784d9e730 con 0x7fdc780a5040 2024-09-19T11:23:23.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.912+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93903098875712] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.912+0000 7fdc89d32800 20 remove_watcher() i=6 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.912+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93903098871728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x556784da1620 con 0x7fdc780a5040 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93903098871728] v31'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc7c03b040 con 0x7fdc780a5040 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc89d32800 20 remove_watcher() i=7 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdc780a5040 msgr2=0x7fdc7808f6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdc780a5040 0x7fdc7808f6b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784d92670 msgr2=0x556784d94b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784d92670 0x556784d94b30 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdc80013070 msgr2=0x7fdc80055950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.913+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdc80013070 0x7fdc80055950 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d96f30 msgr2=0x556784d99340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d96f30 0x556784d99340 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb5003d000 msgr2=0x7fdb5003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb5003d000 0x7fdb5003f4c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800a330 tx=0x7fdc7808a000 comp rx=0 tx=0).stop 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 msgr2=0x556784d86d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 --2- 172.21.15.28:0/261455917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d7f710 0x556784d86d20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc8002aa60 tx=0x7fdc80054870 comp rx=0 tx=0).stop 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 reap_dead start 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc852af640 1 -- 172.21.15.28:0/261455917 reap_dead start 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 shutdown_connections 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 >> 172.21.15.28:0/261455917 conn(0x556784d7bb10 msgr2=0x556784d79d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 shutdown_connections 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/261455917 wait complete. 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6003d000 msgr2=0x7fdb6003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6003d000 0x7fdb6003f4c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x556784d442d0 tx=0x7fdc7805f890 comp rx=0 tx=0).stop 2024-09-19T11:23:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d72bf0 msgr2=0x556784d73010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d72bf0 0x556784d73010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x556784d5f970 tx=0x7fdc800163e0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 shutdown_connections 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6003d000 0x7fdb6003f4c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 --2- 172.21.15.28:0/2678892355 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784d72bf0 0x556784d73010 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 >> 172.21.15.28:0/2678892355 conn(0x556784d66fa0 msgr2=0x556784d67380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 shutdown_connections 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.914+0000 7fdc89d32800 1 -- 172.21.15.28:0/2678892355 wait complete. 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784c5e8d0 msgr2=0x556784d5efa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784c5e8d0 0x556784d5efa0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d61880 msgr2=0x556784d63cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d61880 0x556784d63cf0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6c03d000 msgr2=0x7fdb6c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6c03d000 0x7fdb6c03f4c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c000c00 tx=0x7fdc7c0013b0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 msgr2=0x556784d44df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d44df0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdc78045f20 tx=0x7fdc78002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 shutdown_connections 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556784d61880 0x556784d63cf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdb6c03d000 0x7fdb6c03f4c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556784c5e8d0 0x556784d5efa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 --2- 172.21.15.28:0/4029716488 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556784c5d690 0x556784d44df0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 >> 172.21.15.28:0/4029716488 conn(0x556784c58a60 msgr2=0x556784d4f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 shutdown_connections 2024-09-19T11:23:23.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.915+0000 7fdc89d32800 1 -- 172.21.15.28:0/4029716488 wait complete. 2024-09-19T11:23:23.928 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-09-19T11:23:23.774256Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:23.928 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-09-19T11:23:23.929 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-09-19T11:23:23.929 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-09-19T11:23:23.976 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:23.977 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:23.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73d3210800 1 Processor -- start 2024-09-19T11:23:23.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73d3210800 1 -- start start 2024-09-19T11:23:23.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73d3210800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a4a8733560 0x55a4a87c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73d3210800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a4a87c78c0 con 0x55a4a8732a50 2024-09-19T11:23:23.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73d3210800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55a4a86497d0 con 0x55a4a8733560 2024-09-19T11:23:23.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73c7fff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a4a8733560 0x55a4a87c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73ce9e0640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55a4a8732a50 0x55a4a8732e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33156/0 (socket says 172.21.15.28:33156) 2024-09-19T11:23:23.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.976+0000 7f73ce9e0640 1 -- 172.21.15.28:0/3039969666 learned_addr learned my addr 172.21.15.28:0/3039969666 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:23.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.977+0000 7f73c7fff640 1 -- 172.21.15.28:0/3039969666 >> v1:172.21.15.22:6789/0 conn(0x55a4a8732a50 legacy=0x55a4a8732e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.977+0000 7f73ce1df640 1 -- 172.21.15.28:0/3039969666 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880904404 0 0) 0x55a4a87c78c0 con 0x55a4a8732a50 2024-09-19T11:23:23.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.977+0000 7f73c7fff640 1 -- 172.21.15.28:0/3039969666 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a87c8aa0 con 0x55a4a8733560 2024-09-19T11:23:23.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73c7fff640 1 --2- 172.21.15.28:0/3039969666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8733560 0x55a4a87c7380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73b800a470 tx=0x7f73b8045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279541e951dff440 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73ce1df640 1 -- 172.21.15.28:0/3039969666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b8053570 con 0x55a4a8733560 2024-09-19T11:23:23.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73ce1df640 1 -- 172.21.15.28:0/3039969666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b8058070 con 0x55a4a8733560 2024-09-19T11:23:23.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73ce1df640 1 -- 172.21.15.28:0/3039969666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73b8053880 con 0x55a4a8733560 2024-09-19T11:23:23.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73d3210800 1 -- 172.21.15.28:0/3039969666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8733560 msgr2=0x55a4a87c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:23.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73d3210800 1 --2- 172.21.15.28:0/3039969666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8733560 0x55a4a87c7380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73b800a470 tx=0x7f73b8045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:23.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73d3210800 1 -- 172.21.15.28:0/3039969666 shutdown_connections 2024-09-19T11:23:23.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73d3210800 1 --2- 172.21.15.28:0/3039969666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8733560 0x55a4a87c7380 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:23.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73d3210800 1 -- 172.21.15.28:0/3039969666 >> 172.21.15.28:0/3039969666 conn(0x55a4a87b9c70 msgr2=0x55a4a87bc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:23.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73d3210800 1 -- 172.21.15.28:0/3039969666 shutdown_connections 2024-09-19T11:23:23.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.979+0000 7f73d3210800 1 -- 172.21.15.28:0/3039969666 wait complete. 2024-09-19T11:23:23.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.994+0000 7f73d3210800 1 Processor -- start 2024-09-19T11:23:23.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73d3210800 1 -- start start 2024-09-19T11:23:23.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73d3210800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:23.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73d3210800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4a8656dd0 con 0x55a4a8732a50 2024-09-19T11:23:23.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73ce9e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:23.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73ce9e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59814/0 (socket says 172.21.15.28:59814) 2024-09-19T11:23:23.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73ce9e0640 1 -- 172.21.15.28:0/1303474725 learned_addr learned my addr 172.21.15.28:0/1303474725 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:23.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73ce9e0640 1 -- 172.21.15.28:0/1303474725 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a88b0870 con 0x55a4a8732a50 2024-09-19T11:23:23.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.995+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/1303474725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b0280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f73c0001ea0 tx=0x7f73c000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e391e240eef623b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:23.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f72bce09640 1 -- 172.21.15.28:0/1303474725 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c0008070 con 0x55a4a8732a50 2024-09-19T11:23:23.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f72bce09640 1 -- 172.21.15.28:0/1303474725 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c000d500 con 0x55a4a8732a50 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f72bce09640 1 -- 172.21.15.28:0/1303474725 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c000d880 con 0x55a4a8732a50 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f73d3210800 1 -- 172.21.15.28:0/1303474725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 msgr2=0x55a4a88b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f73d3210800 1 --2- 172.21.15.28:0/1303474725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b0280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f73c0001ea0 tx=0x7f73c000b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f73d3210800 1 -- 172.21.15.28:0/1303474725 shutdown_connections 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f73d3210800 1 --2- 172.21.15.28:0/1303474725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b0280 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f73d3210800 1 -- 172.21.15.28:0/1303474725 >> 172.21.15.28:0/1303474725 conn(0x55a4a87b9c70 msgr2=0x55a4a88abc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f73d3210800 1 -- 172.21.15.28:0/1303474725 shutdown_connections 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.996+0000 7f73d3210800 1 -- 172.21.15.28:0/1303474725 wait complete. 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73d3210800 1 Processor -- start 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73d3210800 1 -- start start 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73d3210800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73d3210800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4a87c7d90 con 0x55a4a8732a50 2024-09-19T11:23:24.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73ce9e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73ce9e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59828/0 (socket says 172.21.15.28:59828) 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73ce9e0640 1 -- 172.21.15.28:0/2673721919 learned_addr learned my addr 172.21.15.28:0/2673721919 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.997+0000 7f73ce9e0640 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a88b8330 con 0x55a4a8732a50 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.998+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b7dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f73c00012f0 tx=0x7f73c0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.998+0000 7f72bb606640 1 -- 172.21.15.28:0/2673721919 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c000f9d0 con 0x55a4a8732a50 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.998+0000 7f72bb606640 1 -- 172.21.15.28:0/2673721919 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c000fb70 con 0x55a4a8732a50 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.998+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4a88b85c0 con 0x55a4a8732a50 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.998+0000 7f72bb606640 1 -- 172.21.15.28:0/2673721919 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c000fe50 con 0x55a4a8732a50 2024-09-19T11:23:24.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.998+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a88b8fc0 con 0x55a4a8732a50 2024-09-19T11:23:24.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f72bb606640 1 -- 172.21.15.28:0/2673721919 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f73c0017040 con 0x55a4a8732a50 2024-09-19T11:23:24.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f72bb606640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72b403d1a0 0x7f72b403f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f72bb606640 1 -- 172.21.15.28:0/2673721919 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f73c0015020 con 0x55a4a8732a50 2024-09-19T11:23:24.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f73c7fff640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72b403d1a0 0x7f72b403f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a8733560 0x55a4a88bf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88bf960 con 0x55a4a8733560 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a8733560 0x55a4a88bf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:23.999+0000 7f73c7fff640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72b403d1a0 0x7f72b403f660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73b8057000 tx=0x7f73b8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.000+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a8733560 0x55a4a88bf420 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.000+0000 7f73cf1e1640 1 -- 172.21.15.28:0/2673721919 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73c800a4d0 con 0x55a4a8733560 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.000+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88bfd00 con 0x55a4a8733560 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.000+0000 7f73cf1e1640 1 -- 172.21.15.28:0/2673721919 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f73c800a4d0 con 0x55a4a8733560 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.000+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88c1800 0x55a4a88c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.000+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88c4410 con 0x55a4a88c1800 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.000+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88c1800 0x55a4a88c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.001+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88c1800 0x55a4a88c3c80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.001+0000 7f73ce9e0640 1 -- 172.21.15.28:0/2673721919 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f73c002d070 con 0x55a4a88c1800 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.001+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88c0550 con 0x55a4a8733560 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.001+0000 7f73cf1e1640 1 -- 172.21.15.28:0/2673721919 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f73c800a4d0 con 0x55a4a8733560 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.001+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88c09a0 con 0x55a4a88c1800 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73ce9e0640 1 -- 172.21.15.28:0/2673721919 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f73c002d070 con 0x55a4a88c1800 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73d3210800 1 Processor -- start 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73d3210800 1 -- start start 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73d3210800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88d2d80 0x55a4a88d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73d3210800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4a86b54a0 con 0x55a4a88d2d80 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73cf1e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88d2d80 0x55a4a88d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73cf1e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88d2d80 0x55a4a88d31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59844/0 (socket says 172.21.15.28:59844) 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.002+0000 7f73cf1e1640 1 -- 172.21.15.28:0/1427267071 learned_addr learned my addr 172.21.15.28:0/1427267071 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.003+0000 7f73cf1e1640 1 -- 172.21.15.28:0/1427267071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a88d37e0 con 0x55a4a88d2d80 2024-09-19T11:23:24.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.003+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88d2d80 0x55a4a88d31a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55a4a88b8dd0 tx=0x7f73c80167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.003+0000 7f72b8e01640 1 -- 172.21.15.28:0/1427267071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c8013640 con 0x55a4a88d2d80 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.003+0000 7f72b8e01640 1 -- 172.21.15.28:0/1427267071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c80137e0 con 0x55a4a88d2d80 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.003+0000 7f72b8e01640 1 -- 172.21.15.28:0/1427267071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c8013ae0 con 0x55a4a88d2d80 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.003+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4a88d4010 con 0x55a4a88d2d80 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.003+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a88d45b0 con 0x55a4a88d2d80 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.004+0000 7f72b8e01640 1 -- 172.21.15.28:0/1427267071 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f73c8011040 con 0x55a4a88d2d80 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.004+0000 7f72b8e01640 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72a803d000 0x7f72a803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.004+0000 7f72b8e01640 1 -- 172.21.15.28:0/1427267071 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f73c800e040 con 0x55a4a88d2d80 2024-09-19T11:23:24.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.004+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72a803d000 0x7f72a803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.004+0000 7f73d3210800 1 Processor -- start 2024-09-19T11:23:24.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.004+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72a803d000 0x7f72a803f4c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73c00531e0 tx=0x7f73c00011b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73d3210800 1 -- start start 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73d3210800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88dfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73d3210800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4a88b0ca0 con 0x55a4a88df7e0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73cf1e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88dfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73cf1e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88dfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59858/0 (socket says 172.21.15.28:59858) 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73cf1e1640 1 -- 172.21.15.28:0/4010912486 learned_addr learned my addr 172.21.15.28:0/4010912486 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73cf1e1640 1 -- 172.21.15.28:0/4010912486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a88e01d0 con 0x55a4a88df7e0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/4010912486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88dfbe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f73c80547f0 tx=0x7f73c80206e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60727e5077bd1e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.005+0000 7f72b27fc640 1 -- 172.21.15.28:0/4010912486 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c8044030 con 0x55a4a88df7e0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f72b27fc640 1 -- 172.21.15.28:0/4010912486 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c8021a90 con 0x55a4a88df7e0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f72b27fc640 1 -- 172.21.15.28:0/4010912486 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c801f470 con 0x55a4a88df7e0 2024-09-19T11:23:24.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 -- 172.21.15.28:0/4010912486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 msgr2=0x55a4a88dfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 --2- 172.21.15.28:0/4010912486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88dfbe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f73c80547f0 tx=0x7f73c80206e0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 -- 172.21.15.28:0/4010912486 shutdown_connections 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 --2- 172.21.15.28:0/4010912486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88dfbe0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 -- 172.21.15.28:0/4010912486 >> 172.21.15.28:0/4010912486 conn(0x55a4a88dbc00 msgr2=0x55a4a88d9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 -- 172.21.15.28:0/4010912486 shutdown_connections 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 -- 172.21.15.28:0/4010912486 wait complete. 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 Processor -- start 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.006+0000 7f73d3210800 1 -- start start 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73d3210800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88e6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73d3210800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4a88e07f0 con 0x55a4a88df7e0 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73cf1e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88e6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73cf1e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88e6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59870/0 (socket says 172.21.15.28:59870) 2024-09-19T11:23:24.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 learned_addr learned my addr 172.21.15.28:0/929058572 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a88e7370 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88e6e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73c8021a60 tx=0x7f73c804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f72b0ff9640 1 -- 172.21.15.28:0/929058572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c8021570 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f72b0ff9640 1 -- 172.21.15.28:0/929058572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c8056070 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4a88e7600 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f72b0ff9640 1 -- 172.21.15.28:0/929058572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73c80459c0 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.007+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a88e7600 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.008+0000 7f72b0ff9640 1 -- 172.21.15.28:0/929058572 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f73c8045b60 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.008+0000 7f72b0ff9640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f729803d000 0x7f729803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.008+0000 7f72b0ff9640 1 -- 172.21.15.28:0/929058572 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f73c8017c10 con 0x55a4a88df7e0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.009+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f729803d000 0x7f729803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.009+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f729803d000 0x7f729803f4c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73c000efd0 tx=0x7f73c0041000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.009+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.009+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a88f2510 0x55a4a88f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.009+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88f5120 con 0x55a4a88f2510 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.009+0000 7f73c7fff640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a88f2510 0x55a4a88f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.009+0000 7f73c7fff640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a88f2510 0x55a4a88f49d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 realm 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88f5550 con 0x55a4a88f2510 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88f5e20 con 0x55a4a88f2510 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88f61c0 con 0x55a4a88f2510 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88f6f20 0x55a4a88f9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.010+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88f9920 con 0x55a4a88f6f20 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88f6f20 0x55a4a88f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88f6f20 0x55a4a88f9310 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73d3210800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73d3210800 20 searching for the correct realm 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73cf1e1640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f73c8017450 0x7f73c8017eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.011+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c8017450 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.012+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f73c8017450 0x7f73c8017eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.012+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f73c8017450 0x7f73c8017eb0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.012+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.012+0000 7f73ce9e0640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f73c0053530 0x7f73c0053970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.012+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.012+0000 7f73c7fff640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f73c0053530 0x7f73c0053970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.013+0000 7f73c7fff640 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f73c0053530 0x7f73c0053970 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.013+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.013+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.013+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.013+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.013+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.013+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c8017450 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.014+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c8017450 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.015+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.016+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.017+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c8017450 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.018+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.019+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.019+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.019+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c8017450 2024-09-19T11:23:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.019+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.019+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f6f20 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.019+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.019+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c0053530 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x55a4a88f2510 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb250 con 0x7f73c8017450 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fa310 con 0x55a4a88f2510 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.020+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fab60 con 0x55a4a88f2510 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fb6d0 con 0x55a4a88f6f20 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 zone default found 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 4 Realm: () 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 zonegroup default 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fba70 con 0x7f73c0053530 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fd8a0 con 0x55a4a88f6f20 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.021+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fdd60 con 0x55a4a88f6f20 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.022+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.022+0000 7f73d3210800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.022+0000 7f73d3210800 20 started sync module instance, tier type = 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.022+0000 7f73d3210800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.022+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fcd10 con 0x7f73c8017450 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.023+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.023+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94165690994944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fe670 con 0x7f73c8017450 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.023+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88feec0 con 0x55a4a88f2510 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.024+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.024+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94165690994944] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.024+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94165691001360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88ff440 con 0x55a4a88f2510 2024-09-19T11:23:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.024+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88fe1e0 con 0x55a4a88f6f20 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.025+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94165691001360] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.025+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.025+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94165691002512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a88ff400 con 0x55a4a88f6f20 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.025+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8900500 con 0x7f73c8017450 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.026+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94165691002512] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.026+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.026+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94165691007056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a89008f0 con 0x7f73c8017450 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.026+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8901320 con 0x55a4a88f2510 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.027+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.027+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94165691010608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8901bf0 con 0x55a4a88f2510 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.027+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8902690 con 0x7f73c0053530 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.028+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94165691007056] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.028+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.028+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94165691015552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8902650 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.028+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8903110 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.028+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94165691010608] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.029+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94165691015552] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.029+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.029+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94165691005488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8902c20 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.029+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a89035e0 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.030+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94165691005488] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94165691012928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a89035e0 con 0x7f73c0053530 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 20 add_watcher() i=0 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 20 add_watcher() i=1 2024-09-19T11:23:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 20 add_watcher() i=2 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 20 add_watcher() i=3 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 20 add_watcher() i=4 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 20 add_watcher() i=5 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73d3210800 20 add_watcher() i=6 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.031+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94165691012928] v31'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x7f73c0053530 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.032+0000 7f73d3210800 20 add_watcher() i=7 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.032+0000 7f73d3210800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.032+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8906920 con 0x55a4a88f2510 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.032+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.032+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94165691019936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8903b70 con 0x55a4a88f2510 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.033+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94165691019936] v31'82 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f73b8051070 con 0x55a4a88f2510 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.033+0000 7f72857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.033+0000 7f73d3210800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:24.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.033+0000 7f73d3210800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.033+0000 7f73d3210800 5 note: GC not initialized 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.033+0000 7f71fffff640 20 reqs_thread_entry: start 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 20 Filter name: none 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f71fdffb640 20 reqs_thread_entry: start 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8939790 con 0x55a4a88f2510 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 20 rados_obj.operate() r=0 bl.length=673 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.034+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8940210 con 0x55a4a88f2510 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~582] v31'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73d3210800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73d3210800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73d3210800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94165691268176 in=770b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a893b8f0 con 0x55a4a88f2510 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94165691010608 notify 133143986184 ret 0) ==== 800+0+0 (crc 0 0 0) 0x55a4a87c8aa0 con 0x55a4a88f2510 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f72b27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986184 cookie 94165691010608 notifier 4404 bl.length()=758 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f72b27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(64 notify.4 [notify cookie 94165691268176 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f72b27fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f72b27fc640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f7294003ea0 con 0x55a4a88f2510 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94165691268176 notify 133143986184 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f73b805fc40 con 0x55a4a88f2510 2024-09-19T11:23:24.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.036+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a893fc50 con 0x55a4a88f2510 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.037+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(66 foo:foo1 [delete] v31'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.037+0000 7f73d3210800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.037+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94165691268176 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55a4a893b440 con 0x7f73c8017450 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.038+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== watch-notify(notify (1) cookie 94165690994944 notify 133143986180 ret 0) ==== 201+0+0 (crc 0 0 0) 0x55a4a88b8fc0 con 0x7f73c8017450 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.038+0000 7f72b27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986180 cookie 94165690994944 notifier 4404 bl.length()=159 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.038+0000 7f72b27fc640 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f7294003ea0 con 0x7f73c8017450 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.038+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.0 [notify cookie 94165691268176 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.038+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== watch-notify(notify_complete (2) cookie 94165691268176 notify 133143986180 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a4a88b85c0 con 0x7f73c8017450 2024-09-19T11:23:24.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.038+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.050 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-09-19T11:23:23.774256Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.039+0000 7f72857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.039+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94165691019936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x55a4a88f2510 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.040+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94165691019936] v31'83 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.040+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94165690994944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x7f73c8017450 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.041+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94165690994944] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.041+0000 7f73d3210800 20 remove_watcher() i=0 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.041+0000 7f73d3210800 2 removed watcher, disabling cache 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.041+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94165691001360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x55a4a88f2510 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.042+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94165691001360] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.042+0000 7f73d3210800 20 remove_watcher() i=1 2024-09-19T11:23:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.042+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94165691002512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x55a4a88f6f20 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.043+0000 7f73cf1e1640 1 -- 172.21.15.28:0/929058572 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94165691002512] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c8053020 con 0x55a4a88f6f20 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.043+0000 7f73d3210800 20 remove_watcher() i=2 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.043+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94165691007056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x7f73c8017450 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.044+0000 7f73ce9e0640 1 -- 172.21.15.28:0/929058572 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94165691007056] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73c0039070 con 0x7f73c8017450 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.044+0000 7f73d3210800 20 remove_watcher() i=3 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.044+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94165691010608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x55a4a88f2510 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.045+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94165691010608] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x55a4a88f2510 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.045+0000 7f73d3210800 20 remove_watcher() i=4 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.045+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94165691015552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x7f73c0053530 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.046+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94165691015552] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x7f73c0053530 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.046+0000 7f73d3210800 20 remove_watcher() i=5 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.047+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94165691005488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x7f73c0053530 2024-09-19T11:23:24.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.047+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94165691005488] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x7f73c0053530 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.047+0000 7f73d3210800 20 remove_watcher() i=6 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.047+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94165691012928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55a4a8937630 con 0x7f73c0053530 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.048+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94165691012928] v31'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f73b807f040 con 0x7f73c0053530 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.048+0000 7f73d3210800 20 remove_watcher() i=7 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f73c0053530 msgr2=0x7f73c0053970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f73c0053530 0x7f73c0053970 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a88f2510 msgr2=0x55a4a88f49d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a88f2510 0x55a4a88f49d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f73c8017450 msgr2=0x7f73c8017eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f73c8017450 0x7f73c8017eb0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88f6f20 msgr2=0x55a4a88f9310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88f6f20 0x55a4a88f9310 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f729803d000 msgr2=0x7f729803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f729803d000 0x7f729803f4c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73c000efd0 tx=0x7f73c0041000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 msgr2=0x55a4a88e6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/929058572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88df7e0 0x55a4a88e6e10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73c8021a60 tx=0x7f73c804e000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 reap_dead start 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73c7fff640 1 -- 172.21.15.28:0/929058572 reap_dead start 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 shutdown_connections 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 >> 172.21.15.28:0/929058572 conn(0x55a4a88dbc00 msgr2=0x55a4a88d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 shutdown_connections 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/929058572 wait complete. 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72a803d000 msgr2=0x7f72a803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72a803d000 0x7f72a803f4c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f73c00531e0 tx=0x7f73c00011b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88d2d80 msgr2=0x55a4a88d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88d2d80 0x55a4a88d31a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55a4a88b8dd0 tx=0x7f73c80167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 shutdown_connections 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72a803d000 0x7f72a803f4c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 --2- 172.21.15.28:0/1427267071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a88d2d80 0x55a4a88d31a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 >> 172.21.15.28:0/1427267071 conn(0x55a4a88c7130 msgr2=0x55a4a88c7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 shutdown_connections 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.049+0000 7f73d3210800 1 -- 172.21.15.28:0/1427267071 wait complete. 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a8733560 msgr2=0x55a4a88bf420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a8733560 0x55a4a88bf420 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88c1800 msgr2=0x55a4a88c3c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88c1800 0x55a4a88c3c80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72b403d1a0 msgr2=0x7f72b403f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72b403d1a0 0x7f72b403f660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f73b8057000 tx=0x7f73b8052000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 msgr2=0x55a4a88b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b7dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f73c00012f0 tx=0x7f73c0000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 shutdown_connections 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4a88c1800 0x55a4a88c3c80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f72b403d1a0 0x7f72b403f660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4a8733560 0x55a4a88bf420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 --2- 172.21.15.28:0/2673721919 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4a8732a50 0x55a4a88b7dd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 >> 172.21.15.28:0/2673721919 conn(0x55a4a87b9c70 msgr2=0x55a4a88b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 shutdown_connections 2024-09-19T11:23:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.050+0000 7f73d3210800 1 -- 172.21.15.28:0/2673721919 wait complete. 2024-09-19T11:23:24.061 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-09-19T11:23:23.774256Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:24.061 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-09-19T11:23:24.061 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-09-19T11:23:24.061 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-09-19T11:23:24.107 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:24.107 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:24.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.106+0000 7f06f7939800 1 Processor -- start 2024-09-19T11:23:24.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.106+0000 7f06f7939800 1 -- start start 2024-09-19T11:23:24.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f7939800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5653ee658860 0x5653ee661380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f7939800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5653ee6618c0 con 0x5653ee656d40 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f7939800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5653ee4e37d0 con 0x5653ee658860 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f2e24640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5653ee658860 0x5653ee661380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f3625640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5653ee656d40 0x5653ee657140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33166/0 (socket says 172.21.15.28:33166) 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f3625640 1 -- 172.21.15.28:0/3381104918 learned_addr learned my addr 172.21.15.28:0/3381104918 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f2e24640 1 -- 172.21.15.28:0/3381104918 >> v1:172.21.15.22:6789/0 conn(0x5653ee656d40 legacy=0x5653ee657140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f2e24640 1 -- 172.21.15.28:0/3381104918 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ee662aa0 con 0x5653ee658860 2024-09-19T11:23:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.107+0000 7f06f2623640 1 -- 172.21.15.28:0/3381104918 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541842493 0 0) 0x5653ee6618c0 con 0x5653ee656d40 2024-09-19T11:23:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.109+0000 7f06f2e24640 1 --2- 172.21.15.28:0/3381104918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee658860 0x5653ee661380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00a470 tx=0x7f06dc045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48535d9390fcd7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.109+0000 7f06f2623640 1 -- 172.21.15.28:0/3381104918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06dc053570 con 0x5653ee658860 2024-09-19T11:23:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.109+0000 7f06f2623640 1 -- 172.21.15.28:0/3381104918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc058070 con 0x5653ee658860 2024-09-19T11:23:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.109+0000 7f06f2623640 1 -- 172.21.15.28:0/3381104918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06dc053880 con 0x5653ee658860 2024-09-19T11:23:24.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.110+0000 7f06f7939800 1 -- 172.21.15.28:0/3381104918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee658860 msgr2=0x5653ee661380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.110+0000 7f06f7939800 1 --2- 172.21.15.28:0/3381104918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee658860 0x5653ee661380 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00a470 tx=0x7f06dc045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:24.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.110+0000 7f06f7939800 1 -- 172.21.15.28:0/3381104918 shutdown_connections 2024-09-19T11:23:24.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.110+0000 7f06f7939800 1 --2- 172.21.15.28:0/3381104918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee658860 0x5653ee661380 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.110+0000 7f06f7939800 1 -- 172.21.15.28:0/3381104918 >> 172.21.15.28:0/3381104918 conn(0x5653ee652a50 msgr2=0x5653ee654eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.110+0000 7f06f7939800 1 -- 172.21.15.28:0/3381104918 shutdown_connections 2024-09-19T11:23:24.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.110+0000 7f06f7939800 1 -- 172.21.15.28:0/3381104918 wait complete. 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.125+0000 7f06f7939800 1 Processor -- start 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.125+0000 7f06f7939800 1 -- start start 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.125+0000 7f06f7939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee74a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.125+0000 7f06f7939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653ee4f0dd0 con 0x5653ee656d40 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.125+0000 7f06f3625640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee74a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.125+0000 7f06f3625640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee74a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59894/0 (socket says 172.21.15.28:59894) 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.125+0000 7f06f3625640 1 -- 172.21.15.28:0/922109631 learned_addr learned my addr 172.21.15.28:0/922109631 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.126+0000 7f06f3625640 1 -- 172.21.15.28:0/922109631 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ee74a990 con 0x5653ee656d40 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.126+0000 7f06f3625640 1 --2- 172.21.15.28:0/922109631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee74a3a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f06ec001ea0 tx=0x7f06ec00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310f3808f8138339 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.126+0000 7f05e160a640 1 -- 172.21.15.28:0/922109631 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec008070 con 0x5653ee656d40 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.126+0000 7f05e160a640 1 -- 172.21.15.28:0/922109631 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ec00d500 con 0x5653ee656d40 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.126+0000 7f05e160a640 1 -- 172.21.15.28:0/922109631 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec00d880 con 0x5653ee656d40 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 -- 172.21.15.28:0/922109631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 msgr2=0x5653ee74a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 --2- 172.21.15.28:0/922109631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee74a3a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f06ec001ea0 tx=0x7f06ec00b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 -- 172.21.15.28:0/922109631 shutdown_connections 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 --2- 172.21.15.28:0/922109631 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee74a3a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 -- 172.21.15.28:0/922109631 >> 172.21.15.28:0/922109631 conn(0x5653ee652a50 msgr2=0x5653ee745d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 -- 172.21.15.28:0/922109631 shutdown_connections 2024-09-19T11:23:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 -- 172.21.15.28:0/922109631 wait complete. 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 Processor -- start 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 -- start start 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee751e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.127+0000 7f06f7939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653ee661d90 con 0x5653ee656d40 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f06f3625640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee751e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f06f3625640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee751e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59898/0 (socket says 172.21.15.28:59898) 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f06f3625640 1 -- 172.21.15.28:0/3226425241 learned_addr learned my addr 172.21.15.28:0/3226425241 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f06f3625640 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ee7523f0 con 0x5653ee656d40 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f06f3625640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee751e90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0012f0 tx=0x7f06ec000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f05dfe07640 1 -- 172.21.15.28:0/3226425241 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec00f9d0 con 0x5653ee656d40 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f05dfe07640 1 -- 172.21.15.28:0/3226425241 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ec00fb70 con 0x5653ee656d40 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f05dfe07640 1 -- 172.21.15.28:0/3226425241 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06ec00fe50 con 0x5653ee656d40 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.128+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653ee752680 con 0x5653ee656d40 2024-09-19T11:23:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.129+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ee753080 con 0x5653ee656d40 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.129+0000 7f05dfe07640 1 -- 172.21.15.28:0/3226425241 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f06ec017040 con 0x5653ee656d40 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f05dfe07640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05d803d1f0 0x7f05d803f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f05dfe07640 1 -- 172.21.15.28:0/3226425241 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f06ec015020 con 0x5653ee656d40 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f06f2e24640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05d803d1f0 0x7f05d803f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee658860 0x5653ee7594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee759a20 con 0x5653ee658860 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f06f3e26640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee658860 0x5653ee7594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f06f2e24640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05d803d1f0 0x7f05d803f6b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f06dc057000 tx=0x7f06dc052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.130+0000 7f06f3e26640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee658860 0x5653ee7594e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.131+0000 7f06f3e26640 1 -- 172.21.15.28:0/3226425241 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f06e400a4d0 con 0x5653ee658860 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.131+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee759dc0 con 0x5653ee658860 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.131+0000 7f06f3e26640 1 -- 172.21.15.28:0/3226425241 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f06e400a4d0 con 0x5653ee658860 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.131+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee75b8c0 0x5653ee75dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.131+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee75e4d0 con 0x5653ee75b8c0 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.131+0000 7f06f3625640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee75b8c0 0x5653ee75dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.131+0000 7f06f3625640 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee75b8c0 0x5653ee75dd40 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.132+0000 7f06f3625640 1 -- 172.21.15.28:0/3226425241 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f06ec054040 con 0x5653ee75b8c0 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.132+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee75a610 con 0x5653ee658860 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.132+0000 7f06f3e26640 1 -- 172.21.15.28:0/3226425241 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f06e400a4d0 con 0x5653ee658860 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.132+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee75aa60 con 0x5653ee75b8c0 2024-09-19T11:23:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.132+0000 7f06f3625640 1 -- 172.21.15.28:0/3226425241 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f06ec054040 con 0x5653ee75b8c0 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.132+0000 7f06f7939800 1 Processor -- start 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f7939800 1 -- start start 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f7939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee76ce60 0x5653ee76d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f7939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653ee54f4a0 con 0x5653ee76ce60 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f3e26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee76ce60 0x5653ee76d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f3e26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee76ce60 0x5653ee76d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59904/0 (socket says 172.21.15.28:59904) 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f3e26640 1 -- 172.21.15.28:0/3215162630 learned_addr learned my addr 172.21.15.28:0/3215162630 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f3e26640 1 -- 172.21.15.28:0/3215162630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ee76d8c0 con 0x5653ee76ce60 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.133+0000 7f06f3e26640 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee76ce60 0x5653ee76d280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5653ee752e90 tx=0x7f06e40167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.134+0000 7f05dd602640 1 -- 172.21.15.28:0/3215162630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e4013640 con 0x5653ee76ce60 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.134+0000 7f05dd602640 1 -- 172.21.15.28:0/3215162630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e40137e0 con 0x5653ee76ce60 2024-09-19T11:23:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.134+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653ee76e0f0 con 0x5653ee76ce60 2024-09-19T11:23:24.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.134+0000 7f05dd602640 1 -- 172.21.15.28:0/3215162630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e4013ac0 con 0x5653ee76ce60 2024-09-19T11:23:24.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.134+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ee76e690 con 0x5653ee76ce60 2024-09-19T11:23:24.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.134+0000 7f05dd602640 1 -- 172.21.15.28:0/3215162630 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f06e4011040 con 0x5653ee76ce60 2024-09-19T11:23:24.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f05dd602640 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05cc03d000 0x7f05cc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f05dd602640 1 -- 172.21.15.28:0/3215162630 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f06e400e040 con 0x5653ee76ce60 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f3625640 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05cc03d000 0x7f05cc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f3625640 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05cc03d000 0x7f05cc03f4c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00ecb0 tx=0x7f06ec001460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f7939800 1 Processor -- start 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f7939800 1 -- start start 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f7939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee779ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f7939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653ee74adc0 con 0x5653ee7798a0 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f3e26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee779ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f3e26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee779ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59908/0 (socket says 172.21.15.28:59908) 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.135+0000 7f06f3e26640 1 -- 172.21.15.28:0/1848758469 learned_addr learned my addr 172.21.15.28:0/1848758469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f3e26640 1 -- 172.21.15.28:0/1848758469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ee77a290 con 0x5653ee7798a0 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f3e26640 1 --2- 172.21.15.28:0/1848758469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee779ca0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06e400afe0 tx=0x7f06e40204b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834d677c9056f1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f05d6ffd640 1 -- 172.21.15.28:0/1848758469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e4044030 con 0x5653ee7798a0 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f05d6ffd640 1 -- 172.21.15.28:0/1848758469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e4021a90 con 0x5653ee7798a0 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f05d6ffd640 1 -- 172.21.15.28:0/1848758469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e4021570 con 0x5653ee7798a0 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f7939800 1 -- 172.21.15.28:0/1848758469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 msgr2=0x5653ee779ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f7939800 1 --2- 172.21.15.28:0/1848758469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee779ca0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06e400afe0 tx=0x7f06e40204b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f7939800 1 -- 172.21.15.28:0/1848758469 shutdown_connections 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f7939800 1 --2- 172.21.15.28:0/1848758469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee779ca0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f7939800 1 -- 172.21.15.28:0/1848758469 >> 172.21.15.28:0/1848758469 conn(0x5653ee775ce0 msgr2=0x5653ee773f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f7939800 1 -- 172.21.15.28:0/1848758469 shutdown_connections 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.136+0000 7f06f7939800 1 -- 172.21.15.28:0/1848758469 wait complete. 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f7939800 1 Processor -- start 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f7939800 1 -- start start 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f7939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee780ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f7939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653ee77a8b0 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f3e26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee780ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f3e26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee780ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59910/0 (socket says 172.21.15.28:59910) 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 learned_addr learned my addr 172.21.15.28:0/3519112063 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ee781430 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.137+0000 7f06f3e26640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee780ed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f06e400afe0 tx=0x7f06e40547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.138+0000 7f05d57fa640 1 -- 172.21.15.28:0/3519112063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e4020de0 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.138+0000 7f05d57fa640 1 -- 172.21.15.28:0/3519112063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e401fa90 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.138+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653ee7816c0 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.138+0000 7f05d57fa640 1 -- 172.21.15.28:0/3519112063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06e401fd70 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.138+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ee782250 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.138+0000 7f05d57fa640 1 -- 172.21.15.28:0/3519112063 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f06e401f500 con 0x5653ee7798a0 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f05d57fa640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05bc03cfb0 0x7f05bc03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f05d57fa640 1 -- 172.21.15.28:0/3519112063 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f06e4034ca0 con 0x5653ee7798a0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f06f3625640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05bc03cfb0 0x7f05bc03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f06f3625640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05bc03cfb0 0x7f05bc03f470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00bbf0 tx=0x7f06ec04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee78c780 0x5653ee78ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee78f390 con 0x5653ee78c780 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.139+0000 7f06f2e24640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee78c780 0x5653ee78ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f2e24640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee78c780 0x5653ee78ec40 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 20 realm 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee78f7c0 con 0x5653ee78c780 2024-09-19T11:23:24.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.140+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee790090 con 0x5653ee78c780 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f7939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee790430 con 0x5653ee78c780 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f7939800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee791190 0x5653ee793580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee793b90 con 0x5653ee791190 2024-09-19T11:23:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f3e26640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee791190 0x5653ee793580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.141+0000 7f06f3e26640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee791190 0x5653ee793580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f7939800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f7939800 20 searching for the correct realm 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x5653ee791190 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f3e26640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f06e403c3f0 0x7f06e403d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x7f06e403c3f0 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f3625640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f06e403c3f0 0x7f06e403d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.142+0000 7f06f3625640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f06e403c3f0 0x7f06e403d7f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.143+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.143+0000 7f06f3625640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f06ec03c070 0x7f06ec0542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.143+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x7f06ec03c070 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.143+0000 7f06f2e24640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f06ec03c070 0x7f06ec0542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.143+0000 7f06f2e24640 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f06ec03c070 0x7f06ec0542d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.143+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.143+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x5653ee78c780 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x5653ee78c780 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x7f06ec03c070 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x7f06ec03c070 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.144+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x7f06e403c3f0 2024-09-19T11:23:24.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x5653ee791190 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x5653ee78c780 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x7f06ec03c070 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.145+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x5653ee78c780 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7954c0 con 0x7f06e403c3f0 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee791190 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee791190 2024-09-19T11:23:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.146+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06ec03c070 2024-09-19T11:23:24.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee791190 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee791190 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee78c780 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee78c780 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.147+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06ec03c070 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee791190 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06e403c3f0 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee78c780 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.148+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06ec03c070 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.149+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.149+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee791190 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.149+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.149+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06ec03c070 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.149+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.149+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06e403c3f0 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.149+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.150+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee791190 2024-09-19T11:23:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.150+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.150+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06ec03c070 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.150+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.150+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x5653ee78c780 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.150+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.150+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f06ec061060 con 0x7f06e403c3f0 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee794580 con 0x5653ee78c780 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee794dd0 con 0x5653ee78c780 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee795940 con 0x5653ee791190 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.151+0000 7f06f7939800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 zone default found 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 4 Realm: () 2024-09-19T11:23:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 zonegroup default 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee795d40 con 0x7f06ec03c070 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee798020 con 0x5653ee791190 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7983c0 con 0x5653ee791190 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 started sync module instance, tier type = 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.152+0000 7f06f7939800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.153+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7974b0 con 0x7f06e403c3f0 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.154+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.154+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94918483212496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee797110 con 0x7f06e403c3f0 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.154+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7992c0 con 0x5653ee78c780 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.155+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94918483212496] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.155+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.155+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94918483217936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7996e0 con 0x5653ee78c780 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.155+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79a0a0 con 0x5653ee791190 2024-09-19T11:23:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.156+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94918483217936] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.156+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.156+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94918483219472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79a0a0 con 0x5653ee791190 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.156+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79aae0 con 0x7f06e403c3f0 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.157+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94918483219472] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.157+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.157+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94918483221488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79a010 con 0x7f06e403c3f0 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.157+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79b9a0 con 0x5653ee78c780 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.158+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94918483221488] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.159+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.159+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94918483226320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79b9a0 con 0x5653ee78c780 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.159+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79c3f0 con 0x7f06ec03c070 2024-09-19T11:23:24.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.160+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94918483226320] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.160+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.160+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94918483227888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79c3b0 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.160+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79ce80 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.161+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94918483227888] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.161+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.161+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94918483230464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79c320 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.161+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79d910 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94918483230464] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94918483235936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79d880 con 0x7f06ec03c070 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 20 add_watcher() i=0 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 20 add_watcher() i=1 2024-09-19T11:23:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 20 add_watcher() i=2 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 20 add_watcher() i=3 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 20 add_watcher() i=4 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 20 add_watcher() i=5 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.162+0000 7f06f7939800 20 add_watcher() i=6 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.163+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94918483235936] v31'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.163+0000 7f06f7939800 20 add_watcher() i=7 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.163+0000 7f06f7939800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.163+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7a0cb0 con 0x5653ee78c780 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.163+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.163+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94918483234368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee79e170 con 0x5653ee78c780 2024-09-19T11:23:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.164+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94918483234368] v31'84 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.164+0000 7f05a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 5 note: GC not initialized 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f05a8fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 20 Filter name: none 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f05227fc640 20 reqs_thread_entry: start 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.165+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d3b20 con 0x5653ee78c780 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.166+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~582 out=582b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+705 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.166+0000 7f06f7939800 20 rados_obj.operate() r=0 bl.length=582 2024-09-19T11:23:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.166+0000 7f06f7939800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.166+0000 7f06f7939800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.166+0000 7f06f7939800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.166+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7da000 con 0x5653ee78c780 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v31'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f06f7939800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f06f7939800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f06f7939800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94918483483136 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d5970 con 0x5653ee78c780 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94918483226320 notify 133143986185 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7f06dc081040 con 0x5653ee78c780 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(64 notify.4 [notify cookie 94918483483136 out=8b] v0'0 uv75 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f05d6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986185 cookie 94918483226320 notifier 4411 bl.length()=732 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f05d6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f05d6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.167+0000 7f05d6ffd640 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f05b8003ea0 con 0x5653ee78c780 2024-09-19T11:23:24.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.168+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== watch-notify(notify_complete (2) cookie 94918483483136 notify 133143986185 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5653ee662aa0 con 0x5653ee78c780 2024-09-19T11:23:24.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.168+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.178 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-09-19T11:23:23.774256Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.169+0000 7f05a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.169+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94918483234368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x5653ee78c780 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.170+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94918483234368] v31'85 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.170+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94918483212496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x7f06e403c3f0 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.171+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94918483212496] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.171+0000 7f06f7939800 20 remove_watcher() i=0 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.171+0000 7f06f7939800 2 removed watcher, disabling cache 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.171+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94918483217936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x5653ee78c780 2024-09-19T11:23:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.172+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94918483217936] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.172+0000 7f06f7939800 20 remove_watcher() i=1 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.172+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94918483219472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x5653ee791190 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.173+0000 7f06f3e26640 1 -- 172.21.15.28:0/3519112063 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94918483219472] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06e4053020 con 0x5653ee791190 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.173+0000 7f06f7939800 20 remove_watcher() i=2 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.173+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94918483221488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x7f06e403c3f0 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.174+0000 7f06f3625640 1 -- 172.21.15.28:0/3519112063 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94918483221488] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06ec054040 con 0x7f06e403c3f0 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.174+0000 7f06f7939800 20 remove_watcher() i=3 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.174+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94918483226320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x5653ee78c780 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.175+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94918483226320] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x5653ee78c780 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.175+0000 7f06f7939800 20 remove_watcher() i=4 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.175+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94918483227888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x7f06ec03c070 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.176+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94918483227888] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.176+0000 7f06f7939800 20 remove_watcher() i=5 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.176+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94918483230464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x7f06ec03c070 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.177+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94918483230464] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.177+0000 7f06f7939800 20 remove_watcher() i=6 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.177+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94918483235936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5653ee7d19c0 con 0x7f06ec03c070 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94918483235936] v31'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06dc051070 con 0x7f06ec03c070 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 20 remove_watcher() i=7 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f06ec03c070 msgr2=0x7f06ec0542d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f06ec03c070 0x7f06ec0542d0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee78c780 msgr2=0x5653ee78ec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee78c780 0x5653ee78ec40 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f06e403c3f0 msgr2=0x7f06e403d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f06e403c3f0 0x7f06e403d7f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee791190 msgr2=0x5653ee793580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee791190 0x5653ee793580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05bc03cfb0 msgr2=0x7f05bc03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05bc03cfb0 0x7f05bc03f470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00bbf0 tx=0x7f06ec04f000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 msgr2=0x5653ee780ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f2e24640 1 -- 172.21.15.28:0/3519112063 reap_dead start 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee780ed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f06e400afe0 tx=0x7f06e40547f0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 shutdown_connections 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 --2- 172.21.15.28:0/3519112063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee7798a0 0x5653ee780ed0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 >> 172.21.15.28:0/3519112063 conn(0x5653ee775ce0 msgr2=0x5653ee773ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 shutdown_connections 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.178+0000 7f06f7939800 1 -- 172.21.15.28:0/3519112063 wait complete. 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05cc03d000 msgr2=0x7f05cc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05cc03d000 0x7f05cc03f4c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00ecb0 tx=0x7f06ec001460 comp rx=0 tx=0).stop 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee76ce60 msgr2=0x5653ee76d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee76ce60 0x5653ee76d280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5653ee752e90 tx=0x7f06e40167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 shutdown_connections 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05cc03d000 0x7f05cc03f4c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3215162630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee76ce60 0x5653ee76d280 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 >> 172.21.15.28:0/3215162630 conn(0x5653ee761210 msgr2=0x5653ee7615f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 shutdown_connections 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3215162630 wait complete. 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee658860 msgr2=0x5653ee7594e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee658860 0x5653ee7594e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee75b8c0 msgr2=0x5653ee75dd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee75b8c0 0x5653ee75dd40 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05d803d1f0 msgr2=0x7f05d803f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05d803d1f0 0x7f05d803f6b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f06dc057000 tx=0x7f06dc052000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 msgr2=0x5653ee751e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee751e90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0012f0 tx=0x7f06ec000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 shutdown_connections 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653ee75b8c0 0x5653ee75dd40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f05d803d1f0 0x7f05d803f6b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653ee658860 0x5653ee7594e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 --2- 172.21.15.28:0/3226425241 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653ee656d40 0x5653ee751e90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 >> 172.21.15.28:0/3226425241 conn(0x5653ee652a50 msgr2=0x5653ee74b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 shutdown_connections 2024-09-19T11:23:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.179+0000 7f06f7939800 1 -- 172.21.15.28:0/3226425241 wait complete. 2024-09-19T11:23:24.190 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-09-19T11:23:23.774256Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:24.190 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-09-19T11:23:24.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-09-19T11:23:24.190 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-09-19T11:23:24.239 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:24.239 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:24.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.238+0000 7ff0a0020800 1 Processor -- start 2024-09-19T11:23:24.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.238+0000 7ff0a0020800 1 -- start start 2024-09-19T11:23:24.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.238+0000 7ff0a0020800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55845495e8b0 0x55845495f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.238+0000 7ff0a0020800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55845495ef60 con 0x55845495f9b0 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff0a0020800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5584547e97d0 con 0x55845495e8b0 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff09db0a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55845495e8b0 0x55845495f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff09db0a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55845495e8b0 0x55845495f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59916/0 (socket says 172.21.15.28:59916) 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff09db0a640 1 -- 172.21.15.28:0/371291353 learned_addr learned my addr 172.21.15.28:0/371291353 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff09d309640 1 --1- 172.21.15.28:0/371291353 >> v1:172.21.15.22:6789/0 conn(0x55845495f9b0 0x558454964710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33182/0 (socket says 172.21.15.28:33182) 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff09db0a640 1 -- 172.21.15.28:0/371291353 >> v1:172.21.15.22:6789/0 conn(0x55845495f9b0 legacy=0x558454964710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff09db0a640 1 -- 172.21.15.28:0/371291353 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845495fe60 con 0x55845495e8b0 2024-09-19T11:23:24.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.239+0000 7ff09cb08640 1 -- 172.21.15.28:0/371291353 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383511420 0 0) 0x55845495ef60 con 0x55845495f9b0 2024-09-19T11:23:24.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.241+0000 7ff09db0a640 1 --2- 172.21.15.28:0/371291353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x55845495f470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff09800a520 tx=0x7ff098045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd73e2b0508bc29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.241+0000 7ff09cb08640 1 -- 172.21.15.28:0/371291353 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff098055070 con 0x55845495e8b0 2024-09-19T11:23:24.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.241+0000 7ff09cb08640 1 -- 172.21.15.28:0/371291353 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0980515b0 con 0x55845495e8b0 2024-09-19T11:23:24.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.241+0000 7ff09cb08640 1 -- 172.21.15.28:0/371291353 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0980518d0 con 0x55845495e8b0 2024-09-19T11:23:24.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.242+0000 7ff0a0020800 1 -- 172.21.15.28:0/371291353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 msgr2=0x55845495f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.242+0000 7ff0a0020800 1 --2- 172.21.15.28:0/371291353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x55845495f470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff09800a520 tx=0x7ff098045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.242+0000 7ff0a0020800 1 -- 172.21.15.28:0/371291353 shutdown_connections 2024-09-19T11:23:24.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.242+0000 7ff0a0020800 1 --2- 172.21.15.28:0/371291353 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x55845495f470 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.242+0000 7ff0a0020800 1 -- 172.21.15.28:0/371291353 >> 172.21.15.28:0/371291353 conn(0x558454959c80 msgr2=0x55845495c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.242+0000 7ff0a0020800 1 -- 172.21.15.28:0/371291353 shutdown_connections 2024-09-19T11:23:24.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.242+0000 7ff0a0020800 1 -- 172.21.15.28:0/371291353 wait complete. 2024-09-19T11:23:24.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.257+0000 7ff0a0020800 1 Processor -- start 2024-09-19T11:23:24.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.257+0000 7ff0a0020800 1 -- start start 2024-09-19T11:23:24.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.257+0000 7ff0a0020800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x5584548d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.257+0000 7ff0a0020800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5584547f6dd0 con 0x55845495e8b0 2024-09-19T11:23:24.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.257+0000 7ff09db0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x5584548d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.257+0000 7ff09db0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x5584548d41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59930/0 (socket says 172.21.15.28:59930) 2024-09-19T11:23:24.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.257+0000 7ff09db0a640 1 -- 172.21.15.28:0/3460341166 learned_addr learned my addr 172.21.15.28:0/3460341166 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.258+0000 7ff09db0a640 1 -- 172.21.15.28:0/3460341166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584548d4790 con 0x55845495e8b0 2024-09-19T11:23:24.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.258+0000 7ff09db0a640 1 --2- 172.21.15.28:0/3460341166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x5584548d41a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff098000c00 tx=0x7ff098051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c16815362b0e42a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.258+0000 7fef8be07640 1 -- 172.21.15.28:0/3460341166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0980519d0 con 0x55845495e8b0 2024-09-19T11:23:24.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.258+0000 7fef8be07640 1 -- 172.21.15.28:0/3460341166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff098051b70 con 0x55845495e8b0 2024-09-19T11:23:24.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.258+0000 7fef8be07640 1 -- 172.21.15.28:0/3460341166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff098051e50 con 0x55845495e8b0 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 -- 172.21.15.28:0/3460341166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 msgr2=0x5584548d41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 --2- 172.21.15.28:0/3460341166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x5584548d41a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff098000c00 tx=0x7ff098051000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 -- 172.21.15.28:0/3460341166 shutdown_connections 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 --2- 172.21.15.28:0/3460341166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x5584548d41a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 -- 172.21.15.28:0/3460341166 >> 172.21.15.28:0/3460341166 conn(0x558454959c80 msgr2=0x558454a49ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 -- 172.21.15.28:0/3460341166 shutdown_connections 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 -- 172.21.15.28:0/3460341166 wait complete. 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 Processor -- start 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 -- start start 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x558454a57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff0a0020800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558454965e30 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff09db0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x558454a57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff09db0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x558454a57cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59942/0 (socket says 172.21.15.28:59942) 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.259+0000 7ff09db0a640 1 -- 172.21.15.28:0/1230300021 learned_addr learned my addr 172.21.15.28:0/1230300021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.260+0000 7ff09db0a640 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558454a58250 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.260+0000 7ff09db0a640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x558454a57cf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff098005570 tx=0x7ff098002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.260+0000 7fef8a604640 1 -- 172.21.15.28:0/1230300021 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff098073070 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.260+0000 7fef8a604640 1 -- 172.21.15.28:0/1230300021 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09805a580 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.260+0000 7fef8a604640 1 -- 172.21.15.28:0/1230300021 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff09805a8a0 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.260+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558454a584e0 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.260+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558454a590f0 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.261+0000 7fef8a604640 1 -- 172.21.15.28:0/1230300021 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff098076070 con 0x55845495e8b0 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.261+0000 7fef8a604640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef8403d000 0x7fef8403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.261+0000 7fef8a604640 1 -- 172.21.15.28:0/1230300021 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7ff098062070 con 0x55845495e8b0 2024-09-19T11:23:24.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.261+0000 7ff09d309640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef8403d000 0x7fef8403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.261+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55845495f9b0 0x558454a5f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.261+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a5f880 con 0x55845495f9b0 2024-09-19T11:23:24.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.261+0000 7ff09e30b640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55845495f9b0 0x558454a5f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff09d309640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef8403d000 0x7fef8403f4c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff088000c00 tx=0x7ff0880013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff09e30b640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55845495f9b0 0x558454a5f340 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff09e30b640 1 -- 172.21.15.28:0/1230300021 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff09000a4d0 con 0x55845495f9b0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a5fc20 con 0x55845495f9b0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff09e30b640 1 -- 172.21.15.28:0/1230300021 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff09000a4d0 con 0x55845495f9b0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a616e0 0x558454a63b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a64310 con 0x558454a616e0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.262+0000 7ff09db0a640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a616e0 0x558454a63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.263+0000 7ff09db0a640 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a616e0 0x558454a63b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.263+0000 7ff09db0a640 1 -- 172.21.15.28:0/1230300021 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff09807f070 con 0x558454a616e0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.263+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a60470 con 0x55845495f9b0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.263+0000 7ff09e30b640 1 -- 172.21.15.28:0/1230300021 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff09000a4d0 con 0x55845495f9b0 2024-09-19T11:23:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.263+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a608c0 con 0x558454a616e0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff09db0a640 1 -- 172.21.15.28:0/1230300021 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff09807f070 con 0x558454a616e0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff0a0020800 1 Processor -- start 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff0a0020800 1 -- start start 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff0a0020800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a72ca0 0x558454a730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff0a0020800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5584548554a0 con 0x558454a72ca0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff09e30b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a72ca0 0x558454a730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff09e30b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a72ca0 0x558454a730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59958/0 (socket says 172.21.15.28:59958) 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.264+0000 7ff09e30b640 1 -- 172.21.15.28:0/3215739041 learned_addr learned my addr 172.21.15.28:0/3215739041 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.265+0000 7ff09e30b640 1 -- 172.21.15.28:0/3215739041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558454a73700 con 0x558454a72ca0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.265+0000 7ff09e30b640 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a72ca0 0x558454a730c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x558454a58cf0 tx=0x7ff0900167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.265+0000 7fef837fe640 1 -- 172.21.15.28:0/3215739041 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff090013640 con 0x558454a72ca0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.265+0000 7fef837fe640 1 -- 172.21.15.28:0/3215739041 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0900137e0 con 0x558454a72ca0 2024-09-19T11:23:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.265+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558454a73f30 con 0x558454a72ca0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.265+0000 7fef837fe640 1 -- 172.21.15.28:0/3215739041 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff090013ac0 con 0x558454a72ca0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.265+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558454a744d0 con 0x558454a72ca0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.266+0000 7fef837fe640 1 -- 172.21.15.28:0/3215739041 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff090011040 con 0x558454a72ca0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.266+0000 7fef837fe640 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef7803d000 0x7fef7803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.266+0000 7fef837fe640 1 -- 172.21.15.28:0/3215739041 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7ff09000e040 con 0x558454a72ca0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.266+0000 7ff09db0a640 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef7803d000 0x7fef7803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.266+0000 7ff0a0020800 1 Processor -- start 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.266+0000 7ff0a0020800 1 -- start start 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.266+0000 7ff09db0a640 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef7803d000 0x7fef7803f4c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff098051fd0 tx=0x7ff098096c90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7ff0a0020800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a7fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7ff0a0020800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5584548d2b30 con 0x558454a7f720 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7ff09e30b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a7fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7ff09e30b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a7fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59970/0 (socket says 172.21.15.28:59970) 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7ff09e30b640 1 -- 172.21.15.28:0/4038804515 learned_addr learned my addr 172.21.15.28:0/4038804515 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7ff09e30b640 1 -- 172.21.15.28:0/4038804515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558454a80110 con 0x558454a7f720 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7ff09e30b640 1 --2- 172.21.15.28:0/4038804515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a7fb20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff090020510 tx=0x7ff090054970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc7f7f4a5b2782f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7fef817fa640 1 -- 172.21.15.28:0/4038804515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff090044030 con 0x558454a7f720 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7fef817fa640 1 -- 172.21.15.28:0/4038804515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff090021e10 con 0x558454a7f720 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.267+0000 7fef817fa640 1 -- 172.21.15.28:0/4038804515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff09004de60 con 0x558454a7f720 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 -- 172.21.15.28:0/4038804515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 msgr2=0x558454a7fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 --2- 172.21.15.28:0/4038804515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a7fb20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff090020510 tx=0x7ff090054970 comp rx=0 tx=0).stop 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 -- 172.21.15.28:0/4038804515 shutdown_connections 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 --2- 172.21.15.28:0/4038804515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a7fb20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 -- 172.21.15.28:0/4038804515 >> 172.21.15.28:0/4038804515 conn(0x558454a7bb20 msgr2=0x558454a79d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 -- 172.21.15.28:0/4038804515 shutdown_connections 2024-09-19T11:23:24.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 -- 172.21.15.28:0/4038804515 wait complete. 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 Processor -- start 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 -- start start 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a86cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff0a0020800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558454a80730 con 0x558454a7f720 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff09e30b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a86cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff09e30b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a86cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59978/0 (socket says 172.21.15.28:59978) 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.268+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 learned_addr learned my addr 172.21.15.28:0/2852114404 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.269+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558454a87230 con 0x558454a7f720 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.269+0000 7ff09e30b640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a86cd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff090054790 tx=0x7ff0900547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.269+0000 7fef6f7fe640 1 -- 172.21.15.28:0/2852114404 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0900143c0 con 0x558454a7f720 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.269+0000 7fef6f7fe640 1 -- 172.21.15.28:0/2852114404 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff090014560 con 0x558454a7f720 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.269+0000 7fef6f7fe640 1 -- 172.21.15.28:0/2852114404 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff090020d00 con 0x558454a7f720 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.269+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558454a874c0 con 0x558454a7f720 2024-09-19T11:23:24.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.269+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558454a88050 con 0x558454a7f720 2024-09-19T11:23:24.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.270+0000 7fef6f7fe640 1 -- 172.21.15.28:0/2852114404 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff09001fa90 con 0x558454a7f720 2024-09-19T11:23:24.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.270+0000 7fef6f7fe640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef6803d000 0x7fef6803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.270+0000 7fef6f7fe640 1 -- 172.21.15.28:0/2852114404 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7ff090034c80 con 0x558454a7f720 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.270+0000 7ff09db0a640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef6803d000 0x7fef6803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.270+0000 7ff09db0a640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef6803d000 0x7fef6803f4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff0980514a0 tx=0x7ff09808d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558454a925f0 0x558454a94ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a95200 con 0x558454a925f0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff09d309640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558454a925f0 0x558454a94ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff09d309640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558454a925f0 0x558454a94ab0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff08800f5f0 con 0x558454a925f0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff0a0020800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff0a0020800 20 realm 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.271+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a91940 con 0x558454a925f0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff08800f5f0 con 0x558454a925f0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a95cf0 con 0x558454a925f0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff08800f5f0 con 0x558454a925f0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a965c0 con 0x558454a925f0 2024-09-19T11:23:24.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff08800f5f0 con 0x558454a925f0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a96fe0 0x558454a993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a99b40 con 0x558454a96fe0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.272+0000 7ff09e30b640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a96fe0 0x558454a993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff09e30b640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a96fe0 0x558454a993d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff0a0020800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff0a0020800 20 searching for the correct realm 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff09e30b640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff090048970 0x7ff090041210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff090048970 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.273+0000 7ff09db0a640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff090048970 0x7ff090041210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.274+0000 7ff09db0a640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff090048970 0x7ff090041210 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.274+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.274+0000 7ff09db0a640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff098078070 0x7ff098096220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.274+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.274+0000 7ff09d309640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff098078070 0x7ff098096220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.275+0000 7ff09d309640 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff098078070 0x7ff098096220 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.275+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08800f5f0 con 0x7ff098078070 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.275+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.275+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08800f5f0 con 0x558454a925f0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.275+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.275+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x558454a925f0 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.275+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x7ff098078070 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x7ff098078070 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff090048970 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.276+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x558454a925f0 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x7ff098078070 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x558454a925f0 2024-09-19T11:23:24.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff090048970 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.277+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x7ff098078070 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.278+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x558454a925f0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff08802b040 con 0x558454a925f0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x7ff098078070 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.279+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.280+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.280+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff090048970 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.280+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.280+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.280+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08802b040 con 0x558454a925f0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.280+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.280+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.281+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.281+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.281+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.281+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.281+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff090048970 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.281+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.281+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a96fe0 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff098078070 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x558454a925f0 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b470 con 0x7ff090048970 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.282+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a9a530 con 0x558454a925f0 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a9add0 con 0x558454a925f0 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a9b8f0 con 0x558454a96fe0 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 zone default found 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 4 Realm: () 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 zonegroup default 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a9bc90 con 0x7ff098078070 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.283+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a9da70 con 0x558454a96fe0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454a9de60 con 0x558454a96fe0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 20 started sync module instance, tier type = 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.284+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454a9cf10 con 0x7ff090048970 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.285+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.285+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94026844457568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454a9e2c0 con 0x7ff090048970 2024-09-19T11:23:24.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.285+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454a9eb10 con 0x558454a925f0 2024-09-19T11:23:24.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.286+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94026844457568] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.286+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.286+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94026844464736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454a9ef30 con 0x558454a925f0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.286+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454a9f780 con 0x558454a96fe0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.287+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.287+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94026844467504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454a9f750 con 0x558454a96fe0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.287+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa0770 con 0x7ff090048970 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.288+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94026844464736] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.288+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94026844467504] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.288+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.288+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94026844470128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454a9ea80 con 0x7ff090048970 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.288+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa1070 con 0x558454a925f0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.289+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.289+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94026844474208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa1030 con 0x558454a925f0 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.289+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa2050 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.290+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94026844470128] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.290+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.290+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94026844478304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa2050 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.290+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa2aa0 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.290+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94026844474208] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.291+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94026844478304] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.291+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.291+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94026844471872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa25b0 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.292+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa2f70 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.292+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94026844471872] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94026844476464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa2f70 con 0x7ff098078070 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=0 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=1 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=2 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=3 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=4 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=5 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=6 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94026844476464] v31'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 20 add_watcher() i=7 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.293+0000 7ff0a0020800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.294+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454aa63d0 con 0x558454a925f0 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.294+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.294+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94026844482704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454aa38f0 con 0x558454a925f0 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94026844482704] v31'86 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7fef4f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7ff0a0020800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7ff0a0020800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7ff0a0020800 5 note: GC not initialized 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7fef4e7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7ff0a0020800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.295+0000 7ff0a0020800 20 Filter name: none 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7feebffff640 20 reqs_thread_entry: start 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454ad91f0 con 0x558454a925f0 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 20 rados_obj.operate() r=0 bl.length=556 2024-09-19T11:23:24.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.296+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454adf7b0 con 0x558454a925f0 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.297+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v31'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.297+0000 7ff0a0020800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.297+0000 7ff0a0020800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.297+0000 7ff0a0020800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.297+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94026844729840 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454adaed0 con 0x558454a925f0 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94026844474208 notify 133143986186 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7ff088018040 con 0x558454a925f0 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7fef817fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986186 cookie 94026844474208 notifier 4418 bl.length()=567 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7fef817fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:24.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7fef817fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7fef817fa640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fef60003ea0 con 0x558454a925f0 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(64 notify.4 [notify cookie 94026844729840 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94026844729840 notify 133143986186 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff08801db00 con 0x558454a925f0 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.298+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454adf1f0 con 0x7ff090048970 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.299+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 foo:foo2 [delete] v31'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.299+0000 7ff0a0020800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.299+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94026844729840 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558454adaa20 con 0x7ff090048970 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.299+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== watch-notify(notify (1) cookie 94026844457568 notify 133143986181 ret 0) ==== 201+0+0 (crc 0 0 0) 0x7ff0980ac060 con 0x7ff090048970 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.299+0000 7fef80ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986181 cookie 94026844457568 notifier 4418 bl.length()=159 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.299+0000 7fef80ff9640 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7feec80032c0 con 0x7ff090048970 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.299+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [notify cookie 94026844729840 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.300+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== watch-notify(notify_complete (2) cookie 94026844729840 notify 133143986181 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558454a590f0 con 0x7ff090048970 2024-09-19T11:23:24.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.300+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.310 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.301+0000 7fef4f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.301+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94026844482704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x558454a925f0 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.302+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94026844482704] v31'87 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.302+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94026844457568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x7ff090048970 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.303+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94026844457568] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.303+0000 7ff0a0020800 20 remove_watcher() i=0 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.303+0000 7ff0a0020800 2 removed watcher, disabling cache 2024-09-19T11:23:24.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.303+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94026844464736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x558454a925f0 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.304+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94026844464736] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.304+0000 7ff0a0020800 20 remove_watcher() i=1 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.304+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94026844467504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x558454a96fe0 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.305+0000 7ff09e30b640 1 -- 172.21.15.28:0/2852114404 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94026844467504] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff090053020 con 0x558454a96fe0 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.305+0000 7ff0a0020800 20 remove_watcher() i=2 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.305+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94026844470128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x7ff090048970 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.306+0000 7ff09db0a640 1 -- 172.21.15.28:0/2852114404 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94026844470128] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff09807f070 con 0x7ff090048970 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.306+0000 7ff0a0020800 20 remove_watcher() i=3 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.306+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94026844474208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x558454a925f0 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.307+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94026844474208] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x558454a925f0 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.307+0000 7ff0a0020800 20 remove_watcher() i=4 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.307+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94026844478304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x7ff098078070 2024-09-19T11:23:24.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.308+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94026844478304] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.308+0000 7ff0a0020800 20 remove_watcher() i=5 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.308+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94026844471872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x7ff098078070 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.309+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94026844471872] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.309+0000 7ff0a0020800 20 remove_watcher() i=6 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.309+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94026844476464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558454ad7090 con 0x7ff098078070 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94026844476464] v31'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff08803b040 con 0x7ff098078070 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 20 remove_watcher() i=7 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff098078070 msgr2=0x7ff098096220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff098078070 0x7ff098096220 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558454a925f0 msgr2=0x558454a94ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558454a925f0 0x558454a94ab0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff090048970 msgr2=0x7ff090041210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff090048970 0x7ff090041210 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a96fe0 msgr2=0x558454a993d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a96fe0 0x558454a993d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef6803d000 msgr2=0x7fef6803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef6803d000 0x7fef6803f4c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff0980514a0 tx=0x7ff09808d000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 msgr2=0x558454a86cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff0a0020800 1 --2- 172.21.15.28:0/2852114404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a7f720 0x558454a86cd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff090054790 tx=0x7ff0900547c0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 reap_dead start 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.310+0000 7ff09d309640 1 -- 172.21.15.28:0/2852114404 reap_dead start 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 shutdown_connections 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 >> 172.21.15.28:0/2852114404 conn(0x558454a7bb20 msgr2=0x558454a79d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 shutdown_connections 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/2852114404 wait complete. 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef7803d000 msgr2=0x7fef7803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef7803d000 0x7fef7803f4c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff098051fd0 tx=0x7ff098096c90 comp rx=0 tx=0).stop 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a72ca0 msgr2=0x558454a730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a72ca0 0x558454a730c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x558454a58cf0 tx=0x7ff0900167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 shutdown_connections 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef7803d000 0x7fef7803f4c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 --2- 172.21.15.28:0/3215739041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558454a72ca0 0x558454a730c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 >> 172.21.15.28:0/3215739041 conn(0x558454a67050 msgr2=0x558454a67430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 shutdown_connections 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/3215739041 wait complete. 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55845495f9b0 msgr2=0x558454a5f340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55845495f9b0 0x558454a5f340 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a616e0 msgr2=0x558454a63b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a616e0 0x558454a63b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef8403d000 msgr2=0x7fef8403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.311+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef8403d000 0x7fef8403f4c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff088000c00 tx=0x7ff0880013b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 msgr2=0x558454a57cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x558454a57cf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff098005570 tx=0x7ff098002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 shutdown_connections 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558454a616e0 0x558454a63b80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fef8403d000 0x7fef8403f4c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55845495f9b0 0x558454a5f340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 --2- 172.21.15.28:0/1230300021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55845495e8b0 0x558454a57cf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 >> 172.21.15.28:0/1230300021 conn(0x558454959c80 msgr2=0x558454a49620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 shutdown_connections 2024-09-19T11:23:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.312+0000 7ff0a0020800 1 -- 172.21.15.28:0/1230300021 wait complete. 2024-09-19T11:23:24.322 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:24.322 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-09-19T11:23:24.322 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-09-19T11:23:24.323 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-09-19T11:23:24.368 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:24.368 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.367+0000 7f8d8d92e800 1 Processor -- start 2024-09-19T11:23:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.367+0000 7f8d8d92e800 1 -- start start 2024-09-19T11:23:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.367+0000 7f8d8d92e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x559949cd6a00 0x559949cd6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.367+0000 7f8d8d92e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559949cd87a0 con 0x559949cd7340 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.367+0000 7f8d8d92e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x559949bed7d0 con 0x559949cd6a00 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.368+0000 7f8d896dd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x559949cd6a00 0x559949cd6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.368+0000 7f8d88edc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x559949cd7340 0x559949d6a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33190/0 (socket says 172.21.15.28:33190) 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.368+0000 7f8d896dd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x559949cd6a00 0x559949cd6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59990/0 (socket says 172.21.15.28:59990) 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.368+0000 7f8d88edc640 1 -- 172.21.15.28:0/2226199818 learned_addr learned my addr 172.21.15.28:0/2226199818 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.368+0000 7f8d896dd640 1 -- 172.21.15.28:0/2226199818 >> v1:172.21.15.22:6789/0 conn(0x559949cd7340 legacy=0x559949d6a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.368+0000 7f8d83fff640 1 -- 172.21.15.28:0/2226199818 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685006234 0 0) 0x559949cd87a0 con 0x559949cd7340 2024-09-19T11:23:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.368+0000 7f8d896dd640 1 -- 172.21.15.28:0/2226199818 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559949d6cad0 con 0x559949cd6a00 2024-09-19T11:23:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.370+0000 7f8d896dd640 1 --2- 172.21.15.28:0/2226199818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949cd6e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00a470 tx=0x7f8d7c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9964d1316e3335b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.370+0000 7f8d83fff640 1 -- 172.21.15.28:0/2226199818 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d7c053570 con 0x559949cd6a00 2024-09-19T11:23:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.370+0000 7f8d83fff640 1 -- 172.21.15.28:0/2226199818 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7c058070 con 0x559949cd6a00 2024-09-19T11:23:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.370+0000 7f8d83fff640 1 -- 172.21.15.28:0/2226199818 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d7c053880 con 0x559949cd6a00 2024-09-19T11:23:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.371+0000 7f8d8d92e800 1 -- 172.21.15.28:0/2226199818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 msgr2=0x559949cd6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.371+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/2226199818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949cd6e00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00a470 tx=0x7f8d7c045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.371+0000 7f8d8d92e800 1 -- 172.21.15.28:0/2226199818 shutdown_connections 2024-09-19T11:23:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.371+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/2226199818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949cd6e00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.371+0000 7f8d8d92e800 1 -- 172.21.15.28:0/2226199818 >> 172.21.15.28:0/2226199818 conn(0x559949d5dc60 msgr2=0x559949d600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.371+0000 7f8d8d92e800 1 -- 172.21.15.28:0/2226199818 shutdown_connections 2024-09-19T11:23:24.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.371+0000 7f8d8d92e800 1 -- 172.21.15.28:0/2226199818 wait complete. 2024-09-19T11:23:24.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d8d92e800 1 Processor -- start 2024-09-19T11:23:24.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d8d92e800 1 -- start start 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d8d92e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e54340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d8d92e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559949bfadd0 con 0x559949cd6a00 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d896dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e54340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d896dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e54340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:59992/0 (socket says 172.21.15.28:59992) 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d896dd640 1 -- 172.21.15.28:0/3089704540 learned_addr learned my addr 172.21.15.28:0/3089704540 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.386+0000 7f8d896dd640 1 -- 172.21.15.28:0/3089704540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559949e54930 con 0x559949cd6a00 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.387+0000 7f8d896dd640 1 --2- 172.21.15.28:0/3089704540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e54340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005870 tx=0x7f8d7c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bed80ed1bd3b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.387+0000 7f8c77606640 1 -- 172.21.15.28:0/3089704540 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d7c05c580 con 0x559949cd6a00 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.387+0000 7f8c77606640 1 -- 172.21.15.28:0/3089704540 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7c063030 con 0x559949cd6a00 2024-09-19T11:23:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.387+0000 7f8c77606640 1 -- 172.21.15.28:0/3089704540 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d7c05c850 con 0x559949cd6a00 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.387+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3089704540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 msgr2=0x559949e54340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.387+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3089704540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e54340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c005870 tx=0x7f8d7c051000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3089704540 shutdown_connections 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3089704540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e54340 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3089704540 >> 172.21.15.28:0/3089704540 conn(0x559949d5dc60 msgr2=0x559949e4fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3089704540 shutdown_connections 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3089704540 wait complete. 2024-09-19T11:23:24.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 Processor -- start 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 -- start start 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e5bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d8d92e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559949d6bd90 con 0x559949cd6a00 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d896dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e5bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d896dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e5bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60002/0 (socket says 172.21.15.28:60002) 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d896dd640 1 -- 172.21.15.28:0/739747512 learned_addr learned my addr 172.21.15.28:0/739747512 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.388+0000 7f8d896dd640 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559949e5c2d0 con 0x559949cd6a00 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.389+0000 7f8d896dd640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e5bd70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c045f50 tx=0x7f8d7c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.389+0000 7f8c75e03640 1 -- 172.21.15.28:0/739747512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d7c078070 con 0x559949cd6a00 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.389+0000 7f8c75e03640 1 -- 172.21.15.28:0/739747512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7c054a70 con 0x559949cd6a00 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.389+0000 7f8c75e03640 1 -- 172.21.15.28:0/739747512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d7c054d50 con 0x559949cd6a00 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.389+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559949e5c560 con 0x559949cd6a00 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.389+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559949e5d170 con 0x559949cd6a00 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8c75e03640 1 -- 172.21.15.28:0/739747512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8d7c07b070 con 0x559949cd6a00 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8c75e03640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c7003d000 0x7f8c7003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8c75e03640 1 -- 172.21.15.28:0/739747512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f8d7c070070 con 0x559949cd6a00 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8d88edc640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c7003d000 0x7f8c7003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949cd7340 0x559949e633c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e63900 con 0x559949cd7340 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8d89ede640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949cd7340 0x559949e633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.390+0000 7f8d88edc640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c7003d000 0x7f8c7003f4c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d740013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.391+0000 7f8d89ede640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949cd7340 0x559949e633c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.391+0000 7f8d89ede640 1 -- 172.21.15.28:0/739747512 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949cd7340 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.391+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e63ca0 con 0x559949cd7340 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.391+0000 7f8d89ede640 1 -- 172.21.15.28:0/739747512 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949cd7340 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.391+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e657a0 0x559949e67c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.391+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e683b0 con 0x559949e657a0 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.391+0000 7f8d896dd640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e657a0 0x559949e67c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.392+0000 7f8d896dd640 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e657a0 0x559949e67c20 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.392+0000 7f8d896dd640 1 -- 172.21.15.28:0/739747512 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8d7c07a030 con 0x559949e657a0 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.392+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e644f0 con 0x559949cd7340 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.392+0000 7f8d89ede640 1 -- 172.21.15.28:0/739747512 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949cd7340 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.393+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e64940 con 0x559949e657a0 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.393+0000 7f8d896dd640 1 -- 172.21.15.28:0/739747512 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8d7c07a030 con 0x559949e657a0 2024-09-19T11:23:24.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.393+0000 7f8d8d92e800 1 Processor -- start 2024-09-19T11:23:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.393+0000 7f8d8d92e800 1 -- start start 2024-09-19T11:23:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.393+0000 7f8d8d92e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e76d40 0x559949e77160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.393+0000 7f8d8d92e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559949c594a0 con 0x559949e76d40 2024-09-19T11:23:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8d89ede640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e76d40 0x559949e77160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8d89ede640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e76d40 0x559949e77160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60008/0 (socket says 172.21.15.28:60008) 2024-09-19T11:23:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8d89ede640 1 -- 172.21.15.28:0/3369959551 learned_addr learned my addr 172.21.15.28:0/3369959551 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8d89ede640 1 -- 172.21.15.28:0/3369959551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559949e777a0 con 0x559949e76d40 2024-09-19T11:23:24.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8d89ede640 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e76d40 0x559949e77160 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x559949e5d390 tx=0x7f8d840167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8c6effd640 1 -- 172.21.15.28:0/3369959551 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d84013640 con 0x559949e76d40 2024-09-19T11:23:24.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8c6effd640 1 -- 172.21.15.28:0/3369959551 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d840137e0 con 0x559949e76d40 2024-09-19T11:23:24.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8c6effd640 1 -- 172.21.15.28:0/3369959551 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d84013ae0 con 0x559949e76d40 2024-09-19T11:23:24.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559949e77fd0 con 0x559949e76d40 2024-09-19T11:23:24.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.394+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559949e78570 con 0x559949e76d40 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.395+0000 7f8c6effd640 1 -- 172.21.15.28:0/3369959551 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8d84011040 con 0x559949e76d40 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.395+0000 7f8c6effd640 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c6403d000 0x7f8c6403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.395+0000 7f8c6effd640 1 -- 172.21.15.28:0/3369959551 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f8d8400e040 con 0x559949e76d40 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.395+0000 7f8d896dd640 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c6403d000 0x7f8c6403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d8d92e800 1 Processor -- start 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d896dd640 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c6403d000 0x7f8c6403f4c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c096760 tx=0x7f8d7c054ef0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d8d92e800 1 -- start start 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d8d92e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e83bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d8d92e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559949e54d00 con 0x559949e837e0 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d89ede640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e83bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d89ede640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e83bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60014/0 (socket says 172.21.15.28:60014) 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d89ede640 1 -- 172.21.15.28:0/3559468058 learned_addr learned my addr 172.21.15.28:0/3559468058 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.396+0000 7f8d89ede640 1 -- 172.21.15.28:0/3559468058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559949e841b0 con 0x559949e837e0 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d89ede640 1 --2- 172.21.15.28:0/3559468058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e83bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d840547f0 tx=0x7f8d8401fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4986988921210fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8c6cff9640 1 -- 172.21.15.28:0/3559468058 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d84051070 con 0x559949e837e0 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8c6cff9640 1 -- 172.21.15.28:0/3559468058 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d84053440 con 0x559949e837e0 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8c6cff9640 1 -- 172.21.15.28:0/3559468058 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d84021a90 con 0x559949e837e0 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3559468058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 msgr2=0x559949e83bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3559468058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e83bc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8d840547f0 tx=0x7f8d8401fe70 comp rx=0 tx=0).stop 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3559468058 shutdown_connections 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3559468058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e83bc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3559468058 >> 172.21.15.28:0/3559468058 conn(0x559949e7fbe0 msgr2=0x559949e7de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3559468058 shutdown_connections 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.397+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3559468058 wait complete. 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d8d92e800 1 Processor -- start 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d8d92e800 1 -- start start 2024-09-19T11:23:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d8d92e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e8ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d8d92e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559949e847f0 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d89ede640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e8ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d89ede640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e8ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60030/0 (socket says 172.21.15.28:60030) 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 learned_addr learned my addr 172.21.15.28:0/730091430 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559949e8b360 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8d89ede640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e8ae00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8d840547f0 tx=0x7f8d84054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.398+0000 7f8c5affd640 1 -- 172.21.15.28:0/730091430 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d84021440 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.399+0000 7f8c5affd640 1 -- 172.21.15.28:0/730091430 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d8404c070 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.399+0000 7f8c5affd640 1 -- 172.21.15.28:0/730091430 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d840459c0 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.399+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559949e8b5f0 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.399+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559949e8c180 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.399+0000 7f8c5affd640 1 -- 172.21.15.28:0/730091430 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8d84045b60 con 0x559949e837e0 2024-09-19T11:23:24.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8c5affd640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c5403d000 0x7f8c5403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8c5affd640 1 -- 172.21.15.28:0/730091430 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f8d840539d0 con 0x559949e837e0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8d896dd640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c5403d000 0x7f8c5403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949e966e0 0x559949e98ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e992f0 con 0x559949e966e0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8d88edc640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949e966e0 0x559949e98ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8d896dd640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c5403d000 0x7f8c5403f4c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c045a60 tx=0x7f8d7c091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.400+0000 7f8d88edc640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949e966e0 0x559949e98ba0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8d7400f5f0 con 0x559949e966e0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 realm 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e95a30 con 0x559949e966e0 2024-09-19T11:23:24.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8d7400f5f0 con 0x559949e966e0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e99dd0 con 0x559949e966e0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8d7400f5f0 con 0x559949e966e0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e9a6a0 con 0x559949e966e0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8d7400f5f0 con 0x559949e966e0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e9afa0 0x559949e9d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e9db10 con 0x559949e9afa0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.401+0000 7f8d89ede640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e9afa0 0x559949e9d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d89ede640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e9afa0 0x559949e9d3b0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d8d92e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d8d92e800 20 searching for the correct realm 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d89ede640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8d8404d2a0 0x7f8d8403d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d8404d2a0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.402+0000 7f8d896dd640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8d8404d2a0 0x7f8d8403d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.403+0000 7f8d896dd640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8d8404d2a0 0x7f8d8403d7f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.403+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.403+0000 7f8d896dd640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8d7c0a6040 0x7f8d7c05e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.403+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.403+0000 7f8d88edc640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8d7c0a6040 0x7f8d7c05e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.403+0000 7f8d88edc640 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8d7c0a6040 0x7f8d7c05e850 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7400f5f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7400f5f0 con 0x559949e966e0 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x559949e966e0 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.404+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.405+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.405+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d8404d2a0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.405+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.405+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.405+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.406+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.406+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x559949e966e0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.406+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.406+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.406+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.406+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x559949e966e0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.406+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d8404d2a0 2024-09-19T11:23:24.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.407+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x559949e966e0 2024-09-19T11:23:24.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8d7402b040 con 0x559949e966e0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.408+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d8404d2a0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7402b040 con 0x559949e966e0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.409+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.410+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.410+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.410+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.410+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.410+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.410+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d8404d2a0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e9afa0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x559949e966e0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.411+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559949e9f4f0 con 0x7f8d8404d2a0 2024-09-19T11:23:24.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e9e5c0 con 0x559949e966e0 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e9ee60 con 0x559949e966e0 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e9e200 con 0x559949e9afa0 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.412+0000 7f8d8d92e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 zone default found 2024-09-19T11:23:24.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 4 Realm: () 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 zonegroup default 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949e9ea90 con 0x7f8d7c0a6040 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949ea18e0 con 0x559949e9afa0 2024-09-19T11:23:24.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949ea1e20 con 0x559949e9afa0 2024-09-19T11:23:24.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 started sync module instance, tier type = 2024-09-19T11:23:24.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:24.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.413+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea0d50 con 0x7f8d8404d2a0 2024-09-19T11:23:24.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.414+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.414+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94116858431648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea22b0 con 0x7f8d8404d2a0 2024-09-19T11:23:24.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.414+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea2b00 con 0x559949e966e0 2024-09-19T11:23:24.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.415+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94116858431648] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.415+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.415+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94116858439248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea1250 con 0x559949e966e0 2024-09-19T11:23:24.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.416+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea36a0 con 0x559949e9afa0 2024-09-19T11:23:24.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.416+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94116858439248] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.417+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.417+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94116858442096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea3660 con 0x559949e9afa0 2024-09-19T11:23:24.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.417+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea4150 con 0x7f8d8404d2a0 2024-09-19T11:23:24.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.418+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94116858442096] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.418+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.418+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94116858444896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea4540 con 0x7f8d8404d2a0 2024-09-19T11:23:24.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.418+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea4f80 con 0x559949e966e0 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.419+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.419+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94116858448464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea4f80 con 0x559949e966e0 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.419+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea5f80 con 0x7f8d7c0a6040 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.419+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94116858444896] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.420+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.420+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94116858452592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea5f40 con 0x7f8d7c0a6040 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.420+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea69d0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.420+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94116858448464] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.421+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94116858452592] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.421+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.421+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94116858455264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea69d0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.421+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea6e00 con 0x7f8d7c0a6040 2024-09-19T11:23:24.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94116858455264] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94116858440304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea53b0 con 0x7f8d7c0a6040 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 20 add_watcher() i=0 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 20 add_watcher() i=1 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 20 add_watcher() i=2 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 20 add_watcher() i=3 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 20 add_watcher() i=4 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 20 add_watcher() i=5 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.422+0000 7f8d8d92e800 20 add_watcher() i=6 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.423+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94116858440304] v31'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.423+0000 7f8d8d92e800 20 add_watcher() i=7 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.423+0000 7f8d8d92e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:24.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.423+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949eaa250 con 0x559949e966e0 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.423+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.423+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94116858456736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949ea7710 con 0x559949e966e0 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.424+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94116858456736] v31'88 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.424+0000 7f8c3f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.424+0000 7f8d8d92e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.424+0000 7f8d8d92e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.424+0000 7f8d8d92e800 5 note: GC not initialized 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8c3e7e4640 20 reqs_thread_entry: start 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d8d92e800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d8d92e800 20 Filter name: none 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8baffff640 20 reqs_thread_entry: start 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949edcc40 con 0x559949e966e0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d8d92e800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d8d92e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.425+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949edd500 con 0x559949e966e0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.426+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.426+0000 7f8d8d92e800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.426+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559949ede510 con 0x7f8d8404d2a0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.426+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.427+0000 7f8c3f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.427+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94116858456736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x559949e966e0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.428+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94116858456736] v31'89 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.428+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94116858431648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x7f8d8404d2a0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.429+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94116858431648] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.429+0000 7f8d8d92e800 20 remove_watcher() i=0 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.429+0000 7f8d8d92e800 2 removed watcher, disabling cache 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.429+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94116858439248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x559949e966e0 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.430+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94116858439248] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.430+0000 7f8d8d92e800 20 remove_watcher() i=1 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.430+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94116858442096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x559949e9afa0 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.431+0000 7f8d89ede640 1 -- 172.21.15.28:0/730091430 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94116858442096] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d8400a4d0 con 0x559949e9afa0 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.431+0000 7f8d8d92e800 20 remove_watcher() i=2 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.431+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94116858444896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x7f8d8404d2a0 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.432+0000 7f8d896dd640 1 -- 172.21.15.28:0/730091430 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94116858444896] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7c07a030 con 0x7f8d8404d2a0 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.432+0000 7f8d8d92e800 20 remove_watcher() i=3 2024-09-19T11:23:24.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.432+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94116858448464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x559949e966e0 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.433+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94116858448464] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x559949e966e0 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.433+0000 7f8d8d92e800 20 remove_watcher() i=4 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.433+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94116858452592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x7f8d7c0a6040 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.434+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94116858452592] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.434+0000 7f8d8d92e800 20 remove_watcher() i=5 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.434+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94116858455264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x7f8d7c0a6040 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.435+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94116858455264] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.435+0000 7f8d8d92e800 20 remove_watcher() i=6 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.435+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94116858440304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559949edab80 con 0x7f8d7c0a6040 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94116858440304] v31'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8d7403b040 con 0x7f8d7c0a6040 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 20 remove_watcher() i=7 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8d7c0a6040 msgr2=0x7f8d7c05e850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8d7c0a6040 0x7f8d7c05e850 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949e966e0 msgr2=0x559949e98ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949e966e0 0x559949e98ba0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8d8404d2a0 msgr2=0x7f8d8403d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8d8404d2a0 0x7f8d8403d7f0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e9afa0 msgr2=0x559949e9d3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e9afa0 0x559949e9d3b0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c5403d000 msgr2=0x7f8c5403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c5403d000 0x7f8c5403f4c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c045a60 tx=0x7f8d7c091000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 msgr2=0x559949e8ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e8ae00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8d840547f0 tx=0x7f8d84054780 comp rx=0 tx=0).stop 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d88edc640 1 -- 172.21.15.28:0/730091430 reap_dead start 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 shutdown_connections 2024-09-19T11:23:24.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/730091430 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e837e0 0x559949e8ae00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 >> 172.21.15.28:0/730091430 conn(0x559949e7fbe0 msgr2=0x559949e7ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 shutdown_connections 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/730091430 wait complete. 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c6403d000 msgr2=0x7f8c6403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c6403d000 0x7f8c6403f4c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c096760 tx=0x7f8d7c054ef0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e76d40 msgr2=0x559949e77160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e76d40 0x559949e77160 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x559949e5d390 tx=0x7f8d840167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 shutdown_connections 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c6403d000 0x7f8c6403f4c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/3369959551 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949e76d40 0x559949e77160 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 >> 172.21.15.28:0/3369959551 conn(0x559949e6b0f0 msgr2=0x559949e6b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.437+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 shutdown_connections 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/3369959551 wait complete. 2024-09-19T11:23:24.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949cd7340 msgr2=0x559949e633c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949cd7340 0x559949e633c0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e657a0 msgr2=0x559949e67c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e657a0 0x559949e67c20 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c7003d000 msgr2=0x7f8c7003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c7003d000 0x7f8c7003f4c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d740013b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 msgr2=0x559949e5bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e5bd70 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c045f50 tx=0x7f8d7c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 shutdown_connections 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559949e657a0 0x559949e67c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f8c7003d000 0x7f8c7003f4c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559949cd7340 0x559949e633c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 --2- 172.21.15.28:0/739747512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559949cd6a00 0x559949e5bd70 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 >> 172.21.15.28:0/739747512 conn(0x559949d5dc60 msgr2=0x559949e4e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 shutdown_connections 2024-09-19T11:23:24.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.438+0000 7f8d8d92e800 1 -- 172.21.15.28:0/739747512 wait complete. 2024-09-19T11:23:24.465 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T11:23:24.465 DEBUG:tasks.util.rgw: json result: [] 2024-09-19T11:23:24.465 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-09-19T11:23:24.465 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-09-19T11:23:24.465 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-09-19T11:23:24.511 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:24.511 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.510+0000 7fc492216800 1 Processor -- start 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.510+0000 7fc492216800 1 -- start start 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.510+0000 7fc492216800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d292626a90 0x55d292628f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.511+0000 7fc492216800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d2926295d0 con 0x55d292622f20 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.511+0000 7fc492216800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55d2924ad7d0 con 0x55d292626a90 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.511+0000 7fc4851e6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d292626a90 0x55d292628f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.511+0000 7fc48d9e7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55d292622f20 0x55d292625310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33206/0 (socket says 172.21.15.28:33206) 2024-09-19T11:23:24.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.511+0000 7fc48d9e7640 1 -- 172.21.15.28:0/879644909 learned_addr learned my addr 172.21.15.28:0/879644909 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.511+0000 7fc4851e6640 1 -- 172.21.15.28:0/879644909 >> v1:172.21.15.22:6789/0 conn(0x55d292622f20 legacy=0x55d292625310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.511+0000 7fc4851e6640 1 -- 172.21.15.28:0/879644909 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29262a800 con 0x55d292626a90 2024-09-19T11:23:24.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.513+0000 7fc4851e6640 1 --2- 172.21.15.28:0/879644909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292626a90 0x55d292628f50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc47800a470 tx=0x7fc478045880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49429ee0448fde7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.513+0000 7fc48d1e6640 1 -- 172.21.15.28:0/879644909 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc478059070 con 0x55d292626a90 2024-09-19T11:23:24.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.513+0000 7fc48d1e6640 1 -- 172.21.15.28:0/879644909 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc478054570 con 0x55d292626a90 2024-09-19T11:23:24.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.513+0000 7fc48d1e6640 1 -- 172.21.15.28:0/879644909 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc478054870 con 0x55d292626a90 2024-09-19T11:23:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.513+0000 7fc492216800 1 -- 172.21.15.28:0/879644909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292626a90 msgr2=0x55d292628f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.513+0000 7fc492216800 1 --2- 172.21.15.28:0/879644909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292626a90 0x55d292628f50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc47800a470 tx=0x7fc478045880 comp rx=0 tx=0).stop 2024-09-19T11:23:24.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.514+0000 7fc492216800 1 -- 172.21.15.28:0/879644909 shutdown_connections 2024-09-19T11:23:24.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.514+0000 7fc492216800 1 --2- 172.21.15.28:0/879644909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292626a90 0x55d292628f50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.514+0000 7fc492216800 1 -- 172.21.15.28:0/879644909 >> 172.21.15.28:0/879644909 conn(0x55d29261ca60 msgr2=0x55d29261eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.514+0000 7fc492216800 1 -- 172.21.15.28:0/879644909 shutdown_connections 2024-09-19T11:23:24.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.514+0000 7fc492216800 1 -- 172.21.15.28:0/879644909 wait complete. 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.529+0000 7fc492216800 1 Processor -- start 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.529+0000 7fc492216800 1 -- start start 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.529+0000 7fc492216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d2926226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.529+0000 7fc492216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2924badd0 con 0x55d292622f20 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.529+0000 7fc48d9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d2926226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.529+0000 7fc48d9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d2926226d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60058/0 (socket says 172.21.15.28:60058) 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.529+0000 7fc48d9e7640 1 -- 172.21.15.28:0/2895954188 learned_addr learned my addr 172.21.15.28:0/2895954188 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.530+0000 7fc48d9e7640 1 -- 172.21.15.28:0/2895954188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d292622cc0 con 0x55d292622f20 2024-09-19T11:23:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.530+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/2895954188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d2926226d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc48000b510 tx=0x7fc48000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d364aa05a1e675 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.530+0000 7fc37be07640 1 -- 172.21.15.28:0/2895954188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc480008070 con 0x55d292622f20 2024-09-19T11:23:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.530+0000 7fc37be07640 1 -- 172.21.15.28:0/2895954188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48000d5c0 con 0x55d292622f20 2024-09-19T11:23:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.530+0000 7fc37be07640 1 -- 172.21.15.28:0/2895954188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48000d8e0 con 0x55d292622f20 2024-09-19T11:23:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 -- 172.21.15.28:0/2895954188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 msgr2=0x55d2926226d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 --2- 172.21.15.28:0/2895954188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d2926226d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc48000b510 tx=0x7fc48000b9d0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 -- 172.21.15.28:0/2895954188 shutdown_connections 2024-09-19T11:23:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 --2- 172.21.15.28:0/2895954188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d2926226d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 -- 172.21.15.28:0/2895954188 >> 172.21.15.28:0/2895954188 conn(0x55d29261ca60 msgr2=0x55d29270dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 -- 172.21.15.28:0/2895954188 shutdown_connections 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 -- 172.21.15.28:0/2895954188 wait complete. 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 Processor -- start 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 -- start start 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d29271c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc492216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d292629aa0 con 0x55d292622f20 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc48d9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d29271c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc48d9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d29271c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60066/0 (socket says 172.21.15.28:60066) 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.531+0000 7fc48d9e7640 1 -- 172.21.15.28:0/2616777466 learned_addr learned my addr 172.21.15.28:0/2616777466 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.532+0000 7fc48d9e7640 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29271c5f0 con 0x55d292622f20 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.532+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d29271c090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55d292622200 tx=0x7fc480000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.532+0000 7fc37a604640 1 -- 172.21.15.28:0/2616777466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48000fac0 con 0x55d292622f20 2024-09-19T11:23:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.532+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d29271c880 con 0x55d292622f20 2024-09-19T11:23:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.532+0000 7fc37a604640 1 -- 172.21.15.28:0/2616777466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc480020030 con 0x55d292622f20 2024-09-19T11:23:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.532+0000 7fc37a604640 1 -- 172.21.15.28:0/2616777466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48000fdd0 con 0x55d292622f20 2024-09-19T11:23:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.532+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d29271d490 con 0x55d292622f20 2024-09-19T11:23:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.533+0000 7fc37a604640 1 -- 172.21.15.28:0/2616777466 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc48001b430 con 0x55d292622f20 2024-09-19T11:23:24.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.533+0000 7fc37a604640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc37403cfb0 0x7fc37403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.533+0000 7fc37a604640 1 -- 172.21.15.28:0/2616777466 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc480054d80 con 0x55d292622f20 2024-09-19T11:23:24.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.533+0000 7fc4851e6640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc37403cfb0 0x7fc37403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.533+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292626a90 0x55d2927236e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.533+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292723c20 con 0x55d292626a90 2024-09-19T11:23:24.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.534+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292626a90 0x55d2927236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.534+0000 7fc4851e6640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc37403cfb0 0x7fc37403f470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc478056000 tx=0x7fc478050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.534+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292626a90 0x55d2927236e0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.534+0000 7fc48e1e8640 1 -- 172.21.15.28:0/2616777466 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc48800a4d0 con 0x55d292626a90 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.534+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292723fc0 con 0x55d292626a90 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.534+0000 7fc48e1e8640 1 -- 172.21.15.28:0/2616777466 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc48800a4d0 con 0x55d292626a90 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d292725ac0 0x55d292727f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2927286f0 con 0x55d292725ac0 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d292725ac0 0x55d292727f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d292725ac0 0x55d292727f60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc48d9e7640 1 -- 172.21.15.28:0/2616777466 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc48002c070 con 0x55d292725ac0 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292724810 con 0x55d292626a90 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc48e1e8640 1 -- 172.21.15.28:0/2616777466 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc48800a4d0 con 0x55d292626a90 2024-09-19T11:23:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.535+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292724c30 con 0x55d292725ac0 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.536+0000 7fc48d9e7640 1 -- 172.21.15.28:0/2616777466 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc48002c070 con 0x55d292725ac0 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.536+0000 7fc492216800 1 Processor -- start 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.536+0000 7fc492216800 1 -- start start 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.536+0000 7fc492216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292737120 0x55d292737540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.536+0000 7fc492216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2925194a0 con 0x55d292737120 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc48e1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292737120 0x55d292737540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc48e1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292737120 0x55d292737540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60082/0 (socket says 172.21.15.28:60082) 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc48e1e8640 1 -- 172.21.15.28:0/3766398372 learned_addr learned my addr 172.21.15.28:0/3766398372 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc48e1e8640 1 -- 172.21.15.28:0/3766398372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d292737b80 con 0x55d292737120 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292737120 0x55d292737540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55d29271d090 tx=0x7fc4880167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc3737fe640 1 -- 172.21.15.28:0/3766398372 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc488013640 con 0x55d292737120 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc3737fe640 1 -- 172.21.15.28:0/3766398372 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4880137e0 con 0x55d292737120 2024-09-19T11:23:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2927383b0 con 0x55d292737120 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc3737fe640 1 -- 172.21.15.28:0/3766398372 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc488013ae0 con 0x55d292737120 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.537+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d292738950 con 0x55d292737120 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.538+0000 7fc3737fe640 1 -- 172.21.15.28:0/3766398372 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc488011040 con 0x55d292737120 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.538+0000 7fc3737fe640 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc36803d000 0x7fc36803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.538+0000 7fc3737fe640 1 -- 172.21.15.28:0/3766398372 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc48800e040 con 0x55d292737120 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.538+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc36803d000 0x7fc36803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc492216800 1 Processor -- start 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc492216800 1 -- start start 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc36803d000 0x7fc36803f4c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc4800014e0 tx=0x7fc480053060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc492216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d292743fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc492216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d292621140 con 0x55d292743be0 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc48e1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d292743fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc48e1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d292743fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60088/0 (socket says 172.21.15.28:60088) 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc48e1e8640 1 -- 172.21.15.28:0/3890025129 learned_addr learned my addr 172.21.15.28:0/3890025129 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc48e1e8640 1 -- 172.21.15.28:0/3890025129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2927445d0 con 0x55d292743be0 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/3890025129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d292743fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc488054840 tx=0x7fc488014480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73552947eabf855c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc3717fa640 1 -- 172.21.15.28:0/3890025129 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc488021440 con 0x55d292743be0 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc3717fa640 1 -- 172.21.15.28:0/3890025129 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48801f340 con 0x55d292743be0 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.539+0000 7fc3717fa640 1 -- 172.21.15.28:0/3890025129 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4880215e0 con 0x55d292743be0 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 -- 172.21.15.28:0/3890025129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 msgr2=0x55d292743fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 --2- 172.21.15.28:0/3890025129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d292743fe0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc488054840 tx=0x7fc488014480 comp rx=0 tx=0).stop 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 -- 172.21.15.28:0/3890025129 shutdown_connections 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 --2- 172.21.15.28:0/3890025129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d292743fe0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 -- 172.21.15.28:0/3890025129 >> 172.21.15.28:0/3890025129 conn(0x55d29273ffe0 msgr2=0x55d29273e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 -- 172.21.15.28:0/3890025129 shutdown_connections 2024-09-19T11:23:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 -- 172.21.15.28:0/3890025129 wait complete. 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 Processor -- start 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 -- start start 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d29274b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.540+0000 7fc492216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d292744bf0 con 0x55d292743be0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc48e1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d29274b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc48e1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d29274b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60092/0 (socket says 172.21.15.28:60092) 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 learned_addr learned my addr 172.21.15.28:0/1969181096 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29274b720 con 0x55d292743be0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d29274b1c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc488054740 tx=0x7fc4880547a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc35f7fe640 1 -- 172.21.15.28:0/1969181096 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4880459e0 con 0x55d292743be0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc35f7fe640 1 -- 172.21.15.28:0/1969181096 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc488045ba0 con 0x55d292743be0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc35f7fe640 1 -- 172.21.15.28:0/1969181096 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc488054950 con 0x55d292743be0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d29274b9b0 con 0x55d292743be0 2024-09-19T11:23:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.541+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d29274c540 con 0x55d292743be0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.542+0000 7fc35f7fe640 1 -- 172.21.15.28:0/1969181096 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc488054af0 con 0x55d292743be0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.542+0000 7fc35f7fe640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc35803d000 0x7fc35803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.542+0000 7fc35f7fe640 1 -- 172.21.15.28:0/1969181096 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fc488034c80 con 0x55d292743be0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.542+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc35803d000 0x7fc35803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.542+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.542+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292756af0 0x55d292758fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.542+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2927596b0 con 0x55d292756af0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc4851e6640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292756af0 0x55d292758fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc35803d000 0x7fc35803f4c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc4800533f0 tx=0x7fc48004b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc4851e6640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292756af0 0x55d292758fb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 20 realm 2024-09-19T11:23:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292759a80 con 0x55d292756af0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.543+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29275a350 con 0x55d292756af0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc492216800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29275a6f0 con 0x55d292756af0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc492216800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d29275b1a0 0x55d29275d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29275dcf0 con 0x55d29275b1a0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d29275b1a0 0x55d29275d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.544+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d29275b1a0 0x55d29275d590 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc492216800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc492216800 20 searching for the correct realm 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc48e1e8640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc48803c3f0 0x7fc488008970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc48803c3f0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc48803c3f0 0x7fc488008970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.545+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc48803c3f0 0x7fc488008970 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.546+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.546+0000 7fc48d9e7640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc480044040 0x7fc480044800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.546+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.546+0000 7fc4851e6640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc480044040 0x7fc480044800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.546+0000 7fc4851e6640 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc480044040 0x7fc480044800 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.546+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.546+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.547+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc48803c3f0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.548+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.548+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.548+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.548+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.548+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.548+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc48803c3f0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.549+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.550+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc48803c3f0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.551+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.552+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc48803c3f0 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.553+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.553+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d29275b1a0 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.553+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.553+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc480044040 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.553+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.553+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x55d292756af0 2024-09-19T11:23:24.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f6d0 con 0x7fc48803c3f0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29275e7d0 con 0x55d292756af0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29275f070 con 0x55d292756af0 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.554+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29275e410 con 0x55d29275b1a0 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 zone default found 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 4 Realm: () 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:24.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 zonegroup default 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292760050 con 0x7fc480044040 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292761ee0 con 0x55d29275b1a0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.555+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d292762320 con 0x55d29275b1a0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.556+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.556+0000 7fc492216800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.556+0000 7fc492216800 20 started sync module instance, tier type = 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.556+0000 7fc492216800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.556+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292761280 con 0x7fc48803c3f0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.557+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.557+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94362888705696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292760ee0 con 0x7fc48803c3f0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.557+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292763220 con 0x55d292756af0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.558+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94362888705696] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.558+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.558+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94362888711536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292763190 con 0x55d292756af0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.558+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2927640e0 con 0x55d29275b1a0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.559+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.559+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94362888715312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292764090 con 0x55d29275b1a0 2024-09-19T11:23:24.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.559+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2927650a0 con 0x7fc48803c3f0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.559+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94362888711536] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.560+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94362888715312] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.560+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.560+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94362888719184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2927644d0 con 0x7fc48803c3f0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.560+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292765a60 con 0x55d292756af0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.561+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.562+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94362888722320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2927661e0 con 0x55d292756af0 2024-09-19T11:23:24.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.562+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292766d00 con 0x7fc480044040 2024-09-19T11:23:24.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.562+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94362888719184] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.562+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94362888722320] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.563+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.563+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94362888726480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292766cc0 con 0x7fc480044040 2024-09-19T11:23:24.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.563+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2927675a0 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.564+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94362888726480] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.564+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.564+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94362888713744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292766cc0 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.564+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292767bc0 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94362888713744] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94362888728224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292767bc0 con 0x7fc480044040 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 20 add_watcher() i=0 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 20 add_watcher() i=1 2024-09-19T11:23:24.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 20 add_watcher() i=2 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 20 add_watcher() i=3 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 20 add_watcher() i=4 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 20 add_watcher() i=5 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.565+0000 7fc492216800 20 add_watcher() i=6 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.566+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94362888728224] v31'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478054840 con 0x7fc480044040 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.566+0000 7fc492216800 20 add_watcher() i=7 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.566+0000 7fc492216800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.566+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29276af00 con 0x55d292756af0 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.566+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.566+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94362888743200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d292767d30 con 0x55d292756af0 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.567+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94362888743200] v31'90 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc478054840 con 0x55d292756af0 2024-09-19T11:23:24.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.567+0000 7fc33f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc492216800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc492216800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc492216800 5 note: GC not initialized 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc33e7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc492216800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc492216800 20 Filter name: none 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc2affff640 20 reqs_thread_entry: start 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.568+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29279d960 con 0x55d292756af0 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc478081040 con 0x55d292756af0 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc492216800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc492216800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29279e220 con 0x55d292756af0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc478081040 con 0x55d292756af0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc492216800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d29279f230 con 0x7fc48803c3f0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.569+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.570+0000 7fc33f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.570+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94362888743200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x55d292756af0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.571+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94362888743200] v31'91 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc478081040 con 0x55d292756af0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.571+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94362888705696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x7fc48803c3f0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.572+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94362888705696] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.572+0000 7fc492216800 20 remove_watcher() i=0 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.572+0000 7fc492216800 2 removed watcher, disabling cache 2024-09-19T11:23:24.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.572+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94362888711536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x55d292756af0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.573+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94362888711536] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478081040 con 0x55d292756af0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.573+0000 7fc492216800 20 remove_watcher() i=1 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.573+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94362888715312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x55d29275b1a0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.574+0000 7fc48e1e8640 1 -- 172.21.15.28:0/1969181096 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94362888715312] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488053020 con 0x55d29275b1a0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.574+0000 7fc492216800 20 remove_watcher() i=2 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.574+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94362888719184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x7fc48803c3f0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.575+0000 7fc48d9e7640 1 -- 172.21.15.28:0/1969181096 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94362888719184] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc48004f070 con 0x7fc48803c3f0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.575+0000 7fc492216800 20 remove_watcher() i=3 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.575+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94362888722320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x55d292756af0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.576+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94362888722320] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478081040 con 0x55d292756af0 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.576+0000 7fc492216800 20 remove_watcher() i=4 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.576+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94362888726480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x7fc480044040 2024-09-19T11:23:24.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.577+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94362888726480] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478081040 con 0x7fc480044040 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.577+0000 7fc492216800 20 remove_watcher() i=5 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.577+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94362888713744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x7fc480044040 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.579+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94362888713744] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478081040 con 0x7fc480044040 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.579+0000 7fc492216800 20 remove_watcher() i=6 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.579+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94362888728224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d29279b880 con 0x7fc480044040 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94362888728224] v31'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc478081040 con 0x7fc480044040 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 20 remove_watcher() i=7 2024-09-19T11:23:24.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc480044040 msgr2=0x7fc480044800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fc480044040 0x7fc480044800 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292756af0 msgr2=0x55d292758fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292756af0 0x55d292758fb0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc48803c3f0 msgr2=0x7fc488008970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc48803c3f0 0x7fc488008970 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d29275b1a0 msgr2=0x55d29275d590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d29275b1a0 0x55d29275d590 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc35803d000 msgr2=0x7fc35803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc35803d000 0x7fc35803f4c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc4800533f0 tx=0x7fc48004b000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 msgr2=0x55d29274b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/1969181096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292743be0 0x55d29274b1c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc488054740 tx=0x7fc4880547a0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc4851e6640 1 -- 172.21.15.28:0/1969181096 reap_dead start 2024-09-19T11:23:24.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 shutdown_connections 2024-09-19T11:23:24.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 >> 172.21.15.28:0/1969181096 conn(0x55d29273ffe0 msgr2=0x55d29273e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 shutdown_connections 2024-09-19T11:23:24.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/1969181096 wait complete. 2024-09-19T11:23:24.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc36803d000 msgr2=0x7fc36803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc36803d000 0x7fc36803f4c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc4800014e0 tx=0x7fc480053060 comp rx=0 tx=0).stop 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292737120 msgr2=0x55d292737540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.580+0000 7fc492216800 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292737120 0x55d292737540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55d29271d090 tx=0x7fc4880167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 shutdown_connections 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc36803d000 0x7fc36803f4c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/3766398372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292737120 0x55d292737540 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 >> 172.21.15.28:0/3766398372 conn(0x55d29272b4d0 msgr2=0x55d29272b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 shutdown_connections 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/3766398372 wait complete. 2024-09-19T11:23:24.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292626a90 msgr2=0x55d2927236e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292626a90 0x55d2927236e0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d292725ac0 msgr2=0x55d292727f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d292725ac0 0x55d292727f60 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc37403cfb0 msgr2=0x7fc37403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc37403cfb0 0x7fc37403f470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc478056000 tx=0x7fc478050000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 msgr2=0x55d29271c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d29271c090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55d292622200 tx=0x7fc480000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 shutdown_connections 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d292725ac0 0x55d292727f60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc37403cfb0 0x7fc37403f470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d292626a90 0x55d2927236e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 --2- 172.21.15.28:0/2616777466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d292622f20 0x55d29271c090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 >> 172.21.15.28:0/2616777466 conn(0x55d29261ca60 msgr2=0x55d29270d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 shutdown_connections 2024-09-19T11:23:24.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.581+0000 7fc492216800 1 -- 172.21.15.28:0/2616777466 wait complete. 2024-09-19T11:23:24.600 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T11:23:24.600 DEBUG:tasks.util.rgw: json result: [] 2024-09-19T11:23:24.623 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:24.623 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-09-19T11:23:24.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-09-19T11:23:24.624 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-09-19T11:23:24.669 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:24.670 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd4cb32800 1 Processor -- start 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd4cb32800 1 -- start start 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd4cb32800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x555c3e8dcf10 0x555c3e8df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd4cb32800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555c3e8e35c0 con 0x555c3e8df8a0 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd4cb32800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x555c3e7677d0 con 0x555c3e8dcf10 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd487bc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x555c3e8dcf10 0x555c3e8df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd47fbb640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x555c3e8df8a0 0x555c3e8e1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33220/0 (socket says 172.21.15.28:33220) 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd487bc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x555c3e8dcf10 0x555c3e8df300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60106/0 (socket says 172.21.15.28:60106) 2024-09-19T11:23:24.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.669+0000 7fdd47fbb640 1 -- 172.21.15.28:0/2700729507 learned_addr learned my addr 172.21.15.28:0/2700729507 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:24.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.670+0000 7fdd487bc640 1 -- 172.21.15.28:0/2700729507 >> v1:172.21.15.22:6789/0 conn(0x555c3e8df8a0 legacy=0x555c3e8e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.670+0000 7fdd487bc640 1 -- 172.21.15.28:0/2700729507 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3e8e47f0 con 0x555c3e8dcf10 2024-09-19T11:23:24.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.671+0000 7fdd487bc640 1 --2- 172.21.15.28:0/2700729507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8df300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800a470 tx=0x7fdd38045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab32681a47f6daf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd477ba640 1 -- 172.21.15.28:0/2700729507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd38053570 con 0x555c3e8dcf10 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd477ba640 1 -- 172.21.15.28:0/2700729507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38058070 con 0x555c3e8dcf10 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd477ba640 1 -- 172.21.15.28:0/2700729507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd38053880 con 0x555c3e8dcf10 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2700729507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 msgr2=0x555c3e8df300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2700729507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8df300 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800a470 tx=0x7fdd38045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2700729507 shutdown_connections 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2700729507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8df300 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2700729507 >> 172.21.15.28:0/2700729507 conn(0x555c3e8d6a50 msgr2=0x555c3e8d8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2700729507 shutdown_connections 2024-09-19T11:23:24.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.672+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2700729507 wait complete. 2024-09-19T11:23:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd4cb32800 1 Processor -- start 2024-09-19T11:23:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd4cb32800 1 -- start start 2024-09-19T11:23:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd4cb32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8dc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd4cb32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c3e774dd0 con 0x555c3e8dcf10 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd487bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd487bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8dc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60112/0 (socket says 172.21.15.28:60112) 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd487bc640 1 -- 172.21.15.28:0/3741593037 learned_addr learned my addr 172.21.15.28:0/3741593037 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.688+0000 7fdd487bc640 1 -- 172.21.15.28:0/3741593037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3e8dcc70 con 0x555c3e8dcf10 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.689+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3741593037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8dc680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdd38025c20 tx=0x7fdd38051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabbaf0864d5b4df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.689+0000 7fdc3660c640 1 -- 172.21.15.28:0/3741593037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd3805c580 con 0x555c3e8dcf10 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.689+0000 7fdc3660c640 1 -- 172.21.15.28:0/3741593037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38063030 con 0x555c3e8dcf10 2024-09-19T11:23:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.689+0000 7fdc3660c640 1 -- 172.21.15.28:0/3741593037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd3805c850 con 0x555c3e8dcf10 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3741593037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 msgr2=0x555c3e8dc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3741593037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8dc680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdd38025c20 tx=0x7fdd38051000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3741593037 shutdown_connections 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3741593037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e8dc680 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3741593037 >> 172.21.15.28:0/3741593037 conn(0x555c3e8d6a50 msgr2=0x555c3e9c7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3741593037 shutdown_connections 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3741593037 wait complete. 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 Processor -- start 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 -- start start 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e9d5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd4cb32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c3e8e3a90 con 0x555c3e8dcf10 2024-09-19T11:23:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd487bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e9d5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd487bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e9d5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60128/0 (socket says 172.21.15.28:60128) 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.690+0000 7fdd487bc640 1 -- 172.21.15.28:0/2481186990 learned_addr learned my addr 172.21.15.28:0/2481186990 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.691+0000 7fdd487bc640 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3e9d6500 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.691+0000 7fdd487bc640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e9d5fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd38045f50 tx=0x7fdd38002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.691+0000 7fdc34e09640 1 -- 172.21.15.28:0/2481186990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd38054ad0 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.691+0000 7fdc34e09640 1 -- 172.21.15.28:0/2481186990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38054c70 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.691+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555c3e9d6790 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.691+0000 7fdc34e09640 1 -- 172.21.15.28:0/2481186990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd38068420 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.691+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c3e9d7190 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.692+0000 7fdc34e09640 1 -- 172.21.15.28:0/2481186990 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdd380685c0 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.692+0000 7fdc34e09640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc3003d000 0x7fdc3003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.692+0000 7fdc34e09640 1 -- 172.21.15.28:0/2481186990 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fdd38070030 con 0x555c3e8dcf10 2024-09-19T11:23:24.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.692+0000 7fdd47fbb640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc3003d000 0x7fdc3003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.692+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3e8df8a0 0x555c3e9dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.692+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3e9ddb30 con 0x555c3e8df8a0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.692+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3e8df8a0 0x555c3e9dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd47fbb640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc3003d000 0x7fdc3003f4c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c001d70 tx=0x7fdd3c0012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3e8df8a0 0x555c3e9dd5f0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd48fbd640 1 -- 172.21.15.28:0/2481186990 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdd4000a4d0 con 0x555c3e8df8a0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3e9dded0 con 0x555c3e8df8a0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd48fbd640 1 -- 172.21.15.28:0/2481186990 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdd4000a4d0 con 0x555c3e8df8a0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3e9dfa10 0x555c3e9e1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3e9e2610 con 0x555c3e9dfa10 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.693+0000 7fdd487bc640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3e9dfa10 0x555c3e9e1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.694+0000 7fdd487bc640 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3e9dfa10 0x555c3e9e1e80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.694+0000 7fdd487bc640 1 -- 172.21.15.28:0/2481186990 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fdd38051070 con 0x555c3e9dfa10 2024-09-19T11:23:24.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.694+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3e9de720 con 0x555c3e8df8a0 2024-09-19T11:23:24.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.694+0000 7fdd48fbd640 1 -- 172.21.15.28:0/2481186990 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdd4000a4d0 con 0x555c3e8df8a0 2024-09-19T11:23:24.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.694+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3e9deb40 con 0x555c3e9dfa10 2024-09-19T11:23:24.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd487bc640 1 -- 172.21.15.28:0/2481186990 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fdd38051070 con 0x555c3e9dfa10 2024-09-19T11:23:24.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd4cb32800 1 Processor -- start 2024-09-19T11:23:24.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd4cb32800 1 -- start start 2024-09-19T11:23:24.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd4cb32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9f0f70 0x555c3e9f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd4cb32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c3e7d34a0 con 0x555c3e9f0f70 2024-09-19T11:23:24.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd48fbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9f0f70 0x555c3e9f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd48fbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9f0f70 0x555c3e9f1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60142/0 (socket says 172.21.15.28:60142) 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.695+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3111749904 learned_addr learned my addr 172.21.15.28:0/3111749904 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.696+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3111749904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3e9f19d0 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.696+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9f0f70 0x555c3e9f1390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x555c3e9d7360 tx=0x7fdd400167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.696+0000 7fdc2dffb640 1 -- 172.21.15.28:0/3111749904 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd40013640 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.696+0000 7fdc2dffb640 1 -- 172.21.15.28:0/3111749904 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd400137e0 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.696+0000 7fdc2dffb640 1 -- 172.21.15.28:0/3111749904 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd40013b00 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.696+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555c3e9f2200 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.696+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c3e9f27a0 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.697+0000 7fdc2dffb640 1 -- 172.21.15.28:0/3111749904 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdd40011040 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.697+0000 7fdc2dffb640 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc2003d000 0x7fdc2003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.697+0000 7fdc2dffb640 1 -- 172.21.15.28:0/3111749904 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fdd4000e040 con 0x555c3e9f0f70 2024-09-19T11:23:24.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.697+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc2003d000 0x7fdc2003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.697+0000 7fdd4cb32800 1 Processor -- start 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.697+0000 7fdd4cb32800 1 -- start start 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc2003d000 0x7fdc2003f4c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x555c3e9e29b0 tx=0x7fdd38074f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd4cb32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3e9fde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd4cb32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c3e8db0c0 con 0x555c3e9fda30 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd48fbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3e9fde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd48fbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3e9fde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60146/0 (socket says 172.21.15.28:60146) 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd48fbd640 1 -- 172.21.15.28:0/1449678942 learned_addr learned my addr 172.21.15.28:0/1449678942 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd48fbd640 1 -- 172.21.15.28:0/1449678942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3e9fe420 con 0x555c3e9fda30 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/1449678942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3e9fde30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdd40054840 tx=0x7fdd4001f340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623c71ecd3c6a0df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdc1f7fe640 1 -- 172.21.15.28:0/1449678942 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd40013ca0 con 0x555c3e9fda30 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdc1f7fe640 1 -- 172.21.15.28:0/1449678942 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd40052070 con 0x555c3e9fda30 2024-09-19T11:23:24.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.698+0000 7fdc1f7fe640 1 -- 172.21.15.28:0/1449678942 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd4002a3f0 con 0x555c3e9fda30 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 -- 172.21.15.28:0/1449678942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 msgr2=0x555c3e9fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/1449678942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3e9fde30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdd40054840 tx=0x7fdd4001f340 comp rx=0 tx=0).stop 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 -- 172.21.15.28:0/1449678942 shutdown_connections 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/1449678942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3e9fde30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 -- 172.21.15.28:0/1449678942 >> 172.21.15.28:0/1449678942 conn(0x555c3e9f9e30 msgr2=0x555c3e9f8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 -- 172.21.15.28:0/1449678942 shutdown_connections 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 -- 172.21.15.28:0/1449678942 wait complete. 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 Processor -- start 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 -- start start 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3ea04ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.699+0000 7fdd4cb32800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555c3e9fea40 con 0x555c3e9fda30 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdd48fbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3ea04ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdd48fbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3ea04ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60154/0 (socket says 172.21.15.28:60154) 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 learned_addr learned my addr 172.21.15.28:0/3447129954 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c3ea05550 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3ea04ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdd4001f380 tx=0x7fdd40054870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdc1dffb640 1 -- 172.21.15.28:0/3447129954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd40045cc0 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdc1dffb640 1 -- 172.21.15.28:0/3447129954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd40054a20 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdc1dffb640 1 -- 172.21.15.28:0/3447129954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd40045e60 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555c3ea057e0 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.700+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c3ea06160 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.701+0000 7fdc1dffb640 1 -- 172.21.15.28:0/3447129954 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdd40054d00 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.701+0000 7fdc1dffb640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc1003d050 0x7fdc1003f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.701+0000 7fdc1dffb640 1 -- 172.21.15.28:0/3447129954 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fdd4004d050 con 0x555c3e9fda30 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.701+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc1003d050 0x7fdc1003f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3ea108c0 0x555c3ea12d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea134d0 con 0x555c3ea108c0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc1003d050 0x7fdc1003f510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009800 tx=0x7fdd38092000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd47fbb640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3ea108c0 0x555c3ea12d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd47fbb640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3ea108c0 0x555c3ea12d80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdd3c009050 con 0x555c3ea108c0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd4cb32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd4cb32800 20 realm 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.702+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea0fc10 con 0x555c3ea108c0 2024-09-19T11:23:24.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdd3c009050 con 0x555c3ea108c0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea13fb0 con 0x555c3ea108c0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdd3c009050 con 0x555c3ea108c0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea14880 con 0x555c3ea108c0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdd3c009050 con 0x555c3ea108c0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3ea151c0 0x555c3ea175b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea17cf0 con 0x555c3ea151c0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.703+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3ea151c0 0x555c3ea175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3ea151c0 0x555c3ea175b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd4cb32800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd4cb32800 20 searching for the correct realm 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd48fbd640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdd40033a10 0x7fdd4004e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd40033a10 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdd40033a10 0x7fdd4004e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.704+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdd40033a10 0x7fdd4004e9e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.705+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.705+0000 7fdd487bc640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdd380a6040 0x7fdd38098890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.705+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.705+0000 7fdd47fbb640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdd380a6040 0x7fdd38098890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.705+0000 7fdd47fbb640 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdd380a6040 0x7fdd38098890 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.705+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c009050 con 0x7fdd380a6040 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.705+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.706+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c009050 con 0x555c3ea108c0 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.706+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.706+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.706+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.706+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.706+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd40033a10 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.707+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd40033a10 2024-09-19T11:23:24.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.708+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.709+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd40033a10 2024-09-19T11:23:24.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.710+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.711+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.711+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.711+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.711+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.711+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.711+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.712+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.712+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd40033a10 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.712+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.712+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea151c0 2024-09-19T11:23:24.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.713+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.713+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd380a6040 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.713+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.713+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x555c3ea108c0 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.713+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.713+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea196a0 con 0x7fdd40033a10 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea18770 con 0x555c3ea108c0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea18380 con 0x555c3ea108c0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea19060 con 0x555c3ea151c0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 zone default found 2024-09-19T11:23:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 4 Realm: () 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 zonegroup default 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.714+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea18c30 con 0x7fdd380a6040 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1bf40 con 0x555c3ea151c0 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1c360 con 0x555c3ea151c0 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 20 started sync module instance, tier type = 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.715+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1c330 con 0x7fdd40033a10 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.716+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.717+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93854676136912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1c7a0 con 0x7fdd40033a10 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.717+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1cff0 con 0x555c3ea108c0 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.718+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93854676136912] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.718+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.718+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93854676142912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1d410 con 0x555c3ea108c0 2024-09-19T11:23:24.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.718+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1ddd0 con 0x555c3ea151c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.719+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93854676142912] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.719+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.719+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93854676144448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1ddd0 con 0x555c3ea151c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.719+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1e7d0 con 0x7fdd40033a10 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.720+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93854676144448] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.720+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.720+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93854676146464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1dd40 con 0x7fdd40033a10 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.720+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1f690 con 0x555c3ea108c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.721+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.721+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93854676153280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea1fdd0 con 0x555c3ea108c0 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.721+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea208f0 con 0x7fdd380a6040 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.721+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93854676146464] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.722+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93854676153280] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.722+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.722+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93854676157376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea208b0 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.722+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea21190 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.723+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93854676157376] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.723+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.723+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93854676151232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea208b0 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.723+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea217b0 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.724+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93854676151232] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93854676159632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea217b0 con 0x7fdd380a6040 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 20 add_watcher() i=0 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 20 add_watcher() i=1 2024-09-19T11:23:24.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 20 add_watcher() i=2 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 20 add_watcher() i=3 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 20 add_watcher() i=4 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 20 add_watcher() i=5 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.725+0000 7fdd4cb32800 20 add_watcher() i=6 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.726+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93854676159632] v31'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.726+0000 7fdd4cb32800 20 add_watcher() i=7 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.726+0000 7fdd4cb32800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.726+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea24ae0 con 0x555c3ea108c0 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.726+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.726+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93854676174096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea21920 con 0x555c3ea108c0 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.727+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93854676174096] v31'92 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.727+0000 7fdbfefdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.727+0000 7fdd4cb32800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.727+0000 7fdd4cb32800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.727+0000 7fdd4cb32800 5 note: GC not initialized 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.727+0000 7fdbfdfdb640 20 reqs_thread_entry: start 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.727+0000 7fdd4cb32800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd4cb32800 20 Filter name: none 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdb6f7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea575e0 con 0x555c3ea108c0 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd4cb32800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd4cb32800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea57ea0 con 0x555c3ea108c0 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.728+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.729+0000 7fdd4cb32800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.729+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea58eb0 con 0x7fdd40033a10 2024-09-19T11:23:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.729+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.730+0000 7fdbfefdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.730+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93854676174096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x555c3ea108c0 2024-09-19T11:23:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.731+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93854676174096] v31'93 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.731+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93854676136912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x7fdd40033a10 2024-09-19T11:23:24.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.732+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93854676136912] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.732+0000 7fdd4cb32800 20 remove_watcher() i=0 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.732+0000 7fdd4cb32800 2 removed watcher, disabling cache 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.732+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93854676142912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x555c3ea108c0 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.733+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93854676142912] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.733+0000 7fdd4cb32800 20 remove_watcher() i=1 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.733+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93854676144448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x555c3ea151c0 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.734+0000 7fdd48fbd640 1 -- 172.21.15.28:0/3447129954 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93854676144448] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd4003a070 con 0x555c3ea151c0 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.734+0000 7fdd4cb32800 20 remove_watcher() i=2 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.734+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93854676146464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x7fdd40033a10 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.735+0000 7fdd487bc640 1 -- 172.21.15.28:0/3447129954 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93854676146464] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3808a050 con 0x7fdd40033a10 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.735+0000 7fdd4cb32800 20 remove_watcher() i=3 2024-09-19T11:23:24.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.735+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93854676153280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x555c3ea108c0 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.736+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93854676153280] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x555c3ea108c0 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.736+0000 7fdd4cb32800 20 remove_watcher() i=4 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.736+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93854676157376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x7fdd380a6040 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.737+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93854676157376] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.738+0000 7fdd4cb32800 20 remove_watcher() i=5 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.738+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93854676151232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x7fdd380a6040 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.739+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93854676151232] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.739+0000 7fdd4cb32800 20 remove_watcher() i=6 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.739+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93854676159632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x555c3ea55540 con 0x7fdd380a6040 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93854676159632] v31'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdd3c029040 con 0x7fdd380a6040 2024-09-19T11:23:24.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 20 remove_watcher() i=7 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdd380a6040 msgr2=0x7fdd38098890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdd380a6040 0x7fdd38098890 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3ea108c0 msgr2=0x555c3ea12d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3ea108c0 0x555c3ea12d80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdd40033a10 msgr2=0x7fdd4004e9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdd40033a10 0x7fdd4004e9e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3ea151c0 msgr2=0x555c3ea175b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3ea151c0 0x555c3ea175b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc1003d050 msgr2=0x7fdc1003f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc1003d050 0x7fdc1003f510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009800 tx=0x7fdd38092000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 msgr2=0x555c3ea04ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3ea04ff0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdd4001f380 tx=0x7fdd40054870 comp rx=0 tx=0).stop 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd47fbb640 1 -- 172.21.15.28:0/3447129954 reap_dead start 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 shutdown_connections 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3447129954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9fda30 0x555c3ea04ff0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 >> 172.21.15.28:0/3447129954 conn(0x555c3e9f9e30 msgr2=0x555c3e9f8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 shutdown_connections 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.740+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3447129954 wait complete. 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc2003d000 msgr2=0x7fdc2003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc2003d000 0x7fdc2003f4c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x555c3e9e29b0 tx=0x7fdd38074f50 comp rx=0 tx=0).stop 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9f0f70 msgr2=0x555c3e9f1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9f0f70 0x555c3e9f1390 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x555c3e9d7360 tx=0x7fdd400167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 shutdown_connections 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc2003d000 0x7fdc2003f4c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/3111749904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e9f0f70 0x555c3e9f1390 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 >> 172.21.15.28:0/3111749904 conn(0x555c3e9e5320 msgr2=0x555c3e9e5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 shutdown_connections 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/3111749904 wait complete. 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3e8df8a0 msgr2=0x555c3e9dd5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3e8df8a0 0x555c3e9dd5f0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3e9dfa10 msgr2=0x555c3e9e1e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3e9dfa10 0x555c3e9e1e80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc3003d000 msgr2=0x7fdc3003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc3003d000 0x7fdc3003f4c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c001d70 tx=0x7fdd3c0012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 msgr2=0x555c3e9d5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e9d5fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd38045f50 tx=0x7fdd38002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 shutdown_connections 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x555c3e9dfa10 0x555c3e9e1e80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdc3003d000 0x7fdc3003f4c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x555c3e8df8a0 0x555c3e9dd5f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 --2- 172.21.15.28:0/2481186990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555c3e8dcf10 0x555c3e9d5fa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 >> 172.21.15.28:0/2481186990 conn(0x555c3e8d6a50 msgr2=0x555c3e9c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 shutdown_connections 2024-09-19T11:23:24.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.741+0000 7fdd4cb32800 1 -- 172.21.15.28:0/2481186990 wait complete. 2024-09-19T11:23:24.753 INFO:teuthology.orchestra.run.smithi028.stdout:["myfoo"] 2024-09-19T11:23:24.753 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-09-19T11:23:24.757 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:24.757 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-09-19T11:23:24.757 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-09-19T11:23:24.757 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-09-19T11:23:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.800+0000 7f0dd8b22800 1 Processor -- start 2024-09-19T11:23:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.800+0000 7f0dd8b22800 1 -- start start 2024-09-19T11:23:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd8b22800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e5dbeaed10 0x55e5dbeaf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd8b22800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e5dbeb0210 con 0x55e5dbeaf650 2024-09-19T11:23:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd8b22800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55e5dbd3b7d0 con 0x55e5dbeaed10 2024-09-19T11:23:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd47d2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e5dbeaed10 0x55e5dbeaf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd47d2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e5dbeaed10 0x55e5dbeaf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60160/0 (socket says 172.21.15.28:60160) 2024-09-19T11:23:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd47d2640 1 -- 172.21.15.28:0/609500740 learned_addr learned my addr 172.21.15.28:0/609500740 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd3fd1640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55e5dbeaf650 0x55e5dbeafb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33224/0 (socket says 172.21.15.28:33224) 2024-09-19T11:23:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd47d2640 1 -- 172.21.15.28:0/609500740 >> v1:172.21.15.22:6789/0 conn(0x55e5dbeaf650 legacy=0x55e5dbeafb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.801+0000 7f0dd47d2640 1 -- 172.21.15.28:0/609500740 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dbeb03e0 con 0x55e5dbeaed10 2024-09-19T11:23:24.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.803+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/609500740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbeaf110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400a520 tx=0x7f0dc4045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e49627b8ec42cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.803+0000 7f0dd37d0640 1 -- 172.21.15.28:0/609500740 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc4055070 con 0x55e5dbeaed10 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd37d0640 1 -- 172.21.15.28:0/609500740 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc40515b0 con 0x55e5dbeaed10 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd37d0640 1 -- 172.21.15.28:0/609500740 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc40518d0 con 0x55e5dbeaed10 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd8b22800 1 -- 172.21.15.28:0/609500740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 msgr2=0x55e5dbeaf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/609500740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbeaf110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400a520 tx=0x7f0dc4045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd8b22800 1 -- 172.21.15.28:0/609500740 shutdown_connections 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/609500740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbeaf110 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd8b22800 1 -- 172.21.15.28:0/609500740 >> 172.21.15.28:0/609500740 conn(0x55e5dbeaaa40 msgr2=0x55e5dbeacea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd8b22800 1 -- 172.21.15.28:0/609500740 shutdown_connections 2024-09-19T11:23:24.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.804+0000 7f0dd8b22800 1 -- 172.21.15.28:0/609500740 wait complete. 2024-09-19T11:23:24.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.819+0000 7f0dd8b22800 1 Processor -- start 2024-09-19T11:23:24.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.819+0000 7f0dd8b22800 1 -- start start 2024-09-19T11:23:24.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.820+0000 7f0dd8b22800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.820+0000 7f0dd8b22800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5dbd48dd0 con 0x55e5dbeaed10 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.820+0000 7f0dd47d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.820+0000 7f0dd47d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60176/0 (socket says 172.21.15.28:60176) 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.820+0000 7f0dd47d2640 1 -- 172.21.15.28:0/435814602 learned_addr learned my addr 172.21.15.28:0/435814602 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.820+0000 7f0dd47d2640 1 -- 172.21.15.28:0/435814602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dbfa2b10 con 0x55e5dbeaed10 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.820+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/435814602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa2520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4025da0 tx=0x7f0dc4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dd4950367dcc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0cc260c640 1 -- 172.21.15.28:0/435814602 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc4051950 con 0x55e5dbeaed10 2024-09-19T11:23:24.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0cc260c640 1 -- 172.21.15.28:0/435814602 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc4051af0 con 0x55e5dbeaed10 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0cc260c640 1 -- 172.21.15.28:0/435814602 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc4051dd0 con 0x55e5dbeaed10 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0dd8b22800 1 -- 172.21.15.28:0/435814602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 msgr2=0x55e5dbfa2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/435814602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa2520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4025da0 tx=0x7f0dc4051000 comp rx=0 tx=0).stop 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0dd8b22800 1 -- 172.21.15.28:0/435814602 shutdown_connections 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/435814602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa2520 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0dd8b22800 1 -- 172.21.15.28:0/435814602 >> 172.21.15.28:0/435814602 conn(0x55e5dbeaaa40 msgr2=0x55e5dbf9dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0dd8b22800 1 -- 172.21.15.28:0/435814602 shutdown_connections 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.821+0000 7f0dd8b22800 1 -- 172.21.15.28:0/435814602 wait complete. 2024-09-19T11:23:24.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd8b22800 1 Processor -- start 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd8b22800 1 -- start start 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd8b22800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd8b22800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5dbeb0970 con 0x55e5dbeaed10 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd47d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd47d2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60188/0 (socket says 172.21.15.28:60188) 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd47d2640 1 -- 172.21.15.28:0/537514148 learned_addr learned my addr 172.21.15.28:0/537514148 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.822+0000 7f0dd47d2640 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dbfaa450 con 0x55e5dbeaed10 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.823+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa9ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4067040 tx=0x7f0dc4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.823+0000 7f0cc0e09640 1 -- 172.21.15.28:0/537514148 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc4073070 con 0x55e5dbeaed10 2024-09-19T11:23:24.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.823+0000 7f0cc0e09640 1 -- 172.21.15.28:0/537514148 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc405a450 con 0x55e5dbeaed10 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.823+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5dbfaa6e0 con 0x55e5dbeaed10 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.823+0000 7f0cc0e09640 1 -- 172.21.15.28:0/537514148 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dc405a730 con 0x55e5dbeaed10 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.823+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5dbfab2f0 con 0x55e5dbeaed10 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0cc0e09640 1 -- 172.21.15.28:0/537514148 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0dc4076070 con 0x55e5dbeaed10 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0cc0e09640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cbc03d000 0x7f0cbc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0cc0e09640 1 -- 172.21.15.28:0/537514148 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0dc4062070 con 0x55e5dbeaed10 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0dd3fd1640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cbc03d000 0x7f0cbc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbeaf650 0x55e5dbfb1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfb1a80 con 0x55e5dbeaf650 2024-09-19T11:23:24.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbeaf650 0x55e5dbfb1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.824+0000 7f0dd3fd1640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cbc03d000 0x7f0cbc03f4c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001e30 tx=0x7f0dc8001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.825+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbeaf650 0x55e5dbfb1540 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.825+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/537514148 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbeaf650 2024-09-19T11:23:24.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.825+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfb1e20 con 0x55e5dbeaf650 2024-09-19T11:23:24.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.825+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/537514148 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbeaf650 2024-09-19T11:23:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.825+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfb38e0 0x55e5dbfb5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.825+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfb6510 con 0x55e5dbfb38e0 2024-09-19T11:23:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.825+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfb38e0 0x55e5dbfb5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.826+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfb38e0 0x55e5dbfb5d80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.826+0000 7f0dd47d2640 1 -- 172.21.15.28:0/537514148 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0dc4075030 con 0x55e5dbfb38e0 2024-09-19T11:23:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.826+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfb2670 con 0x55e5dbeaf650 2024-09-19T11:23:24.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.826+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/537514148 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbeaf650 2024-09-19T11:23:24.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.826+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfb2a10 con 0x55e5dbfb38e0 2024-09-19T11:23:24.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.826+0000 7f0dd47d2640 1 -- 172.21.15.28:0/537514148 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0dc4075030 con 0x55e5dbfb38e0 2024-09-19T11:23:24.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.827+0000 7f0dd8b22800 1 Processor -- start 2024-09-19T11:23:24.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.827+0000 7f0dd8b22800 1 -- start start 2024-09-19T11:23:24.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.827+0000 7f0dd8b22800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfc4e70 0x55e5dbfc5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.827+0000 7f0dd8b22800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5dbda74a0 con 0x55e5dbfc4e70 2024-09-19T11:23:24.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.827+0000 7f0dd4fd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfc4e70 0x55e5dbfc5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.827+0000 7f0dd4fd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfc4e70 0x55e5dbfc5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60204/0 (socket says 172.21.15.28:60204) 2024-09-19T11:23:24.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.827+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/2589655188 learned_addr learned my addr 172.21.15.28:0/2589655188 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.828+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/2589655188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dbfc58d0 con 0x55e5dbfc4e70 2024-09-19T11:23:24.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.828+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfc4e70 0x55e5dbfc5290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55e5dbfaad80 tx=0x7f0dcc0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.828+0000 7f0cb9ffb640 1 -- 172.21.15.28:0/2589655188 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dcc013640 con 0x55e5dbfc4e70 2024-09-19T11:23:24.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.828+0000 7f0cb9ffb640 1 -- 172.21.15.28:0/2589655188 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0137e0 con 0x55e5dbfc4e70 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.828+0000 7f0cb9ffb640 1 -- 172.21.15.28:0/2589655188 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dcc013ae0 con 0x55e5dbfc4e70 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.828+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5dbfc6100 con 0x55e5dbfc4e70 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.828+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5dbfc66a0 con 0x55e5dbfc4e70 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.829+0000 7f0cb9ffb640 1 -- 172.21.15.28:0/2589655188 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0dcc011040 con 0x55e5dbfc4e70 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.829+0000 7f0cb9ffb640 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cac03d000 0x7f0cac03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.829+0000 7f0cb9ffb640 1 -- 172.21.15.28:0/2589655188 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0dcc00e040 con 0x55e5dbfc4e70 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.829+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cac03d000 0x7f0cac03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.829+0000 7f0dd8b22800 1 Processor -- start 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.829+0000 7f0dd8b22800 1 -- start start 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.829+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cac03d000 0x7f0cac03f4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55e5dbfabff0 tx=0x7f0dc4096fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0dd8b22800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0dd8b22800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5dbfa2e80 con 0x55e5dbfd18f0 2024-09-19T11:23:24.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0dd4fd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0dd4fd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60212/0 (socket says 172.21.15.28:60212) 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/2987409180 learned_addr learned my addr 172.21.15.28:0/2987409180 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/2987409180 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dbfd22e0 con 0x55e5dbfd18f0 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/2987409180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd1cf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc016cd0 tx=0x7f0dcc020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a60d5d84d81501 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0cab7fe640 1 -- 172.21.15.28:0/2987409180 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dcc051070 con 0x55e5dbfd18f0 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0cab7fe640 1 -- 172.21.15.28:0/2987409180 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dcc01fdb0 con 0x55e5dbfd18f0 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.830+0000 7f0cab7fe640 1 -- 172.21.15.28:0/2987409180 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dcc053440 con 0x55e5dbfd18f0 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2987409180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 msgr2=0x55e5dbfd1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/2987409180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd1cf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc016cd0 tx=0x7f0dcc020460 comp rx=0 tx=0).stop 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2987409180 shutdown_connections 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/2987409180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd1cf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2987409180 >> 172.21.15.28:0/2987409180 conn(0x55e5dbfcdcf0 msgr2=0x55e5dbfcbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2987409180 shutdown_connections 2024-09-19T11:23:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2987409180 wait complete. 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 Processor -- start 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 -- start start 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd8b22800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5dbfd2900 con 0x55e5dbfd18f0 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd4fd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd4fd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60226/0 (socket says 172.21.15.28:60226) 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.831+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 learned_addr learned my addr 172.21.15.28:0/915358029 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.832+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5dbfd94a0 con 0x55e5dbfd18f0 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.832+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd8f40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc056420 tx=0x7f0dcc054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.832+0000 7f0ca9ffb640 1 -- 172.21.15.28:0/915358029 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dcc021440 con 0x55e5dbfd18f0 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.832+0000 7f0ca9ffb640 1 -- 172.21.15.28:0/915358029 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dcc04c070 con 0x55e5dbfd18f0 2024-09-19T11:23:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.832+0000 7f0ca9ffb640 1 -- 172.21.15.28:0/915358029 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dcc01f340 con 0x55e5dbfd18f0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.832+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5dbfd9730 con 0x55e5dbfd18f0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.832+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5dbfd9730 con 0x55e5dbfd18f0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0ca9ffb640 1 -- 172.21.15.28:0/915358029 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0dcc0459d0 con 0x55e5dbfd18f0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0ca9ffb640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c9c03d000 0x7f0c9c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0ca9ffb640 1 -- 172.21.15.28:0/915358029 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0dcc004720 con 0x55e5dbfd18f0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c9c03d000 0x7f0c9c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbfe4630 0x55e5dbfe6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfe7240 con 0x55e5dbfe4630 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.833+0000 7f0dd3fd1640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbfe4630 0x55e5dbfe6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c9c03d000 0x7f0c9c03f4c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0dc401bb90 tx=0x7f0dc40873b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd3fd1640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbfe4630 0x55e5dbfe6af0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0dc800f610 con 0x55e5dbfe4630 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 20 realm 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfe3980 con 0x55e5dbfe4630 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0dc800f610 con 0x55e5dbfe4630 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfe7d20 con 0x55e5dbfe4630 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0dc800f610 con 0x55e5dbfe4630 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.834+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfe85f0 con 0x55e5dbfe4630 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0dc800f610 con 0x55e5dbfe4630 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd8b22800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfe8f30 0x55e5dbfeb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfeba60 con 0x55e5dbfe8f30 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfe8f30 0x55e5dbfeb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfe8f30 0x55e5dbfeb320 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd8b22800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd8b22800 20 searching for the correct realm 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.835+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd4fd3640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0dcc014ac0 0x7f0dcc0123d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dcc014ac0 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0dcc014ac0 0x7f0dcc0123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0dcc014ac0 0x7f0dcc0123d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd47d2640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0dc40a6040 0x7f0dc4050610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.836+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.837+0000 7f0dd3fd1640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0dc40a6040 0x7f0dc4050610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.837+0000 7f0dd3fd1640 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0dc40a6040 0x7f0dc4050610 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.837+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc800f610 con 0x7f0dc40a6040 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.837+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.837+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc800f610 con 0x55e5dbfe4630 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.837+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dcc014ac0 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.838+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.839+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dcc014ac0 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.840+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0dc802b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.841+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dcc014ac0 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc802b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.842+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.843+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.843+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.843+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.843+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dcc014ac0 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe8f30 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dc40a6040 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.844+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x55e5dbfe4630 2024-09-19T11:23:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfed410 con 0x7f0dcc014ac0 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec4e0 con 0x55e5dbfe4630 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec120 con 0x55e5dbfe4630 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.845+0000 7f0dd8b22800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfece40 con 0x55e5dbfe8f30 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 zone default found 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 4 Realm: () 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 zonegroup default 2024-09-19T11:23:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec9b0 con 0x7f0dc40a6040 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfef7d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 20 rados->read ofs=0 len=0 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.846+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfefc90 con 0x55e5dbfe8f30 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.847+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.847+0000 7f0dd8b22800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.847+0000 7f0dd8b22800 20 started sync module instance, tier type = 2024-09-19T11:23:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.847+0000 7f0dd8b22800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.847+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfeec70 con 0x7f0dcc014ac0 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.848+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.848+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94445726789728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfee8d0 con 0x7f0dcc014ac0 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.848+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff0bc0 con 0x55e5dbfe4630 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.849+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94445726789728] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.849+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.849+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94445726795536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff0fe0 con 0x55e5dbfe4630 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.849+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff19a0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.850+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.850+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94445726799088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff1950 con 0x55e5dbfe8f30 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.850+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dcc014ac0 2024-09-19T11:23:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.850+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94445726795536] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.851+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94445726799088] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.851+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.851+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94445726801296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff0b30 con 0x7f0dcc014ac0 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.851+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff3250 con 0x55e5dbfe4630 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.852+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.852+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94445726805344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff3200 con 0x55e5dbfe4630 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.852+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff4220 con 0x7f0dc40a6040 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.852+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94445726801296] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.853+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94445726805344] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.853+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.853+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94445726809360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff41e0 con 0x7f0dc40a6040 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.853+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff4ac0 con 0x7f0dc40a6040 2024-09-19T11:23:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.854+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.854+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94445726811664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff3760 con 0x7f0dc40a6040 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.854+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff55b0 con 0x7f0dc40a6040 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.854+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94445726809360] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94445726811664] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94445726814432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff4a30 con 0x7f0dc40a6040 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 20 add_watcher() i=0 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 20 add_watcher() i=1 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 20 add_watcher() i=2 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 20 add_watcher() i=3 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 20 add_watcher() i=4 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 20 add_watcher() i=5 2024-09-19T11:23:24.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.855+0000 7f0dd8b22800 20 add_watcher() i=6 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.856+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94445726814432] v31'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.856+0000 7f0dd8b22800 20 add_watcher() i=7 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.856+0000 7f0dd8b22800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.856+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff85e0 con 0x55e5dbfe4630 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.857+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.857+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94445726803040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff5a60 con 0x55e5dbfe4630 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94445726803040] v31'94 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0c8afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0dd8b22800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0dd8b22800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0dd8b22800 5 note: GC not initialized 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0c89fdb640 20 reqs_thread_entry: start 2024-09-19T11:23:24.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0dd8b22800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0dd8b22800 20 Filter name: none 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0bfb7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.858+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dc02ab80 con 0x55e5dbfe8f30 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.859+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(62 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.859+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfefc90 con 0x55e5dbfe8f30 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.859+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(63 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.859+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfefc90 con 0x55e5dbfe4630 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.859+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(64 [pgnls start_epoch 31 out=49b] v31'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.859+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec120 con 0x55e5dbfe4630 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 [pgnls start_epoch 31 out=49b] v31'11 uv11 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec120 con 0x7f0dcc014ac0 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(67 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe4630 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(69 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.860+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dcc014ac0 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.861+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 [pgnls start_epoch 31 out=49b] v31'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.861+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.861+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(71 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.861+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe4630 2024-09-19T11:23:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.861+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(72 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.861+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.862+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.862+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.862+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(74 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.862+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.862+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(75 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.862+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(76 [pgnls start_epoch 31 out=127b] v31'1 uv1 ondisk = 1) ==== 144+0+127 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(77 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(78 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(79 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.863+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dcc014ac0 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(80 [pgnls start_epoch 31 out=70b] v31'2 uv2 ondisk = 1) ==== 144+0+70 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe4630 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(81 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe4630 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(82 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(83 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.864+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dcc014ac0 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(84 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(85 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe4630 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(86 [pgnls start_epoch 31 out=49b] v31'1 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(87 [pgnls start_epoch 31 out=49b] v31'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.865+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.866+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(88 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.866+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dcc014ac0 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.866+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(89 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.866+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dc40a6040 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.866+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(90 [pgnls start_epoch 31 out=49b] v31'1 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.866+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe4630 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.867+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(91 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.867+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x55e5dbfe8f30 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.867+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(92 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.867+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbff2970 con 0x7f0dcc014ac0 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.867+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(93 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.867+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.867+0000 7f0dd8b22800 20 RGWRados::pool_iterate: got myfoo 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.868+0000 7f0c8afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.868+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94445726803040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfefc90 con 0x55e5dbfe4630 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.869+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94445726803040] v31'95 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.869+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94445726789728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec120 con 0x7f0dcc014ac0 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.870+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94445726789728] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.870+0000 7f0dd8b22800 20 remove_watcher() i=0 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.870+0000 7f0dd8b22800 2 removed watcher, disabling cache 2024-09-19T11:23:24.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.870+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94445726795536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec120 con 0x55e5dbfe4630 2024-09-19T11:23:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.871+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94445726795536] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.871+0000 7f0dd8b22800 20 remove_watcher() i=1 2024-09-19T11:23:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.871+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94445726799088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dbfec120 con 0x55e5dbfe8f30 2024-09-19T11:23:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.872+0000 7f0dd4fd3640 1 -- 172.21.15.28:0/915358029 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94445726799088] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dcc00a4d0 con 0x55e5dbfe8f30 2024-09-19T11:23:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.872+0000 7f0dd8b22800 20 remove_watcher() i=2 2024-09-19T11:23:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.872+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94445726801296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dc028da0 con 0x7f0dcc014ac0 2024-09-19T11:23:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.873+0000 7f0dd47d2640 1 -- 172.21.15.28:0/915358029 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94445726801296] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc4075030 con 0x7f0dcc014ac0 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.873+0000 7f0dd8b22800 20 remove_watcher() i=3 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.873+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94445726805344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dc028da0 con 0x55e5dbfe4630 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.874+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94445726805344] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x55e5dbfe4630 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.874+0000 7f0dd8b22800 20 remove_watcher() i=4 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.874+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94445726809360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dc028da0 con 0x7f0dc40a6040 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.875+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94445726809360] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.875+0000 7f0dd8b22800 20 remove_watcher() i=5 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.875+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94445726811664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dc028da0 con 0x7f0dc40a6040 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.876+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94445726811664] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.876+0000 7f0dd8b22800 20 remove_watcher() i=6 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.876+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94445726814432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5dc028da0 con 0x7f0dc40a6040 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94445726814432] v31'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0dc803b040 con 0x7f0dc40a6040 2024-09-19T11:23:24.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 20 remove_watcher() i=7 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0dc40a6040 msgr2=0x7f0dc4050610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0dc40a6040 0x7f0dc4050610 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbfe4630 msgr2=0x55e5dbfe6af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbfe4630 0x55e5dbfe6af0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0dcc014ac0 msgr2=0x7f0dcc0123d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0dcc014ac0 0x7f0dcc0123d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfe8f30 msgr2=0x55e5dbfeb320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfe8f30 0x55e5dbfeb320 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c9c03d000 msgr2=0x7f0c9c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0c9c03d000 0x7f0c9c03f4c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0dc401bb90 tx=0x7f0dc40873b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 msgr2=0x55e5dbfd8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/915358029 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfd18f0 0x55e5dbfd8f40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc056420 tx=0x7f0dcc054790 comp rx=0 tx=0).stop 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 reap_dead start 2024-09-19T11:23:24.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd3fd1640 1 -- 172.21.15.28:0/915358029 reap_dead start 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 shutdown_connections 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 >> 172.21.15.28:0/915358029 conn(0x55e5dbfcdcf0 msgr2=0x55e5dbfcbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 shutdown_connections 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/915358029 wait complete. 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cac03d000 msgr2=0x7f0cac03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cac03d000 0x7f0cac03f4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55e5dbfabff0 tx=0x7f0dc4096fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfc4e70 msgr2=0x55e5dbfc5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.878+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfc4e70 0x55e5dbfc5290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55e5dbfaad80 tx=0x7f0dcc0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 shutdown_connections 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cac03d000 0x7f0cac03f4c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/2589655188 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbfc4e70 0x55e5dbfc5290 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 >> 172.21.15.28:0/2589655188 conn(0x55e5dbfb9220 msgr2=0x55e5dbfb9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 shutdown_connections 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/2589655188 wait complete. 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbeaf650 msgr2=0x55e5dbfb1540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbeaf650 0x55e5dbfb1540 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfb38e0 msgr2=0x55e5dbfb5d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfb38e0 0x55e5dbfb5d80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cbc03d000 msgr2=0x7f0cbc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cbc03d000 0x7f0cbc03f4c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001e30 tx=0x7f0dc8001390 comp rx=0 tx=0).stop 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 msgr2=0x55e5dbfa9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa9ef0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4067040 tx=0x7f0dc4002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 shutdown_connections 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5dbfb38e0 0x55e5dbfb5d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f0cbc03d000 0x7f0cbc03f4c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5dbeaf650 0x55e5dbfb1540 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 --2- 172.21.15.28:0/537514148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5dbeaed10 0x55e5dbfa9ef0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:24.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 >> 172.21.15.28:0/537514148 conn(0x55e5dbeaaa40 msgr2=0x55e5dbfa3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:24.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 shutdown_connections 2024-09-19T11:23:24.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:24.879+0000 7f0dd8b22800 1 -- 172.21.15.28:0/537514148 wait complete. 2024-09-19T11:23:24.907 INFO:teuthology.orchestra.run.smithi028.stdout:["myfoo"] 2024-09-19T11:23:24.907 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-09-19T11:23:24.918 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:24.927 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:24.937 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:24.981 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-09-19T11:23:25.009 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:25.036 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:25.062 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:25.062 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-09-19T11:23:25.062 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-09-19T11:23:25.062 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-09-19T11:23:25.108 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:25.108 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd950c800 1 Processor -- start 2024-09-19T11:23:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd950c800 1 -- start start 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd950c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x56357fd8ed30 0x56357fd8f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd950c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56357fd90230 con 0x56357fd8f670 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd950c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x56357fc1b7d0 con 0x56357fd8ed30 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd6ff6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x56357fd8ed30 0x56357fd8f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd67f5640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x56357fd8f670 0x56357fd8fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:33226/0 (socket says 172.21.15.28:33226) 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd6ff6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x56357fd8ed30 0x56357fd8f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:60234/0 (socket says 172.21.15.28:60234) 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd67f5640 1 -- 172.21.15.28:0/3850552728 learned_addr learned my addr 172.21.15.28:0/3850552728 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/3850552728 >> v1:172.21.15.22:6789/0 conn(0x56357fd8f670 legacy=0x56357fd8fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.108+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/3850552728 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357fd90400 con 0x56357fd8ed30 2024-09-19T11:23:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.110+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/3850552728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fd8f130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005fd0 tx=0x7f3cc8045c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8277efc4965f2dac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.110+0000 7f3cd5ff4640 1 -- 172.21.15.28:0/3850552728 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cc8058530 con 0x56357fd8ed30 2024-09-19T11:23:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.110+0000 7f3cd5ff4640 1 -- 172.21.15.28:0/3850552728 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc80586d0 con 0x56357fd8ed30 2024-09-19T11:23:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.110+0000 7f3cd5ff4640 1 -- 172.21.15.28:0/3850552728 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cc80589d0 con 0x56357fd8ed30 2024-09-19T11:23:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.111+0000 7f3cd950c800 1 -- 172.21.15.28:0/3850552728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 msgr2=0x56357fd8f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.111+0000 7f3cd950c800 1 --2- 172.21.15.28:0/3850552728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fd8f130 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005fd0 tx=0x7f3cc8045c30 comp rx=0 tx=0).stop 2024-09-19T11:23:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.111+0000 7f3cd950c800 1 -- 172.21.15.28:0/3850552728 shutdown_connections 2024-09-19T11:23:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.111+0000 7f3cd950c800 1 --2- 172.21.15.28:0/3850552728 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fd8f130 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.111+0000 7f3cd950c800 1 -- 172.21.15.28:0/3850552728 >> 172.21.15.28:0/3850552728 conn(0x56357fd8aa60 msgr2=0x56357fd8cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.111+0000 7f3cd950c800 1 -- 172.21.15.28:0/3850552728 shutdown_connections 2024-09-19T11:23:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.111+0000 7f3cd950c800 1 -- 172.21.15.28:0/3850552728 wait complete. 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.126+0000 7f3cd950c800 1 Processor -- start 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.126+0000 7f3cd950c800 1 -- start start 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.126+0000 7f3cd950c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe82570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.126+0000 7f3cd950c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56357fc28dd0 con 0x56357fd8ed30 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3cd6ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe82570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3cd6ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe82570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33830/0 (socket says 172.21.15.28:33830) 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/3392420696 learned_addr learned my addr 172.21.15.28:0/3392420696 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/3392420696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357fe82b60 con 0x56357fd8ed30 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/3392420696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe82570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800a3f0 tx=0x7f3cc8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b24dd51d5e99f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3bc4e09640 1 -- 172.21.15.28:0/3392420696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cc80589b0 con 0x56357fd8ed30 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3bc4e09640 1 -- 172.21.15.28:0/3392420696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc8068070 con 0x56357fd8ed30 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.127+0000 7f3bc4e09640 1 -- 172.21.15.28:0/3392420696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cc8058c80 con 0x56357fd8ed30 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 -- 172.21.15.28:0/3392420696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 msgr2=0x56357fe82570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 --2- 172.21.15.28:0/3392420696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe82570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800a3f0 tx=0x7f3cc8051000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 -- 172.21.15.28:0/3392420696 shutdown_connections 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 --2- 172.21.15.28:0/3392420696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe82570 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 -- 172.21.15.28:0/3392420696 >> 172.21.15.28:0/3392420696 conn(0x56357fd8aa60 msgr2=0x56357fe7def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 -- 172.21.15.28:0/3392420696 shutdown_connections 2024-09-19T11:23:25.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 -- 172.21.15.28:0/3392420696 wait complete. 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 Processor -- start 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 -- start start 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.128+0000 7f3cd950c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe89fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd950c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56357fd90990 con 0x56357fd8ed30 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd6ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe89fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd6ff6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe89fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33838/0 (socket says 172.21.15.28:33838) 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/1328041866 learned_addr learned my addr 172.21.15.28:0/1328041866 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357fe8a520 con 0x56357fd8ed30 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe89fc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8065040 tx=0x7f3cc8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3bc3606640 1 -- 172.21.15.28:0/1328041866 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cc8063650 con 0x56357fd8ed30 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3bc3606640 1 -- 172.21.15.28:0/1328041866 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc80637f0 con 0x56357fd8ed30 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56357fe8a7b0 con 0x56357fd8ed30 2024-09-19T11:23:25.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3bc3606640 1 -- 172.21.15.28:0/1328041866 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cc8063af0 con 0x56357fd8ed30 2024-09-19T11:23:25.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.129+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357fe8b1b0 con 0x56357fd8ed30 2024-09-19T11:23:25.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.130+0000 7f3bc3606640 1 -- 172.21.15.28:0/1328041866 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3cc8074020 con 0x56357fd8ed30 2024-09-19T11:23:25.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.130+0000 7f3bc3606640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bbc03cfb0 0x7f3bbc03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.130+0000 7f3bc3606640 1 -- 172.21.15.28:0/1328041866 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f3cc8098490 con 0x56357fd8ed30 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd67f5640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bbc03cfb0 0x7f3bbc03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fd8f670 0x56357fe91610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fe8bda0 con 0x56357fd8f670 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fd8f670 0x56357fe91610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd67f5640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bbc03cfb0 0x7f3bbc03f470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001d70 tx=0x7f3cc00012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fd8f670 0x56357fe91610 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd77f7640 1 -- 172.21.15.28:0/1328041866 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3cd00083e0 con 0x56357fd8f670 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.131+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fe92090 con 0x56357fd8f670 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.132+0000 7f3cd77f7640 1 -- 172.21.15.28:0/1328041866 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3cd00083e0 con 0x56357fd8f670 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.132+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fe939a0 0x56357fe95e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.132+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fe965d0 con 0x56357fe939a0 2024-09-19T11:23:25.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.132+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fe939a0 0x56357fe95e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.132+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fe939a0 0x56357fe95e40 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.132+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/1328041866 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3cc806c910 con 0x56357fe939a0 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.133+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fe928e0 con 0x56357fd8f670 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.133+0000 7f3cd77f7640 1 -- 172.21.15.28:0/1328041866 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3cd00083e0 con 0x56357fd8f670 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.133+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fe92d00 con 0x56357fe939a0 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.133+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/1328041866 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3cc806c910 con 0x56357fe939a0 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.133+0000 7f3cd950c800 1 Processor -- start 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.134+0000 7f3cd950c800 1 -- start start 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.134+0000 7f3cd950c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fea4f60 0x56357fea5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.134+0000 7f3cd950c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56357fc874a0 con 0x56357fea4f60 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.134+0000 7f3cd77f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fea4f60 0x56357fea5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.134+0000 7f3cd77f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fea4f60 0x56357fea5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33854/0 (socket says 172.21.15.28:33854) 2024-09-19T11:23:25.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.134+0000 7f3cd77f7640 1 -- 172.21.15.28:0/1151507582 learned_addr learned my addr 172.21.15.28:0/1151507582 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.134+0000 7f3cd77f7640 1 -- 172.21.15.28:0/1151507582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357fea59c0 con 0x56357fea4f60 2024-09-19T11:23:25.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.135+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fea4f60 0x56357fea5380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56357fe8afc0 tx=0x7f3cd000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.135+0000 7f3bc0e01640 1 -- 172.21.15.28:0/1151507582 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd001c020 con 0x56357fea4f60 2024-09-19T11:23:25.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.135+0000 7f3bc0e01640 1 -- 172.21.15.28:0/1151507582 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd0007c90 con 0x56357fea4f60 2024-09-19T11:23:25.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.135+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56357fea61f0 con 0x56357fea4f60 2024-09-19T11:23:25.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.135+0000 7f3bc0e01640 1 -- 172.21.15.28:0/1151507582 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd0018440 con 0x56357fea4f60 2024-09-19T11:23:25.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.135+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357fea6790 con 0x56357fea4f60 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3bc0e01640 1 -- 172.21.15.28:0/1151507582 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3cd001f020 con 0x56357fea4f60 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3bc0e01640 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bb003d000 0x7f3bb003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3bc0e01640 1 -- 172.21.15.28:0/1151507582 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f3cd0055a00 con 0x56357fea4f60 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bb003d000 0x7f3bb003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3cd950c800 1 Processor -- start 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bb003d000 0x7f3bb003f4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3cc804e510 tx=0x7f3cc8046070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3cd950c800 1 -- start start 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3cd950c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.136+0000 7f3cd950c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56357fe82ed0 con 0x56357feb1a00 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd77f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd77f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33858/0 (socket says 172.21.15.28:33858) 2024-09-19T11:23:25.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd77f7640 1 -- 172.21.15.28:0/4189913303 learned_addr learned my addr 172.21.15.28:0/4189913303 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd77f7640 1 -- 172.21.15.28:0/4189913303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357feb23f0 con 0x56357feb1a00 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/4189913303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb1e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00089f0 tx=0x7f3cd0015630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f39928bdca4a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3bba7fc640 1 -- 172.21.15.28:0/4189913303 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd004a070 con 0x56357feb1a00 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3bba7fc640 1 -- 172.21.15.28:0/4189913303 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd00174d0 con 0x56357feb1a00 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3bba7fc640 1 -- 172.21.15.28:0/4189913303 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd0052070 con 0x56357feb1a00 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd950c800 1 -- 172.21.15.28:0/4189913303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 msgr2=0x56357feb1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd950c800 1 --2- 172.21.15.28:0/4189913303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb1e00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00089f0 tx=0x7f3cd0015630 comp rx=0 tx=0).stop 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd950c800 1 -- 172.21.15.28:0/4189913303 shutdown_connections 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.137+0000 7f3cd950c800 1 --2- 172.21.15.28:0/4189913303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb1e00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd950c800 1 -- 172.21.15.28:0/4189913303 >> 172.21.15.28:0/4189913303 conn(0x56357feade20 msgr2=0x56357feac060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd950c800 1 -- 172.21.15.28:0/4189913303 shutdown_connections 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd950c800 1 -- 172.21.15.28:0/4189913303 wait complete. 2024-09-19T11:23:25.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd950c800 1 Processor -- start 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd950c800 1 -- start start 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd950c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd950c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56357feb2a10 con 0x56357feb1a00 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd77f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd77f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33868/0 (socket says 172.21.15.28:33868) 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 learned_addr learned my addr 172.21.15.28:0/33833427 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.138+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357feb95c0 con 0x56357feb1a00 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.139+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb9060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0016010 tx=0x7f3cd0053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.139+0000 7f3bb8ff9640 1 -- 172.21.15.28:0/33833427 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd004a040 con 0x56357feb1a00 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.139+0000 7f3bb8ff9640 1 -- 172.21.15.28:0/33833427 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cd0013320 con 0x56357feb1a00 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.139+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56357feb9850 con 0x56357feb1a00 2024-09-19T11:23:25.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.139+0000 7f3bb8ff9640 1 -- 172.21.15.28:0/33833427 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cd0013640 con 0x56357feb1a00 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.139+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357feba3e0 con 0x56357feb1a00 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.139+0000 7f3bb8ff9640 1 -- 172.21.15.28:0/33833427 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3cd0016040 con 0x56357feb1a00 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3bb8ff9640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ba003d000 0x7f3ba003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3bb8ff9640 1 -- 172.21.15.28:0/33833427 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f3cd00459b0 con 0x56357feb1a00 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ba003d000 0x7f3ba003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fec48f0 0x56357fec6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fec7500 con 0x56357fec48f0 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3cd67f5640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fec48f0 0x56357fec6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.140+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ba003d000 0x7f3ba003f4c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8059fb0 tx=0x7f3cc808a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd67f5640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fec48f0 0x56357fec6db0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3cc0009050 con 0x56357fec48f0 2024-09-19T11:23:25.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 20 realm 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fec3c40 con 0x56357fec48f0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3cc0009050 con 0x56357fec48f0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fec7fe0 con 0x56357fec48f0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3cc0009050 con 0x56357fec48f0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.141+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fec88b0 con 0x56357fec48f0 2024-09-19T11:23:25.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3cc0009050 con 0x56357fec48f0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fec91f0 0x56357fecb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fecbd20 con 0x56357fec91f0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fec91f0 0x56357fecb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fec91f0 0x56357fecb5e0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd950c800 20 searching for the correct realm 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.142+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd77f7640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3cd0021020 0x7f3cd00419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cd0021020 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3cd0021020 0x7f3cd00419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3cd0021020 0x7f3cd00419e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd6ff6640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3cc80a6040 0x7f3cc80997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.143+0000 7f3cd67f5640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3cc80a6040 0x7f3cc80997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.144+0000 7f3cd67f5640 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3cc80a6040 0x7f3cc80997c0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.144+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0009050 con 0x7f3cc80a6040 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.144+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.144+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0009050 con 0x56357fec48f0 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.144+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cd0021020 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.145+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.146+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cd0021020 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.147+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.148+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cd0021020 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.149+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cd0021020 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.150+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec91f0 2024-09-19T11:23:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cc80a6040 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x56357fec48f0 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd6d0 con 0x7f3cd0021020 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd950c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd950c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd950c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd950c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd950c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.151+0000 7f3cd950c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fecc7a0 con 0x56357fec48f0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fecc3e0 con 0x56357fec48f0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fecd100 con 0x56357fec91f0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 zone default found 2024-09-19T11:23:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 4 Realm: () 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 zonegroup default 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.152+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357feccc40 con 0x7f3cc80a6040 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fecff40 con 0x56357fec91f0 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fecff40 con 0x56357fec91f0 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 20 started sync module instance, tier type = 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.153+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fecf3d0 con 0x7f3cd0021020 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.154+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.154+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94787779491776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed03d0 con 0x7f3cd0021020 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.154+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed0c20 con 0x56357fec48f0 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.155+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94787779491776] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.156+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.156+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94787779495792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed1040 con 0x56357fec48f0 2024-09-19T11:23:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.156+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed18d0 con 0x56357fec91f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.157+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.157+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94787779498048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed14e0 con 0x56357fec91f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.157+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed28a0 con 0x7f3cd0021020 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.157+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94787779495792] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.158+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94787779498048] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.158+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.158+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94787779501248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed0b90 con 0x7f3cd0021020 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.158+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed3180 con 0x56357fec48f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.159+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94787779501248] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.159+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.159+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94787779502992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed3140 con 0x56357fec48f0 2024-09-19T11:23:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.159+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed3bd0 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.160+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94787779502992] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.160+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.160+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94787779507936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed3140 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.160+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed45e0 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.161+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94787779507936] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.161+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.161+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94787779505776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed45e0 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.161+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed4fe0 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.162+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94787779505776] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94787779510576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed5180 con 0x7f3cc80a6040 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 20 add_watcher() i=0 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 20 add_watcher() i=1 2024-09-19T11:23:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 20 add_watcher() i=2 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 20 add_watcher() i=3 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 20 add_watcher() i=4 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 20 add_watcher() i=5 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.163+0000 7f3cd950c800 20 add_watcher() i=6 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.164+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94787779510576] v31'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.164+0000 7f3cd950c800 20 add_watcher() i=7 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.164+0000 7f3cd950c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.164+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357fed8380 con 0x56357fec48f0 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.164+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.164+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94787779525968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357fed5120 con 0x56357fec48f0 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.165+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94787779525968] v31'96 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.165+0000 7f3b8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.165+0000 7f3cd950c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.165+0000 7f3cd950c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.165+0000 7f3cd950c800 5 note: GC not initialized 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.165+0000 7f3b07fff640 20 reqs_thread_entry: start 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd950c800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd950c800 20 Filter name: none 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3b05ffb640 20 reqs_thread_entry: start 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff0b440 con 0x7f3cd0021020 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.166+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff0bcd0 con 0x7f3cc80a6040 2024-09-19T11:23:25.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:25.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd950c800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:25.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff0a750 con 0x7f3cd0021020 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd950c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.167+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff0d310 con 0x7f3cc80a6040 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(65 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 4.33 4:cfe5f5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff0cdd0 con 0x7f3cc80a6040 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 4.8 4:13d14a70:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff11c30 con 0x56357fec91f0 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 4.2b 4:d7493e99:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff144a0 con 0x56357fec48f0 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 4.27 4:e79b5c58:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff16d10 con 0x56357fec48f0 2024-09-19T11:23:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 4.20 4:06d6ec91:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff19770 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 4.f 4:f2b3320c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff1c0f0 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:72 4.d 4:b3b80c54:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff1eaf0 con 0x56357fec91f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(66 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 4.32 4:4d09bac8:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff0c840 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 4.3c 4:3c98ea8c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff21ac0 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 4.c 4:31f23ca3:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff24930 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 4.31 4:8fabc4b1:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x56357ff27240 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(68 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(69 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.168+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(70 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.169+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(71 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.169+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.169+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(72 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.169+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(73 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.169+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(74 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.169+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(76 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.169+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(75 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.170+0000 7f3b8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.170+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94787779525968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x56357fec48f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.171+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94787779525968] v31'97 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.171+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94787779491776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x7f3cd0021020 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.172+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94787779491776] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.172+0000 7f3cd950c800 20 remove_watcher() i=0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.172+0000 7f3cd950c800 2 removed watcher, disabling cache 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.172+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94787779495792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x56357fec48f0 2024-09-19T11:23:25.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.173+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94787779495792] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.173+0000 7f3cd950c800 20 remove_watcher() i=1 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.173+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94787779498048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x56357fec91f0 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.174+0000 7f3cd77f7640 1 -- 172.21.15.28:0/33833427 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94787779498048] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cd004d040 con 0x56357fec91f0 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.174+0000 7f3cd950c800 20 remove_watcher() i=2 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.174+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94787779501248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x7f3cd0021020 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.175+0000 7f3cd6ff6640 1 -- 172.21.15.28:0/33833427 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94787779501248] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc806c910 con 0x7f3cd0021020 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.175+0000 7f3cd950c800 20 remove_watcher() i=3 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.175+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94787779502992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x56357fec48f0 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.176+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94787779502992] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x56357fec48f0 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.176+0000 7f3cd950c800 20 remove_watcher() i=4 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.176+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94787779507936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x7f3cc80a6040 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.177+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94787779507936] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.177+0000 7f3cd950c800 20 remove_watcher() i=5 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.177+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94787779505776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x7f3cc80a6040 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.178+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94787779505776] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.178+0000 7f3cd950c800 20 remove_watcher() i=6 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.178+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94787779510576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x56357ff08e00 con 0x7f3cc80a6040 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94787779510576] v31'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3cc0029040 con 0x7f3cc80a6040 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 20 remove_watcher() i=7 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3cc80a6040 msgr2=0x7f3cc80997c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3cc80a6040 0x7f3cc80997c0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fec48f0 msgr2=0x56357fec6db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fec48f0 0x56357fec6db0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3cd0021020 msgr2=0x7f3cd00419e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3cd0021020 0x7f3cd00419e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fec91f0 msgr2=0x56357fecb5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fec91f0 0x56357fecb5e0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ba003d000 msgr2=0x7f3ba003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ba003d000 0x7f3ba003f4c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8059fb0 tx=0x7f3cc808a000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 msgr2=0x56357feb9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 --2- 172.21.15.28:0/33833427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357feb1a00 0x56357feb9060 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0016010 tx=0x7f3cd0053ce0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 reap_dead start 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd67f5640 1 -- 172.21.15.28:0/33833427 reap_dead start 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 shutdown_connections 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 >> 172.21.15.28:0/33833427 conn(0x56357feade20 msgr2=0x56357feac030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 shutdown_connections 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.179+0000 7f3cd950c800 1 -- 172.21.15.28:0/33833427 wait complete. 2024-09-19T11:23:25.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bb003d000 msgr2=0x7f3bb003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bb003d000 0x7f3bb003f4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3cc804e510 tx=0x7f3cc8046070 comp rx=0 tx=0).stop 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fea4f60 msgr2=0x56357fea5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fea4f60 0x56357fea5380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x56357fe8afc0 tx=0x7f3cd000bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 shutdown_connections 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bb003d000 0x7f3bb003f4c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1151507582 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fea4f60 0x56357fea5380 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 >> 172.21.15.28:0/1151507582 conn(0x56357fe99310 msgr2=0x56357fe996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 shutdown_connections 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1151507582 wait complete. 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fd8f670 msgr2=0x56357fe91610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fd8f670 0x56357fe91610 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fe939a0 msgr2=0x56357fe95e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fe939a0 0x56357fe95e40 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bbc03cfb0 msgr2=0x7f3bbc03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bbc03cfb0 0x7f3bbc03f470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001d70 tx=0x7f3cc00012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 msgr2=0x56357fe89fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe89fc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8065040 tx=0x7f3cc8003330 comp rx=0 tx=0).stop 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 shutdown_connections 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56357fe939a0 0x56357fe95e40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3bbc03cfb0 0x7f3bbc03f470 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56357fd8f670 0x56357fe91610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 --2- 172.21.15.28:0/1328041866 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56357fd8ed30 0x56357fe89fc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 >> 172.21.15.28:0/1328041866 conn(0x56357fd8aa60 msgr2=0x56357fe83960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 shutdown_connections 2024-09-19T11:23:25.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.180+0000 7f3cd950c800 1 -- 172.21.15.28:0/1328041866 wait complete. 2024-09-19T11:23:25.195 INFO:teuthology.orchestra.run.smithi028.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"d3056e04-d444-48b9-8e61-96c58b445cdd","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","marker":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","index_type":"Normal","index_generation":0,"num_shards":11,"reshard_status":"None","judge_reshard_lock_time":"0.000000","object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-09-19T11:23:24.617941Z","creation_time":"2024-09-19T11:23:24.614937Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"read_tracker":1} 2024-09-19T11:23:25.195 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:23:24.617941Z', 'creation_time': '2024-09-19T11:23:24.614937Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 1} 2024-09-19T11:23:25.195 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-09-19T11:23:25.195 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-09-19T11:23:25.195 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-09-19T11:23:25.241 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:25.241 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:25.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.240+0000 7fba7f72a800 1 Processor -- start 2024-09-19T11:23:25.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.240+0000 7fba7f72a800 1 -- start start 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7f72a800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e14cfebd60 0x55e14cfec160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7f72a800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e14cfed260 con 0x55e14cfec6a0 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7f72a800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55e14ce787d0 con 0x55e14cfebd60 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7b4ad640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e14cfebd60 0x55e14cfec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7b4ad640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e14cfebd60 0x55e14cfec160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33874/0 (socket says 172.21.15.28:33874) 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7acac640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55e14cfec6a0 0x55e14cfecb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48270/0 (socket says 172.21.15.28:48270) 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7b4ad640 1 -- 172.21.15.28:0/2115481086 learned_addr learned my addr 172.21.15.28:0/2115481086 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7b4ad640 1 -- 172.21.15.28:0/2115481086 >> v1:172.21.15.22:6789/0 conn(0x55e14cfec6a0 legacy=0x55e14cfecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.241+0000 7fba7b4ad640 1 -- 172.21.15.28:0/2115481086 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14cfed430 con 0x55e14cfebd60 2024-09-19T11:23:25.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.243+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/2115481086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14cfec160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00a520 tx=0x7fba6c045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1bd61e3e9e9356 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.243+0000 7fba7a4ab640 1 -- 172.21.15.28:0/2115481086 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c055070 con 0x55e14cfebd60 2024-09-19T11:23:25.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.243+0000 7fba7a4ab640 1 -- 172.21.15.28:0/2115481086 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c0515b0 con 0x55e14cfebd60 2024-09-19T11:23:25.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.243+0000 7fba7a4ab640 1 -- 172.21.15.28:0/2115481086 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c0518d0 con 0x55e14cfebd60 2024-09-19T11:23:25.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.244+0000 7fba7f72a800 1 -- 172.21.15.28:0/2115481086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 msgr2=0x55e14cfec160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.244+0000 7fba7f72a800 1 --2- 172.21.15.28:0/2115481086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14cfec160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00a520 tx=0x7fba6c045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.244+0000 7fba7f72a800 1 -- 172.21.15.28:0/2115481086 shutdown_connections 2024-09-19T11:23:25.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.244+0000 7fba7f72a800 1 --2- 172.21.15.28:0/2115481086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14cfec160 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.244+0000 7fba7f72a800 1 -- 172.21.15.28:0/2115481086 >> 172.21.15.28:0/2115481086 conn(0x55e14cfe7a50 msgr2=0x55e14cfe9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.244+0000 7fba7f72a800 1 -- 172.21.15.28:0/2115481086 shutdown_connections 2024-09-19T11:23:25.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.244+0000 7fba7f72a800 1 -- 172.21.15.28:0/2115481086 wait complete. 2024-09-19T11:23:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.260+0000 7fba7f72a800 1 Processor -- start 2024-09-19T11:23:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.260+0000 7fba7f72a800 1 -- start start 2024-09-19T11:23:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.261+0000 7fba7f72a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0df530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.261+0000 7fba7f72a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e14ce85dd0 con 0x55e14cfebd60 2024-09-19T11:23:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.261+0000 7fba7b4ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0df530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.261+0000 7fba7b4ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0df530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33890/0 (socket says 172.21.15.28:33890) 2024-09-19T11:23:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.261+0000 7fba7b4ad640 1 -- 172.21.15.28:0/2304933192 learned_addr learned my addr 172.21.15.28:0/2304933192 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.261+0000 7fba7b4ad640 1 -- 172.21.15.28:0/2304933192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14d0dfb20 con 0x55e14cfebd60 2024-09-19T11:23:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.261+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/2304933192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0df530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fba6c025da0 tx=0x7fba6c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc707756859690fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fb96960a640 1 -- 172.21.15.28:0/2304933192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c051950 con 0x55e14cfebd60 2024-09-19T11:23:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fb96960a640 1 -- 172.21.15.28:0/2304933192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c051af0 con 0x55e14cfebd60 2024-09-19T11:23:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fb96960a640 1 -- 172.21.15.28:0/2304933192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c051dd0 con 0x55e14cfebd60 2024-09-19T11:23:25.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fba7f72a800 1 -- 172.21.15.28:0/2304933192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 msgr2=0x55e14d0df530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fba7f72a800 1 --2- 172.21.15.28:0/2304933192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0df530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fba6c025da0 tx=0x7fba6c051000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fba7f72a800 1 -- 172.21.15.28:0/2304933192 shutdown_connections 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fba7f72a800 1 --2- 172.21.15.28:0/2304933192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0df530 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fba7f72a800 1 -- 172.21.15.28:0/2304933192 >> 172.21.15.28:0/2304933192 conn(0x55e14cfe7a50 msgr2=0x55e14d0daeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.263+0000 7fba7f72a800 1 -- 172.21.15.28:0/2304933192 shutdown_connections 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7f72a800 1 -- 172.21.15.28:0/2304933192 wait complete. 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7f72a800 1 Processor -- start 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7f72a800 1 -- start start 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7f72a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7f72a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e14cfed9c0 con 0x55e14cfebd60 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7b4ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7b4ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33906/0 (socket says 172.21.15.28:33906) 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.264+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3962336150 learned_addr learned my addr 172.21.15.28:0/3962336150 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.265+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14d0e7460 con 0x55e14cfebd60 2024-09-19T11:23:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.265+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0e6f00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fba6c067040 tx=0x7fba6c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.265+0000 7fb967e07640 1 -- 172.21.15.28:0/3962336150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c073070 con 0x55e14cfebd60 2024-09-19T11:23:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.265+0000 7fb967e07640 1 -- 172.21.15.28:0/3962336150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c05a450 con 0x55e14cfebd60 2024-09-19T11:23:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.265+0000 7fb967e07640 1 -- 172.21.15.28:0/3962336150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba6c05a730 con 0x55e14cfebd60 2024-09-19T11:23:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.265+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e14d0e76f0 con 0x55e14cfebd60 2024-09-19T11:23:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.265+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14d0e8300 con 0x55e14cfebd60 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.266+0000 7fb967e07640 1 -- 172.21.15.28:0/3962336150 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fba6c076070 con 0x55e14cfebd60 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.266+0000 7fb967e07640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb96003d000 0x7fb96003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.266+0000 7fb967e07640 1 -- 172.21.15.28:0/3962336150 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fba6c062070 con 0x55e14cfebd60 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.266+0000 7fba7acac640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb96003d000 0x7fb96003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.267+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14cfec6a0 0x55e14d0ee550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.267+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d0eea90 con 0x55e14cfec6a0 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.267+0000 7fba7bcae640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14cfec6a0 0x55e14d0ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.267+0000 7fba7acac640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb96003d000 0x7fb96003f4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba64001e30 tx=0x7fba64001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.267+0000 7fba7bcae640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14cfec6a0 0x55e14d0ee550 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.267+0000 7fba7bcae640 1 -- 172.21.15.28:0/3962336150 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba7400a4d0 con 0x55e14cfec6a0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.267+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d0eee30 con 0x55e14cfec6a0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.268+0000 7fba7bcae640 1 -- 172.21.15.28:0/3962336150 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fba7400a4d0 con 0x55e14cfec6a0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.268+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d0f08f0 0x55e14d0f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.268+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d0f3520 con 0x55e14d0f08f0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.268+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d0f08f0 0x55e14d0f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.268+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d0f08f0 0x55e14d0f2d90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.268+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3962336150 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fba6c075030 con 0x55e14d0f08f0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.268+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d0ef680 con 0x55e14cfec6a0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.269+0000 7fba7bcae640 1 -- 172.21.15.28:0/3962336150 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fba7400a4d0 con 0x55e14cfec6a0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.269+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d0efa20 con 0x55e14d0f08f0 2024-09-19T11:23:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.269+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3962336150 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fba6c075030 con 0x55e14d0f08f0 2024-09-19T11:23:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.269+0000 7fba7f72a800 1 Processor -- start 2024-09-19T11:23:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.269+0000 7fba7f72a800 1 -- start start 2024-09-19T11:23:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.269+0000 7fba7f72a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d101e80 0x55e14d1022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.269+0000 7fba7f72a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e14cee44a0 con 0x55e14d101e80 2024-09-19T11:23:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fba7bcae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d101e80 0x55e14d1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fba7bcae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d101e80 0x55e14d1022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33920/0 (socket says 172.21.15.28:33920) 2024-09-19T11:23:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fba7bcae640 1 -- 172.21.15.28:0/4244587870 learned_addr learned my addr 172.21.15.28:0/4244587870 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fba7bcae640 1 -- 172.21.15.28:0/4244587870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14d1028e0 con 0x55e14d101e80 2024-09-19T11:23:25.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fba7bcae640 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d101e80 0x55e14d1022a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55e14d0e7d90 tx=0x7fba740167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fb965602640 1 -- 172.21.15.28:0/4244587870 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba74013640 con 0x55e14d101e80 2024-09-19T11:23:25.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fb965602640 1 -- 172.21.15.28:0/4244587870 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba740137e0 con 0x55e14d101e80 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e14d103110 con 0x55e14d101e80 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.270+0000 7fb965602640 1 -- 172.21.15.28:0/4244587870 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba74013ae0 con 0x55e14d101e80 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.271+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14d1036b0 con 0x55e14d101e80 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.271+0000 7fb965602640 1 -- 172.21.15.28:0/4244587870 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fba74011040 con 0x55e14d101e80 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.271+0000 7fb965602640 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb95403d000 0x7fb95403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fb965602640 1 -- 172.21.15.28:0/4244587870 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fba7400e040 con 0x55e14d101e80 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb95403d000 0x7fb95403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7f72a800 1 Processor -- start 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7f72a800 1 -- start start 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb95403d000 0x7fb95403f4c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55e14d0e9000 tx=0x7fba6c096fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7f72a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d10eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7f72a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e14d0dfe90 con 0x55e14d10e9a0 2024-09-19T11:23:25.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7bcae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d10eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7bcae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d10eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33936/0 (socket says 172.21.15.28:33936) 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7bcae640 1 -- 172.21.15.28:0/3929173191 learned_addr learned my addr 172.21.15.28:0/3929173191 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.272+0000 7fba7bcae640 1 -- 172.21.15.28:0/3929173191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14d10f390 con 0x55e14d10e9a0 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7bcae640 1 --2- 172.21.15.28:0/3929173191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d10eda0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba7402afd0 tx=0x7fba74021e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f779cbd2de0722 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fb95effd640 1 -- 172.21.15.28:0/3929173191 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba74051070 con 0x55e14d10e9a0 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fb95effd640 1 -- 172.21.15.28:0/3929173191 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7401f470 con 0x55e14d10e9a0 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fb95effd640 1 -- 172.21.15.28:0/3929173191 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba740533f0 con 0x55e14d10e9a0 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7f72a800 1 -- 172.21.15.28:0/3929173191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 msgr2=0x55e14d10eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3929173191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d10eda0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fba7402afd0 tx=0x7fba74021e60 comp rx=0 tx=0).stop 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7f72a800 1 -- 172.21.15.28:0/3929173191 shutdown_connections 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3929173191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d10eda0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7f72a800 1 -- 172.21.15.28:0/3929173191 >> 172.21.15.28:0/3929173191 conn(0x55e14d10ada0 msgr2=0x55e14d108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7f72a800 1 -- 172.21.15.28:0/3929173191 shutdown_connections 2024-09-19T11:23:25.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.273+0000 7fba7f72a800 1 -- 172.21.15.28:0/3929173191 wait complete. 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7f72a800 1 Processor -- start 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7f72a800 1 -- start start 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7f72a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7f72a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e14d10f9b0 con 0x55e14d10e9a0 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7bcae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7bcae640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d115ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33946/0 (socket says 172.21.15.28:33946) 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 learned_addr learned my addr 172.21.15.28:0/3557101738 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14d116430 con 0x55e14d10e9a0 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fba7bcae640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d115ed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba7402afd0 tx=0x7fba7401fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.274+0000 7fb95d7fa640 1 -- 172.21.15.28:0/3557101738 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba7404b040 con 0x55e14d10e9a0 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.275+0000 7fb95d7fa640 1 -- 172.21.15.28:0/3557101738 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba740459e0 con 0x55e14d10e9a0 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.275+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e14d1166c0 con 0x55e14d10e9a0 2024-09-19T11:23:25.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.275+0000 7fb95d7fa640 1 -- 172.21.15.28:0/3557101738 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba74045ce0 con 0x55e14d10e9a0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.275+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14d117250 con 0x55e14d10e9a0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.275+0000 7fb95d7fa640 1 -- 172.21.15.28:0/3557101738 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fba74054950 con 0x55e14d10e9a0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.276+0000 7fb95d7fa640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb94403cfb0 0x7fb94403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.276+0000 7fb95d7fa640 1 -- 172.21.15.28:0/3557101738 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fba74034c40 con 0x55e14d10e9a0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.276+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb94403cfb0 0x7fb94403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.276+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.276+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14d1217a0 0x55e14d123c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.276+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1243b0 con 0x55e14d1217a0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.276+0000 7fba7acac640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14d1217a0 0x55e14d123c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb94403cfb0 0x7fb94403f470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba6c005570 tx=0x7fba6c093000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7acac640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14d1217a0 0x55e14d123c60 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba6400f610 con 0x55e14d1217a0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 20 realm 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d120af0 con 0x55e14d1217a0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba6400f610 con 0x55e14d1217a0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d124c80 con 0x55e14d1217a0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba6400f610 con 0x55e14d1217a0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.277+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d125550 con 0x55e14d1217a0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fba6400f610 con 0x55e14d1217a0 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d125e90 0x55e14d128280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1289c0 con 0x55e14d125e90 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7bcae640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d125e90 0x55e14d128280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7bcae640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d125e90 0x55e14d128280 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7f72a800 20 searching for the correct realm 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7bcae640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fba7403c660 0x7fba7403ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.278+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba7403c660 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.279+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fba7403c660 0x7fba7403ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.279+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fba7403c660 0x7fba7403ca80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.279+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.279+0000 7fba7b4ad640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fba6c0a6040 0x7fba6c0981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.279+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.279+0000 7fba7acac640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fba6c0a6040 0x7fba6c0981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.280+0000 7fba7acac640 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fba6c0a6040 0x7fba6c0981d0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.280+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6400f610 con 0x7fba6c0a6040 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.280+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.280+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6400f610 con 0x55e14d1217a0 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.280+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x55e14d1217a0 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba7403c660 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.281+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x55e14d1217a0 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x55e14d1217a0 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba7403c660 2024-09-19T11:23:25.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.282+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.283+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x55e14d1217a0 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fba6402b040 con 0x55e14d1217a0 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.284+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba7403c660 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6402b040 con 0x55e14d1217a0 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.285+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba7403c660 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d125e90 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.286+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba6c0a6040 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x55e14d1217a0 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12a370 con 0x7fba7403c660 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d129440 con 0x55e14d1217a0 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.287+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d129ce0 con 0x55e14d1217a0 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d129080 con 0x55e14d125e90 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 zone default found 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 4 Realm: () 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 zonegroup default 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d129910 con 0x7fba6c0a6040 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.288+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12c830 con 0x55e14d125e90 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7f72a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12cc20 con 0x55e14d125e90 2024-09-19T11:23:25.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7f72a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7f72a800 20 started sync module instance, tier type = 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7f72a800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.289+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12bbd0 con 0x7fba7403c660 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.290+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.290+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94426149076976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12d4e0 con 0x7fba7403c660 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.290+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12dd30 con 0x55e14d1217a0 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.291+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94426149076976] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.291+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.291+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94426149081136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12ddf0 con 0x55e14d1217a0 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.291+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12e910 con 0x55e14d125e90 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.292+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94426149081136] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.292+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.293+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94426149086176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12e8d0 con 0x55e14d125e90 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.293+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12f390 con 0x7fba7403c660 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.293+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94426149086176] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.294+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.294+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94426149088928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d12f780 con 0x7fba7403c660 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.294+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1301e0 con 0x55e14d1217a0 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.295+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.295+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94426149092544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1301e0 con 0x55e14d1217a0 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.295+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1311c0 con 0x7fba6c0a6040 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.295+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94426149088928] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.296+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94426149092544] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.296+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.296+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94426149096656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1303c0 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.296+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d131a60 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.297+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94426149096656] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.297+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.297+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94426149083264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1311c0 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.297+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d132080 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94426149083264] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94426149098400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d132080 con 0x7fba6c0a6040 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 20 add_watcher() i=0 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 20 add_watcher() i=1 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 20 add_watcher() i=2 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 20 add_watcher() i=3 2024-09-19T11:23:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 20 add_watcher() i=4 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 20 add_watcher() i=5 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.298+0000 7fba7f72a800 20 add_watcher() i=6 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.299+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94426149098400] v31'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.299+0000 7fba7f72a800 20 add_watcher() i=7 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.299+0000 7fba7f72a800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.299+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1353d0 con 0x55e14d1217a0 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.300+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.300+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94426149113312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1321f0 con 0x55e14d1217a0 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94426149113312] v31'98 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fb931fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7f72a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7f72a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7f72a800 5 note: GC not initialized 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fb930fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7f72a800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7f72a800 20 Filter name: none 2024-09-19T11:23:25.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fb8aa7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.301+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d167e80 con 0x55e14d1217a0 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.302+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.302+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.302+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.302+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d168740 con 0x55e14d1217a0 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.302+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.302+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.302+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d169750 con 0x7fba7403c660 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.303+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.303+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.303+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d167080 con 0x7fba7403c660 2024-09-19T11:23:25.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.303+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(65 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.303+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.303+0000 7fba7f72a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.303+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d169750 con 0x7fba6c0a6040 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(66 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 4.33 4:cfe5f5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1691c0 con 0x7fba6c0a6040 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 4.8 4:13d14a70:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d16e1b0 con 0x55e14d125e90 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 4.2b 4:d7493e99:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1691c0 con 0x55e14d1217a0 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 4.27 4:e79b5c58:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1691c0 con 0x55e14d1217a0 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 4.20 4:06d6ec91:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d175450 con 0x55e14d1217a0 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(67 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 4.f 4:f2b3320c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d175860 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 4.d 4:b3b80c54:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d1781d0 con 0x55e14d125e90 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 4.32 4:4d09bac8:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d17b070 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 4.3c 4:3c98ea8c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d17db00 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 4.c 4:31f23ca3:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d180510 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(68 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.304+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:77 4.31 4:8fabc4b1:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e14d182fa0 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(69 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(70 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(73 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(71 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(72 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(74 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(75 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(76 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.305+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(77 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.306+0000 7fb931fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.306+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94426149113312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x55e14d1217a0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.307+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94426149113312] v31'99 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.307+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94426149076976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x7fba7403c660 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.308+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94426149076976] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.308+0000 7fba7f72a800 20 remove_watcher() i=0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.308+0000 7fba7f72a800 2 removed watcher, disabling cache 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.308+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94426149081136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x55e14d1217a0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.309+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94426149081136] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.309+0000 7fba7f72a800 20 remove_watcher() i=1 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.309+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94426149086176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x55e14d125e90 2024-09-19T11:23:25.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.310+0000 7fba7bcae640 1 -- 172.21.15.28:0/3557101738 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94426149086176] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba74039020 con 0x55e14d125e90 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.310+0000 7fba7f72a800 20 remove_watcher() i=2 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.310+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94426149088928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x7fba7403c660 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.311+0000 7fba7b4ad640 1 -- 172.21.15.28:0/3557101738 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94426149088928] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6c075030 con 0x7fba7403c660 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.311+0000 7fba7f72a800 20 remove_watcher() i=3 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.311+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94426149092544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x55e14d1217a0 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.312+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94426149092544] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x55e14d1217a0 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.312+0000 7fba7f72a800 20 remove_watcher() i=4 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.312+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94426149096656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x7fba6c0a6040 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.313+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94426149096656] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.313+0000 7fba7f72a800 20 remove_watcher() i=5 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.313+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94426149083264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x7fba6c0a6040 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.314+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94426149083264] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.314+0000 7fba7f72a800 20 remove_watcher() i=6 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.314+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94426149098400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e14d165da0 con 0x7fba6c0a6040 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94426149098400] v31'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba6403b040 con 0x7fba6c0a6040 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 20 remove_watcher() i=7 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fba6c0a6040 msgr2=0x7fba6c0981d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fba6c0a6040 0x7fba6c0981d0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14d1217a0 msgr2=0x55e14d123c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14d1217a0 0x55e14d123c60 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fba7403c660 msgr2=0x7fba7403ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fba7403c660 0x7fba7403ca80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d125e90 msgr2=0x55e14d128280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d125e90 0x55e14d128280 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb94403cfb0 msgr2=0x7fb94403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb94403cfb0 0x7fb94403f470 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fba6c005570 tx=0x7fba6c093000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 msgr2=0x55e14d115ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3557101738 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d10e9a0 0x55e14d115ed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fba7402afd0 tx=0x7fba7401fb20 comp rx=0 tx=0).stop 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 reap_dead start 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7acac640 1 -- 172.21.15.28:0/3557101738 reap_dead start 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 shutdown_connections 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 >> 172.21.15.28:0/3557101738 conn(0x55e14d10ada0 msgr2=0x55e14d108fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 shutdown_connections 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.315+0000 7fba7f72a800 1 -- 172.21.15.28:0/3557101738 wait complete. 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb95403d000 msgr2=0x7fb95403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb95403d000 0x7fb95403f4c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55e14d0e9000 tx=0x7fba6c096fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d101e80 msgr2=0x55e14d1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d101e80 0x55e14d1022a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55e14d0e7d90 tx=0x7fba740167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 shutdown_connections 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb95403d000 0x7fb95403f4c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/4244587870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14d101e80 0x55e14d1022a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 >> 172.21.15.28:0/4244587870 conn(0x55e14d0f6230 msgr2=0x55e14d0f6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 shutdown_connections 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/4244587870 wait complete. 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14cfec6a0 msgr2=0x55e14d0ee550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14cfec6a0 0x55e14d0ee550 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d0f08f0 msgr2=0x55e14d0f2d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d0f08f0 0x55e14d0f2d90 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb96003d000 msgr2=0x7fb96003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb96003d000 0x7fb96003f4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fba64001e30 tx=0x7fba64001390 comp rx=0 tx=0).stop 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 msgr2=0x55e14d0e6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0e6f00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fba6c067040 tx=0x7fba6c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 shutdown_connections 2024-09-19T11:23:25.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e14d0f08f0 0x55e14d0f2d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb96003d000 0x7fb96003f4c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e14cfec6a0 0x55e14d0ee550 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 --2- 172.21.15.28:0/3962336150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e14cfebd60 0x55e14d0e6f00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 >> 172.21.15.28:0/3962336150 conn(0x55e14cfe7a50 msgr2=0x55e14d0e0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 shutdown_connections 2024-09-19T11:23:25.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.316+0000 7fba7f72a800 1 -- 172.21.15.28:0/3962336150 wait complete. 2024-09-19T11:23:25.329 INFO:teuthology.orchestra.run.smithi028.stdout:[{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"d3056e04-d444-48b9-8e61-96c58b445cdd","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","marker":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","index_type":"Normal","index_generation":0,"num_shards":11,"reshard_status":"None","judge_reshard_lock_time":"0.000000","object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-09-19T11:23:24.617941Z","creation_time":"2024-09-19T11:23:24.614937Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"read_tracker":1}] 2024-09-19T11:23:25.329 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:23:24.617941Z', 'creation_time': '2024-09-19T11:23:24.614937Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 1}] 2024-09-19T11:23:25.336 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-19T11:23:25.336 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-09-19T11:23:25.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-09-19T11:23:25.336 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-09-19T11:23:25.384 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:25.384 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.383+0000 7f8429773800 1 Processor -- start 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.383+0000 7f8429773800 1 -- start start 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.383+0000 7f8429773800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55928d6c4aa0 0x55928d6ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.383+0000 7f8429773800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55928d6c3d40 con 0x55928d6c3690 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.383+0000 7f8429773800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55928d54f7d0 con 0x55928d6c4aa0 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.384+0000 7f8426a5c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55928d6c4aa0 0x55928d6ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.384+0000 7f842725d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55928d6c3690 0x55928d6c4390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48276/0 (socket says 172.21.15.28:48276) 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.384+0000 7f8426a5c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55928d6c4aa0 0x55928d6ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33950/0 (socket says 172.21.15.28:33950) 2024-09-19T11:23:25.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.384+0000 7f842725d640 1 -- 172.21.15.28:0/930144076 learned_addr learned my addr 172.21.15.28:0/930144076 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:25.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.384+0000 7f8426a5c640 1 -- 172.21.15.28:0/930144076 >> v1:172.21.15.22:6789/0 conn(0x55928d6c3690 legacy=0x55928d6c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.384+0000 7f8426a5c640 1 -- 172.21.15.28:0/930144076 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d6c4f50 con 0x55928d6c4aa0 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8426a5c640 1 --2- 172.21.15.28:0/930144076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c4aa0 0x55928d6ca640 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f841c00a470 tx=0x7f841c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4498ce5405c1ed3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f842625b640 1 -- 172.21.15.28:0/930144076 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c053570 con 0x55928d6c4aa0 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f842625b640 1 -- 172.21.15.28:0/930144076 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841c058070 con 0x55928d6c4aa0 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f842625b640 1 -- 172.21.15.28:0/930144076 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841c053880 con 0x55928d6c4aa0 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8429773800 1 -- 172.21.15.28:0/930144076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c4aa0 msgr2=0x55928d6ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8429773800 1 --2- 172.21.15.28:0/930144076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c4aa0 0x55928d6ca640 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f841c00a470 tx=0x7f841c045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8429773800 1 -- 172.21.15.28:0/930144076 shutdown_connections 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8429773800 1 --2- 172.21.15.28:0/930144076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c4aa0 0x55928d6ca640 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8429773800 1 -- 172.21.15.28:0/930144076 >> 172.21.15.28:0/930144076 conn(0x55928d6bea60 msgr2=0x55928d6c0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8429773800 1 -- 172.21.15.28:0/930144076 shutdown_connections 2024-09-19T11:23:25.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.386+0000 7f8429773800 1 -- 172.21.15.28:0/930144076 wait complete. 2024-09-19T11:23:25.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.401+0000 7f8429773800 1 Processor -- start 2024-09-19T11:23:25.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.401+0000 7f8429773800 1 -- start start 2024-09-19T11:23:25.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.401+0000 7f8429773800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d7b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.401+0000 7f8429773800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55928d55cdd0 con 0x55928d6c3690 2024-09-19T11:23:25.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f842725d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d7b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f842725d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d7b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33966/0 (socket says 172.21.15.28:33966) 2024-09-19T11:23:25.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f842725d640 1 -- 172.21.15.28:0/1932094566 learned_addr learned my addr 172.21.15.28:0/1932094566 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f842725d640 1 -- 172.21.15.28:0/1932094566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d7b3810 con 0x55928d6c3690 2024-09-19T11:23:25.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f842725d640 1 --2- 172.21.15.28:0/1932094566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d7b3220 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8418001e40 tx=0x7f8418001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4b67c4afe93e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f831560a640 1 -- 172.21.15.28:0/1932094566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8418012020 con 0x55928d6c3690 2024-09-19T11:23:25.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f831560a640 1 -- 172.21.15.28:0/1932094566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841800e5a0 con 0x55928d6c3690 2024-09-19T11:23:25.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.402+0000 7f831560a640 1 -- 172.21.15.28:0/1932094566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841800e8a0 con 0x55928d6c3690 2024-09-19T11:23:25.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.403+0000 7f8429773800 1 -- 172.21.15.28:0/1932094566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 msgr2=0x55928d7b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.403+0000 7f8429773800 1 --2- 172.21.15.28:0/1932094566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d7b3220 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8418001e40 tx=0x7f8418001e70 comp rx=0 tx=0).stop 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.403+0000 7f8429773800 1 -- 172.21.15.28:0/1932094566 shutdown_connections 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.403+0000 7f8429773800 1 --2- 172.21.15.28:0/1932094566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d7b3220 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.403+0000 7f8429773800 1 -- 172.21.15.28:0/1932094566 >> 172.21.15.28:0/1932094566 conn(0x55928d6bea60 msgr2=0x55928d7aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.403+0000 7f8429773800 1 -- 172.21.15.28:0/1932094566 shutdown_connections 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.403+0000 7f8429773800 1 -- 172.21.15.28:0/1932094566 wait complete. 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f8429773800 1 Processor -- start 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f8429773800 1 -- start start 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f8429773800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d637890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f8429773800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55928d6cad90 con 0x55928d6c3690 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f842725d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d637890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f842725d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d637890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33970/0 (socket says 172.21.15.28:33970) 2024-09-19T11:23:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f842725d640 1 -- 172.21.15.28:0/1641677463 learned_addr learned my addr 172.21.15.28:0/1641677463 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f842725d640 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d637df0 con 0x55928d6c3690 2024-09-19T11:23:25.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.404+0000 7f842725d640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d637890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f841800c010 tx=0x7f8418000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.405+0000 7f8313e07640 1 -- 172.21.15.28:0/1641677463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8418012030 con 0x55928d6c3690 2024-09-19T11:23:25.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.405+0000 7f8313e07640 1 -- 172.21.15.28:0/1641677463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84180014a0 con 0x55928d6c3690 2024-09-19T11:23:25.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.405+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55928d7beac0 con 0x55928d6c3690 2024-09-19T11:23:25.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.405+0000 7f8313e07640 1 -- 172.21.15.28:0/1641677463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f841801a7a0 con 0x55928d6c3690 2024-09-19T11:23:25.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.405+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928d7bf080 con 0x55928d6c3690 2024-09-19T11:23:25.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8313e07640 1 -- 172.21.15.28:0/1641677463 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8418015020 con 0x55928d6c3690 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8313e07640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830c03cfb0 0x7f830c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8313e07640 1 -- 172.21.15.28:0/1641677463 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f8418020030 con 0x55928d6c3690 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8426a5c640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830c03cfb0 0x7f830c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d6c4aa0 0x55928d7c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7c36a0 con 0x55928d6c4aa0 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8427a5e640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d6c4aa0 0x55928d7c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.406+0000 7f8426a5c640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830c03cfb0 0x7f830c03f470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f841c057000 tx=0x7f841c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.407+0000 7f8427a5e640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d6c4aa0 0x55928d7c3030 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.407+0000 7f8427a5e640 1 -- 172.21.15.28:0/1641677463 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f842000a4d0 con 0x55928d6c4aa0 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.407+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7c04c0 con 0x55928d6c4aa0 2024-09-19T11:23:25.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.407+0000 7f8427a5e640 1 -- 172.21.15.28:0/1641677463 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f842000a4d0 con 0x55928d6c4aa0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.407+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7c59f0 0x55928d7c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.407+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7c84b0 con 0x55928d7c59f0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.407+0000 7f842725d640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7c59f0 0x55928d7c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.408+0000 7f842725d640 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7c59f0 0x55928d7c7de0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.408+0000 7f842725d640 1 -- 172.21.15.28:0/1641677463 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f841801ab20 con 0x55928d7c59f0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.408+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7c4890 con 0x55928d6c4aa0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.408+0000 7f8427a5e640 1 -- 172.21.15.28:0/1641677463 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f842000a4d0 con 0x55928d6c4aa0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.408+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7c4cf0 con 0x55928d7c59f0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.409+0000 7f842725d640 1 -- 172.21.15.28:0/1641677463 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f841801ab20 con 0x55928d7c59f0 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.409+0000 7f8429773800 1 Processor -- start 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.409+0000 7f8429773800 1 -- start start 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.409+0000 7f8429773800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7d6e40 0x55928d7d7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.409+0000 7f8429773800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55928d5bb4a0 con 0x55928d7d6e40 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.409+0000 7f8427a5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7d6e40 0x55928d7d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8427a5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7d6e40 0x55928d7d7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33976/0 (socket says 172.21.15.28:33976) 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8427a5e640 1 -- 172.21.15.28:0/3323382458 learned_addr learned my addr 172.21.15.28:0/3323382458 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8427a5e640 1 -- 172.21.15.28:0/3323382458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d7d78a0 con 0x55928d7d6e40 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8427a5e640 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7d6e40 0x55928d7d7260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55928d6372c0 tx=0x7f84200167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8311602640 1 -- 172.21.15.28:0/3323382458 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8420013640 con 0x55928d7d6e40 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8311602640 1 -- 172.21.15.28:0/3323382458 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84200137e0 con 0x55928d7d6e40 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8311602640 1 -- 172.21.15.28:0/3323382458 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8420013ac0 con 0x55928d7d6e40 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55928d7d80d0 con 0x55928d7d6e40 2024-09-19T11:23:25.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.410+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928d7d8670 con 0x55928d7d6e40 2024-09-19T11:23:25.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.411+0000 7f8311602640 1 -- 172.21.15.28:0/3323382458 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8420011040 con 0x55928d7d6e40 2024-09-19T11:23:25.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.411+0000 7f8311602640 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830003d050 0x7f830003f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.411+0000 7f8311602640 1 -- 172.21.15.28:0/3323382458 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f842000e040 con 0x55928d7d6e40 2024-09-19T11:23:25.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.411+0000 7f842725d640 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830003d050 0x7f830003f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8429773800 1 Processor -- start 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f842725d640 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830003d050 0x7f830003f510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55928d7c3b70 tx=0x7f84180529c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8429773800 1 -- start start 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8429773800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8429773800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55928d7b3c40 con 0x55928d7e38c0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8427a5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8427a5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:33992/0 (socket says 172.21.15.28:33992) 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8427a5e640 1 -- 172.21.15.28:0/1428450780 learned_addr learned my addr 172.21.15.28:0/1428450780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.412+0000 7f8427a5e640 1 -- 172.21.15.28:0/1428450780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d7e42b0 con 0x55928d7e38c0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8427a5e640 1 --2- 172.21.15.28:0/1428450780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7e3cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8420013dd0 tx=0x7f8420013f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf3a9d2ce56d1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f830affd640 1 -- 172.21.15.28:0/1428450780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8420044030 con 0x55928d7e38c0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f830affd640 1 -- 172.21.15.28:0/1428450780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8420021440 con 0x55928d7e38c0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f830affd640 1 -- 172.21.15.28:0/1428450780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84200215f0 con 0x55928d7e38c0 2024-09-19T11:23:25.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8429773800 1 -- 172.21.15.28:0/1428450780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 msgr2=0x55928d7e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8429773800 1 --2- 172.21.15.28:0/1428450780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7e3cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8420013dd0 tx=0x7f8420013f00 comp rx=0 tx=0).stop 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8429773800 1 -- 172.21.15.28:0/1428450780 shutdown_connections 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8429773800 1 --2- 172.21.15.28:0/1428450780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7e3cc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8429773800 1 -- 172.21.15.28:0/1428450780 >> 172.21.15.28:0/1428450780 conn(0x55928d7dfcc0 msgr2=0x55928d7ddf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8429773800 1 -- 172.21.15.28:0/1428450780 shutdown_connections 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.413+0000 7f8429773800 1 -- 172.21.15.28:0/1428450780 wait complete. 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8429773800 1 Processor -- start 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8429773800 1 -- start start 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8429773800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7bc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8429773800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55928d7e48d0 con 0x55928d7e38c0 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8427a5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7bc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8427a5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7bc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34000/0 (socket says 172.21.15.28:34000) 2024-09-19T11:23:25.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 learned_addr learned my addr 172.21.15.28:0/2851889623 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928d7bc650 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8427a5e640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7bc0f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f84200147b0 tx=0x7f84200547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f83097fa640 1 -- 172.21.15.28:0/2851889623 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8420044040 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f83097fa640 1 -- 172.21.15.28:0/2851889623 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842001fa90 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55928d7baa00 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.414+0000 7f83097fa640 1 -- 172.21.15.28:0/2851889623 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f842001fd70 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.415+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928d7bafa0 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.415+0000 7f83097fa640 1 -- 172.21.15.28:0/2851889623 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f842001f500 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.415+0000 7f83097fa640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f82f003cfb0 0x7f82f003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f83097fa640 1 -- 172.21.15.28:0/2851889623 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f842004d050 con 0x55928d7e38c0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f842725d640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f82f003cfb0 0x7f82f003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d7f89e0 0x55928d7faea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7fb5f0 con 0x55928d7f89e0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f842725d640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f82f003cfb0 0x7f82f003f470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8418001250 tx=0x7f8418051000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f8426a5c640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d7f89e0 0x55928d7faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.416+0000 7f8426a5c640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d7f89e0 0x55928d7faea0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 realm 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7fba20 con 0x55928d7f89e0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7fc2f0 con 0x55928d7f89e0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7fc690 con 0x55928d7f89e0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7fd3f0 0x55928d7ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.417+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d7ffd20 con 0x55928d7fd3f0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8427a5e640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7fd3f0 0x55928d7ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8427a5e640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7fd3f0 0x55928d7ff7e0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:25.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8429773800 20 searching for the correct realm 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8427a5e640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8420045610 0x7f8420008970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.418+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8420045610 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.419+0000 7f842725d640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8420045610 0x7f8420008970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.419+0000 7f842725d640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8420045610 0x7f8420008970 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.419+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.419+0000 7f842725d640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8418017050 0x7f841804b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.419+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.419+0000 7f8426a5c640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8418017050 0x7f841804b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8418017050 0x7f841804b890 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.420+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8420045610 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.421+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8420045610 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.422+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.423+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.424+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8420045610 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.425+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8420045610 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7fd3f0 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8418017050 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.426+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x55928d7f89e0 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55928d801690 con 0x7f8420045610 2024-09-19T11:23:25.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d800760 con 0x55928d7f89e0 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d801000 con 0x55928d7f89e0 2024-09-19T11:23:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.427+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d8003a0 con 0x55928d7fd3f0 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 zone default found 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 4 Realm: () 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 zonegroup default 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d800c30 con 0x7f8418017050 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d803f30 con 0x55928d7fd3f0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.428+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d8042d0 con 0x55928d7fd3f0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.429+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.429+0000 7f8429773800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.429+0000 7f8429773800 20 started sync module instance, tier type = 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.429+0000 7f8429773800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.429+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d8033c0 con 0x7f8420045610 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.430+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.430+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94087927576496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d804b90 con 0x7f8420045610 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.430+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d8053e0 con 0x55928d7f89e0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.431+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94087927576496] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.431+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.431+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94087927580384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d8054c0 con 0x55928d7f89e0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.431+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d805fe0 con 0x55928d7fd3f0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.432+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94087927580384] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.432+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.432+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94087927585456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d805fa0 con 0x55928d7fd3f0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.432+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d806a60 con 0x7f8420045610 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.433+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94087927585456] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.433+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.433+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94087927588208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d806e50 con 0x7f8420045610 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.433+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d807890 con 0x55928d7f89e0 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.434+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.434+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94087927591776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d808130 con 0x55928d7f89e0 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.434+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d808bf0 con 0x7f8418017050 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.434+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94087927588208] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.435+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94087927591776] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.435+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.435+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94087927596832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d807bb0 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.435+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d8094c0 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.436+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94087927596832] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.436+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.436+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94087927582512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d807bb0 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.436+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d809ae0 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94087927582512] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94087927598992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d809ae0 con 0x7f8418017050 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 20 add_watcher() i=0 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 20 add_watcher() i=1 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 20 add_watcher() i=2 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 20 add_watcher() i=3 2024-09-19T11:23:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 20 add_watcher() i=4 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 20 add_watcher() i=5 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.437+0000 7f8429773800 20 add_watcher() i=6 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.438+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94087927598992] v31'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.438+0000 7f8429773800 20 add_watcher() i=7 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.438+0000 7f8429773800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.438+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d80cee0 con 0x55928d7f89e0 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.439+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.439+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94087927601152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d80a3d0 con 0x55928d7f89e0 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.440+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94087927601152] v31'100 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.440+0000 7f82ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.440+0000 7f8429773800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.440+0000 7f8429773800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.440+0000 7f8429773800 5 note: GC not initialized 2024-09-19T11:23:25.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.440+0000 7f82dcfe1640 20 reqs_thread_entry: start 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.440+0000 7f8429773800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f8429773800 20 Filter name: none 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f82567fc640 20 reqs_thread_entry: start 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d83fe20 con 0x7f8420045610 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.441+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d8406b0 con 0x7f8418017050 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d83f170 con 0x7f8420045610 2024-09-19T11:23:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d841cf0 con 0x7f8418017050 2024-09-19T11:23:25.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(65 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 4.33 4:cfe5f5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d841720 con 0x7f8418017050 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 4.8 4:13d14a70:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d846610 con 0x55928d7fd3f0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 4.2b 4:d7493e99:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d848e80 con 0x55928d7f89e0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 4.27 4:e79b5c58:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x55928d7f89e0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 4.20 4:06d6ec91:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d84e710 con 0x55928d7f89e0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 4.f 4:f2b3320c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d8515d0 con 0x55928d7f89e0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.442+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:72 4.d 4:b3b80c54:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d854440 con 0x55928d7fd3f0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 4.32 4:4d09bac8:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d857170 con 0x55928d7f89e0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 4.3c 4:3c98ea8c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x55928d7f89e0 2024-09-19T11:23:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 4.c 4:31f23ca3:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d84e710 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 4.31 4:8fabc4b1:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55928d8515d0 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(66 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(72 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(68 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(69 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(70 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(71 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5 [call out=128b] v0'0 uv7 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(73 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(74 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(76 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.443+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(75 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.444+0000 7f82ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.444+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94087927601152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d8515d0 con 0x55928d7f89e0 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.445+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94087927601152] v31'101 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.445+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94087927576496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x7f8420045610 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.446+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94087927576496] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.446+0000 7f8429773800 20 remove_watcher() i=0 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.446+0000 7f8429773800 2 removed watcher, disabling cache 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.446+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94087927580384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d857170 con 0x55928d7f89e0 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.447+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94087927580384] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.447+0000 7f8429773800 20 remove_watcher() i=1 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.447+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94087927585456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x55928d7fd3f0 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.448+0000 7f8427a5e640 1 -- 172.21.15.28:0/2851889623 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94087927585456] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8420031070 con 0x55928d7fd3f0 2024-09-19T11:23:25.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.448+0000 7f8429773800 20 remove_watcher() i=2 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.448+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94087927588208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x7f8420045610 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.449+0000 7f842725d640 1 -- 172.21.15.28:0/2851889623 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94087927588208] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841801ab20 con 0x7f8420045610 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.449+0000 7f8429773800 20 remove_watcher() i=3 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.449+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94087927591776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x55928d7f89e0 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.450+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94087927591776] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x55928d7f89e0 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.450+0000 7f8429773800 20 remove_watcher() i=4 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.450+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94087927596832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x7f8418017050 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.451+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94087927596832] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.451+0000 7f8429773800 20 remove_watcher() i=5 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.451+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94087927582512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x7f8418017050 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.453+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94087927582512] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.453+0000 7f8429773800 20 remove_watcher() i=6 2024-09-19T11:23:25.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.453+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94087927598992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55928d84b850 con 0x7f8418017050 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94087927598992] v31'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f841c051070 con 0x7f8418017050 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 20 remove_watcher() i=7 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8418017050 msgr2=0x7f841804b890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8418017050 0x7f841804b890 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d7f89e0 msgr2=0x55928d7faea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d7f89e0 0x55928d7faea0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8420045610 msgr2=0x7f8420008970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8420045610 0x7f8420008970 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7fd3f0 msgr2=0x55928d7ff7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7fd3f0 0x55928d7ff7e0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f82f003cfb0 msgr2=0x7f82f003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f82f003cfb0 0x7f82f003f470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8418001250 tx=0x7f8418051000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 msgr2=0x55928d7bc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8426a5c640 1 -- 172.21.15.28:0/2851889623 reap_dead start 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7bc0f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f84200147b0 tx=0x7f84200547f0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 shutdown_connections 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/2851889623 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7e38c0 0x55928d7bc0f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 >> 172.21.15.28:0/2851889623 conn(0x55928d7dfcc0 msgr2=0x55928d7dded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 shutdown_connections 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/2851889623 wait complete. 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830003d050 msgr2=0x7f830003f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830003d050 0x7f830003f510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55928d7c3b70 tx=0x7f84180529c0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7d6e40 msgr2=0x55928d7d7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7d6e40 0x55928d7d7260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55928d6372c0 tx=0x7f84200167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 shutdown_connections 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830003d050 0x7f830003f510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 --2- 172.21.15.28:0/3323382458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d7d6e40 0x55928d7d7260 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.454+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 >> 172.21.15.28:0/3323382458 conn(0x55928d7cb1f0 msgr2=0x55928d7cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 shutdown_connections 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/3323382458 wait complete. 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d6c4aa0 msgr2=0x55928d7c3030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d6c4aa0 0x55928d7c3030 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7c59f0 msgr2=0x55928d7c7de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7c59f0 0x55928d7c7de0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830c03cfb0 msgr2=0x7f830c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830c03cfb0 0x7f830c03f470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f841c057000 tx=0x7f841c052000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 msgr2=0x55928d637890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d637890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f841800c010 tx=0x7f8418000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 shutdown_connections 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55928d7c59f0 0x55928d7c7de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f830c03cfb0 0x7f830c03f470 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55928d6c4aa0 0x55928d7c3030 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 --2- 172.21.15.28:0/1641677463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55928d6c3690 0x55928d637890 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 >> 172.21.15.28:0/1641677463 conn(0x55928d6bea60 msgr2=0x55928d7b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 shutdown_connections 2024-09-19T11:23:25.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.455+0000 7f8429773800 1 -- 172.21.15.28:0/1641677463 wait complete. 2024-09-19T11:23:25.469 INFO:teuthology.orchestra.run.smithi028.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"d3056e04-d444-48b9-8e61-96c58b445cdd","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","marker":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","index_type":"Normal","index_generation":0,"num_shards":11,"reshard_status":"None","judge_reshard_lock_time":"0.000000","object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-09-19T11:23:24.617941Z","creation_time":"2024-09-19T11:23:24.614937Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":3,"size_actual":4096,"size_utilized":3,"size_kb":1,"size_kb_actual":4,"size_kb_utilized":1,"num_objects":1}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"read_tracker":1} 2024-09-19T11:23:25.469 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:23:24.617941Z', 'creation_time': '2024-09-19T11:23:24.614937Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 1} 2024-09-19T11:23:25.470 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-09-19T11:23:25.470 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-09-19T11:23:25.470 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-09-19T11:23:25.517 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:25.517 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:25.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.516+0000 7f4cf6739800 1 Processor -- start 2024-09-19T11:23:25.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.516+0000 7f4cf6739800 1 -- start start 2024-09-19T11:23:25.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.516+0000 7f4cf6739800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55fe61b126a0 0x55fe61b133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.516+0000 7f4cf6739800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55fe61b12d50 con 0x55fe61b138e0 2024-09-19T11:23:25.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.516+0000 7f4cf6739800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55fe6199e7d0 con 0x55fe61b126a0 2024-09-19T11:23:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.517+0000 7f4cf23ce640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55fe61b126a0 0x55fe61b133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.517+0000 7f4cf23ce640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55fe61b126a0 0x55fe61b133a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34002/0 (socket says 172.21.15.28:34002) 2024-09-19T11:23:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.517+0000 7f4cf23ce640 1 -- 172.21.15.28:0/125662115 learned_addr learned my addr 172.21.15.28:0/125662115 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.517+0000 7f4cf1bcd640 1 --1- 172.21.15.28:0/125662115 >> v1:172.21.15.22:6789/0 conn(0x55fe61b138e0 0x55fe61b18640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48280/0 (socket says 172.21.15.28:48280) 2024-09-19T11:23:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.517+0000 7f4cf23ce640 1 -- 172.21.15.28:0/125662115 >> v1:172.21.15.22:6789/0 conn(0x55fe61b138e0 legacy=0x55fe61b18640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.517+0000 7f4cf23ce640 1 -- 172.21.15.28:0/125662115 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe61b13d90 con 0x55fe61b126a0 2024-09-19T11:23:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.519+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/125662115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61b133a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005fd0 tx=0x7f4ce4045cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1eb2a7b64ba900 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.519+0000 7f4cf13cc640 1 -- 172.21.15.28:0/125662115 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce404e600 con 0x55fe61b126a0 2024-09-19T11:23:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.519+0000 7f4cf13cc640 1 -- 172.21.15.28:0/125662115 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce404e7a0 con 0x55fe61b126a0 2024-09-19T11:23:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.519+0000 7f4cf13cc640 1 -- 172.21.15.28:0/125662115 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce404eaa0 con 0x55fe61b126a0 2024-09-19T11:23:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.519+0000 7f4cf6739800 1 -- 172.21.15.28:0/125662115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 msgr2=0x55fe61b133a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.519+0000 7f4cf6739800 1 --2- 172.21.15.28:0/125662115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61b133a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4005fd0 tx=0x7f4ce4045cf0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.520+0000 7f4cf6739800 1 -- 172.21.15.28:0/125662115 shutdown_connections 2024-09-19T11:23:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.520+0000 7f4cf6739800 1 --2- 172.21.15.28:0/125662115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61b133a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.520+0000 7f4cf6739800 1 -- 172.21.15.28:0/125662115 >> 172.21.15.28:0/125662115 conn(0x55fe61b0da70 msgr2=0x55fe61b0fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.520+0000 7f4cf6739800 1 -- 172.21.15.28:0/125662115 shutdown_connections 2024-09-19T11:23:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.520+0000 7f4cf6739800 1 -- 172.21.15.28:0/125662115 wait complete. 2024-09-19T11:23:25.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf6739800 1 Processor -- start 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf6739800 1 -- start start 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf6739800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c01120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf6739800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe619abdd0 con 0x55fe61b126a0 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf23ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c01120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf23ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c01120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34008/0 (socket says 172.21.15.28:34008) 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2869534435 learned_addr learned my addr 172.21.15.28:0/2869534435 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.535+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2869534435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe61c01710 con 0x55fe61b126a0 2024-09-19T11:23:25.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2869534435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c01120 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4045d30 tx=0x7f4ce4050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33f7a8fac149737 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4be0608640 1 -- 172.21.15.28:0/2869534435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce404eb40 con 0x55fe61b126a0 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4be0608640 1 -- 172.21.15.28:0/2869534435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce405c040 con 0x55fe61b126a0 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4be0608640 1 -- 172.21.15.28:0/2869534435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce404ee10 con 0x55fe61b126a0 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf6739800 1 -- 172.21.15.28:0/2869534435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 msgr2=0x55fe61c01120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2869534435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c01120 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4045d30 tx=0x7f4ce4050000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf6739800 1 -- 172.21.15.28:0/2869534435 shutdown_connections 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2869534435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c01120 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf6739800 1 -- 172.21.15.28:0/2869534435 >> 172.21.15.28:0/2869534435 conn(0x55fe61b0da70 msgr2=0x55fe61bfcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf6739800 1 -- 172.21.15.28:0/2869534435 shutdown_connections 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.536+0000 7f4cf6739800 1 -- 172.21.15.28:0/2869534435 wait complete. 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf6739800 1 Processor -- start 2024-09-19T11:23:25.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf6739800 1 -- start start 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf6739800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c0cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf6739800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe61a85ec0 con 0x55fe61b126a0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf23ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c0cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf23ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c0cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34010/0 (socket says 172.21.15.28:34010) 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf23ce640 1 -- 172.21.15.28:0/564949381 learned_addr learned my addr 172.21.15.28:0/564949381 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.537+0000 7f4cf23ce640 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe61c0d510 con 0x55fe61b126a0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.538+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c0cfb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4045d30 tx=0x7f4ce4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.538+0000 7f4bdee05640 1 -- 172.21.15.28:0/564949381 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce4061420 con 0x55fe61b126a0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.538+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe61c0d7a0 con 0x55fe61b126a0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.538+0000 7f4bdee05640 1 -- 172.21.15.28:0/564949381 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce4070070 con 0x55fe61b126a0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.538+0000 7f4bdee05640 1 -- 172.21.15.28:0/564949381 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce40616f0 con 0x55fe61b126a0 2024-09-19T11:23:25.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.538+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe61c0e120 con 0x55fe61b126a0 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.539+0000 7f4bdee05640 1 -- 172.21.15.28:0/564949381 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4ce40618e0 con 0x55fe61b126a0 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.539+0000 7f4bdee05640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bd803d000 0x7f4bd803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.539+0000 7f4bdee05640 1 -- 172.21.15.28:0/564949381 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f4ce4066070 con 0x55fe61b126a0 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.539+0000 7f4cf1bcd640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bd803d000 0x7f4bd803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.539+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61b138e0 0x55fe61c140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.539+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c14710 con 0x55fe61b138e0 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61b138e0 0x55fe61c140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf1bcd640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bd803d000 0x7f4bd803f4c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006e30 tx=0x7f4ce8006390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61b138e0 0x55fe61c140f0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/564949381 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4cec0083e0 con 0x55fe61b138e0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c14ff0 con 0x55fe61b138e0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/564949381 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4cec0083e0 con 0x55fe61b138e0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c16900 0x55fe61c18da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.540+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c19530 con 0x55fe61c16900 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.541+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c16900 0x55fe61c18da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.541+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c16900 0x55fe61c18da0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.541+0000 7f4cf23ce640 1 -- 172.21.15.28:0/564949381 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4ce407c070 con 0x55fe61c16900 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.541+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c15840 con 0x55fe61b138e0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.541+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/564949381 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4cec0083e0 con 0x55fe61b138e0 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.541+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c15c60 con 0x55fe61c16900 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.542+0000 7f4cf23ce640 1 -- 172.21.15.28:0/564949381 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4ce407c070 con 0x55fe61c16900 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.542+0000 7f4cf6739800 1 Processor -- start 2024-09-19T11:23:25.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.542+0000 7f4cf6739800 1 -- start start 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.542+0000 7f4cf6739800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c27ef0 0x55fe61c28310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.542+0000 7f4cf6739800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe61a0a4a0 con 0x55fe61c27ef0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4cf2bcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c27ef0 0x55fe61c28310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4cf2bcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c27ef0 0x55fe61c28310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34014/0 (socket says 172.21.15.28:34014) 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2740968616 learned_addr learned my addr 172.21.15.28:0/2740968616 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2740968616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe61c28950 con 0x55fe61c27ef0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c27ef0 0x55fe61c28310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55fe61c0dfb0 tx=0x7f4cec00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4bd7fff640 1 -- 172.21.15.28:0/2740968616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec01c020 con 0x55fe61c27ef0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4bd7fff640 1 -- 172.21.15.28:0/2740968616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cec007c90 con 0x55fe61c27ef0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe61c29180 con 0x55fe61c27ef0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4bd7fff640 1 -- 172.21.15.28:0/2740968616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec018440 con 0x55fe61c27ef0 2024-09-19T11:23:25.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.543+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe61c29720 con 0x55fe61c27ef0 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.544+0000 7f4bd7fff640 1 -- 172.21.15.28:0/2740968616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4cec01f020 con 0x55fe61c27ef0 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.544+0000 7f4bd7fff640 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bcc03d050 0x7f4bcc03f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.544+0000 7f4bd7fff640 1 -- 172.21.15.28:0/2740968616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f4cec055a00 con 0x55fe61c27ef0 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.544+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bcc03d050 0x7f4bcc03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf6739800 1 Processor -- start 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf6739800 1 -- start start 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bcc03d050 0x7f4bcc03f510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55fe61c15be0 tx=0x7f4ce406c510 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf6739800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c34db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf6739800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe61c01a80 con 0x55fe61c349b0 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf2bcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c34db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf2bcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c34db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34024/0 (socket says 172.21.15.28:34024) 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/1472148102 learned_addr learned my addr 172.21.15.28:0/1472148102 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/1472148102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe61c353a0 con 0x55fe61c349b0 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.545+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/1472148102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c34db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4cec053d80 tx=0x7f4cec015de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171b7ba7be390feb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4bd5ffb640 1 -- 172.21.15.28:0/1472148102 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec015820 con 0x55fe61c349b0 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4bd5ffb640 1 -- 172.21.15.28:0/1472148102 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cec016c50 con 0x55fe61c349b0 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4bd5ffb640 1 -- 172.21.15.28:0/1472148102 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec013320 con 0x55fe61c349b0 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 -- 172.21.15.28:0/1472148102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 msgr2=0x55fe61c34db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 --2- 172.21.15.28:0/1472148102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c34db0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4cec053d80 tx=0x7f4cec015de0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 -- 172.21.15.28:0/1472148102 shutdown_connections 2024-09-19T11:23:25.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 --2- 172.21.15.28:0/1472148102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c34db0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 -- 172.21.15.28:0/1472148102 >> 172.21.15.28:0/1472148102 conn(0x55fe61c30db0 msgr2=0x55fe61c2eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 -- 172.21.15.28:0/1472148102 shutdown_connections 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 -- 172.21.15.28:0/1472148102 wait complete. 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 Processor -- start 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.546+0000 7f4cf6739800 1 -- start start 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf6739800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c3c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf6739800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe61c359c0 con 0x55fe61c349b0 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf2bcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c3c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf2bcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c3c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34034/0 (socket says 172.21.15.28:34034) 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 learned_addr learned my addr 172.21.15.28:0/2947635522 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe61c3c590 con 0x55fe61c349b0 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c3c030 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4cec016010 tx=0x7f4cec053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4bc3fff640 1 -- 172.21.15.28:0/2947635522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec02e020 con 0x55fe61c349b0 2024-09-19T11:23:25.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4bc3fff640 1 -- 172.21.15.28:0/2947635522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cec013d20 con 0x55fe61c349b0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe61c3c820 con 0x55fe61c349b0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4bc3fff640 1 -- 172.21.15.28:0/2947635522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cec0187c0 con 0x55fe61c349b0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.547+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe61c3d1a0 con 0x55fe61c349b0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.548+0000 7f4bc3fff640 1 -- 172.21.15.28:0/2947635522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4cec031070 con 0x55fe61c349b0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.548+0000 7f4bc3fff640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bbc03d000 0x7f4bbc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.548+0000 7f4bc3fff640 1 -- 172.21.15.28:0/2947635522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f4cec03b780 con 0x55fe61c349b0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.549+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bbc03d000 0x7f4bbc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.549+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.549+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61c47900 0x55fe61c49dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.549+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c4a510 con 0x55fe61c47900 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.549+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bbc03d000 0x7f4bbc03f4c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ce404e5f0 tx=0x7f4ce4094000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.549+0000 7f4cf1bcd640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61c47900 0x55fe61c49dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.549+0000 7f4cf1bcd640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61c47900 0x55fe61c49dc0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4ce800ba50 con 0x55fe61c47900 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 realm 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c46c50 con 0x55fe61c47900 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4ce800ba50 con 0x55fe61c47900 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c4aff0 con 0x55fe61c47900 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4ce800ba50 con 0x55fe61c47900 2024-09-19T11:23:25.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c4b8c0 con 0x55fe61c47900 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4ce800ba50 con 0x55fe61c47900 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.550+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c4c200 0x55fe61c4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c4ed30 con 0x55fe61c4c200 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c4c200 0x55fe61c4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c4c200 0x55fe61c4e5f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf6739800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf6739800 20 searching for the correct realm 2024-09-19T11:23:25.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.551+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.552+0000 7f4cf2bcf640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4cec021020 0x7f4cec0525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.552+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4cec021020 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.552+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4cec021020 0x7f4cec0525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.552+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4cec021020 0x7f4cec0525a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.552+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.552+0000 7f4cf23ce640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4ce4098bd0 0x7f4ce4098050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.552+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.553+0000 7f4cf1bcd640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4ce4098bd0 0x7f4ce4098050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.553+0000 7f4cf1bcd640 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4ce4098bd0 0x7f4ce4098050 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.553+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce800ba50 con 0x7f4ce4098bd0 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.553+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.553+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.553+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4cec021020 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.554+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.555+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4cec021020 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.556+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.557+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4cec021020 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.558+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.559+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.559+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.559+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.559+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.559+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.559+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4cec021020 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c4c200 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x55fe61c47900 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.560+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c506e0 con 0x7f4cec021020 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:25.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c4f7b0 con 0x55fe61c47900 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c4f3f0 con 0x55fe61c47900 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c50110 con 0x55fe61c4c200 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.561+0000 7f4cf6739800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:25.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 zone default found 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 4 Realm: () 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 zonegroup default 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c4fc80 con 0x7f4ce4098bd0 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c52f50 con 0x55fe61c4c200 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c52f50 con 0x55fe61c4c200 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 started sync module instance, tier type = 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.562+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c523e0 con 0x7f4cec021020 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.563+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.563+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94551050364928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c53380 con 0x7f4cec021020 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.563+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c53bd0 con 0x55fe61c47900 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.565+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94551050364928] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.565+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.565+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94551050367872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c53c50 con 0x55fe61c47900 2024-09-19T11:23:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.565+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c54610 con 0x55fe61c4c200 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.566+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.566+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94551050371424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c545c0 con 0x55fe61c4c200 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.566+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c555e0 con 0x7f4cec021020 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.566+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94551050367872] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.567+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.567+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94551050371424] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.567+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94551050375376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c555a0 con 0x7f4cec021020 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.567+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c55e80 con 0x55fe61c47900 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.568+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.568+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94551050377680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c562a0 con 0x55fe61c47900 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.568+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c56af0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.568+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94551050375376] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.569+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94551050377680] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.569+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.569+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94551050378736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c566c0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.569+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c56f50 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.570+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.570+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94551050380992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c56ae0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.570+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c57ea0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.570+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94551050378736] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94551050380992] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94551050385808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c572b0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 20 add_watcher() i=0 2024-09-19T11:23:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 20 add_watcher() i=1 2024-09-19T11:23:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 20 add_watcher() i=2 2024-09-19T11:23:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 20 add_watcher() i=3 2024-09-19T11:23:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 20 add_watcher() i=4 2024-09-19T11:23:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 20 add_watcher() i=5 2024-09-19T11:23:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.571+0000 7f4cf6739800 20 add_watcher() i=6 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.572+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94551050385808] v31'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.572+0000 7f4cf6739800 20 add_watcher() i=7 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.572+0000 7f4cf6739800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.572+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c5ae90 con 0x55fe61c47900 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.573+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.573+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94551050379792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c58310 con 0x55fe61c47900 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.574+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94551050379792] v31'102 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.574+0000 7f4ba8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.574+0000 7f4cf6739800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.574+0000 7f4cf6739800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.574+0000 7f4cf6739800 5 note: GC not initialized 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.574+0000 7f4b237fe640 20 reqs_thread_entry: start 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.574+0000 7f4cf6739800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf6739800 20 Filter name: none 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4b217fa640 20 reqs_thread_entry: start 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c8d750 con 0x55fe61c47900 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf6739800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c8eb70 con 0x7f4cec021020 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.575+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c8eb40 con 0x7f4ce4098bd0 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(64 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 4.33 4:cfe5f5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c906f0 con 0x7f4ce4098bd0 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 4.8 4:13d14a70:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c933a0 con 0x55fe61c4c200 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 4.2b 4:d7493e99:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c96050 con 0x55fe61c47900 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 4.27 4:e79b5c58:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c933a0 con 0x55fe61c47900 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 4.20 4:06d6ec91:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9b1c0 con 0x55fe61c47900 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 4.f 4:f2b3320c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9dd50 con 0x55fe61c47900 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 4.d 4:b3b80c54:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61ca05c0 con 0x55fe61c4c200 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 4.32 4:4d09bac8:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9b1c0 con 0x55fe61c47900 2024-09-19T11:23:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.576+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 4.3c 4:3c98ea8c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9b1c0 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 4.c 4:31f23ca3:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61ca7a80 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 4.31 4:8fabc4b1:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61caa6b0 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(65 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(71 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(67 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(69 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(73 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(74 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(72 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(70 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5 [call out=128b] v0'0 uv7 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(75 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.577+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c92d10 con 0x7f4cec021020 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.578+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(76 foo.buckets [call] v31'15 uv15 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.578+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x7f4cec021020 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.579+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(77 foo.buckets [call] v31'16 uv16 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.579+0000 7f4cf6739800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.579+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x55fe61c47900 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.580+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(78 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.580+0000 7f4cf6739800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.580+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x7f4cec021020 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.580+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv16 ondisk = 0) ==== 155+0+58 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.581+0000 7f4ba8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.581+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94551050379792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x55fe61c47900 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.582+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 94551050379792] v31'103 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.582+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94551050364928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x7f4cec021020 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.583+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 94551050364928] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.583+0000 7f4cf6739800 20 remove_watcher() i=0 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.583+0000 7f4cf6739800 2 removed watcher, disabling cache 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.583+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94551050367872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x55fe61c47900 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.584+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 94551050367872] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.584+0000 7f4cf6739800 20 remove_watcher() i=1 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.584+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94551050371424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x55fe61c4c200 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.585+0000 7f4cf2bcf640 1 -- 172.21.15.28:0/2947635522 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 94551050371424] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4cec030030 con 0x55fe61c4c200 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.585+0000 7f4cf6739800 20 remove_watcher() i=2 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.585+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94551050375376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x7f4cec021020 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.586+0000 7f4cf23ce640 1 -- 172.21.15.28:0/2947635522 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 94551050375376] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce407c070 con 0x7f4cec021020 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.586+0000 7f4cf6739800 20 remove_watcher() i=3 2024-09-19T11:23:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.586+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94551050377680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x55fe61c47900 2024-09-19T11:23:25.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.587+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 94551050377680] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x55fe61c47900 2024-09-19T11:23:25.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.587+0000 7f4cf6739800 20 remove_watcher() i=4 2024-09-19T11:23:25.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.587+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94551050378736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x7f4ce4098bd0 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.589+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 94551050378736] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.589+0000 7f4cf6739800 20 remove_watcher() i=5 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.589+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94551050380992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x7f4ce4098bd0 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.590+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 94551050380992] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.590+0000 7f4cf6739800 20 remove_watcher() i=6 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.590+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94551050385808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55fe61c9ab30 con 0x7f4ce4098bd0 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 94551050385808] v31'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4ce8024040 con 0x7f4ce4098bd0 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 20 remove_watcher() i=7 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4ce4098bd0 msgr2=0x7f4ce4098050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4ce4098bd0 0x7f4ce4098050 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61c47900 msgr2=0x55fe61c49dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61c47900 0x55fe61c49dc0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4cec021020 msgr2=0x7f4cec0525a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4cec021020 0x7f4cec0525a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c4c200 msgr2=0x55fe61c4e5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c4c200 0x55fe61c4e5f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bbc03d000 msgr2=0x7f4bbc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bbc03d000 0x7f4bbc03f4c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4ce404e5f0 tx=0x7f4ce4094000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf1bcd640 1 -- 172.21.15.28:0/2947635522 reap_dead start 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 msgr2=0x55fe61c3c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c3c030 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4cec016010 tx=0x7f4cec053ce0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 shutdown_connections 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2947635522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c349b0 0x55fe61c3c030 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 >> 172.21.15.28:0/2947635522 conn(0x55fe61c30db0 msgr2=0x55fe61c2efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 shutdown_connections 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2947635522 wait complete. 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bcc03d050 msgr2=0x7f4bcc03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bcc03d050 0x7f4bcc03f510 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55fe61c15be0 tx=0x7f4ce406c510 comp rx=0 tx=0).stop 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c27ef0 msgr2=0x55fe61c28310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c27ef0 0x55fe61c28310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55fe61c0dfb0 tx=0x7f4cec00bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 shutdown_connections 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bcc03d050 0x7f4bcc03f510 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 --2- 172.21.15.28:0/2740968616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61c27ef0 0x55fe61c28310 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.591+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 >> 172.21.15.28:0/2740968616 conn(0x55fe61c1c2a0 msgr2=0x55fe61c1c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 shutdown_connections 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/2740968616 wait complete. 2024-09-19T11:23:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61b138e0 msgr2=0x55fe61c140f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61b138e0 0x55fe61c140f0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c16900 msgr2=0x55fe61c18da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c16900 0x55fe61c18da0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bd803d000 msgr2=0x7f4bd803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bd803d000 0x7f4bd803f4c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8006e30 tx=0x7f4ce8006390 comp rx=0 tx=0).stop 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 msgr2=0x55fe61c0cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c0cfb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4045d30 tx=0x7f4ce4003330 comp rx=0 tx=0).stop 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 shutdown_connections 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fe61c16900 0x55fe61c18da0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4bd803d000 0x7f4bd803f4c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fe61b138e0 0x55fe61c140f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 --2- 172.21.15.28:0/564949381 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe61b126a0 0x55fe61c0cfb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 >> 172.21.15.28:0/564949381 conn(0x55fe61b0da70 msgr2=0x55fe61c02510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 shutdown_connections 2024-09-19T11:23:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.592+0000 7f4cf6739800 1 -- 172.21.15.28:0/564949381 wait complete. 2024-09-19T11:23:25.603 INFO:teuthology.orchestra.run.smithi028.stdout:{"stats":{"size":3,"size_actual":4096,"size_kb":1,"size_kb_actual":4,"num_objects":1},"last_stats_sync":"2024-09-19T11:23:25.579579Z","last_stats_update":"2024-09-19T11:23:25.578378Z"} 2024-09-19T11:23:25.603 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-09-19T11:23:25.579579Z', 'last_stats_update': '2024-09-19T11:23:25.578378Z'} 2024-09-19T11:23:25.609 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:25.609 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-09-19T11:23:25.609 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-09-19T11:23:25.609 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-09-19T11:23:25.656 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:25.657 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:25.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.656+0000 7f242ec63800 1 Processor -- start 2024-09-19T11:23:25.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.656+0000 7f242ec63800 1 -- start start 2024-09-19T11:23:25.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.656+0000 7f242ec63800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5646f4013fd0 0x5646f40163c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.656+0000 7f242ec63800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5646f401a5f0 con 0x5646f4016900 2024-09-19T11:23:25.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.656+0000 7f242ec63800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5646f3e9e7d0 con 0x5646f4013fd0 2024-09-19T11:23:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.657+0000 7f242c754640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5646f4013fd0 0x5646f40163c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.657+0000 7f242c754640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5646f4013fd0 0x5646f40163c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34036/0 (socket says 172.21.15.28:34036) 2024-09-19T11:23:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.657+0000 7f242c754640 1 -- 172.21.15.28:0/3600774772 learned_addr learned my addr 172.21.15.28:0/3600774772 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.657+0000 7f242bf53640 1 --1- 172.21.15.28:0/3600774772 >> v1:172.21.15.22:6789/0 conn(0x5646f4016900 0x5646f4018d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48282/0 (socket says 172.21.15.28:48282) 2024-09-19T11:23:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.657+0000 7f242c754640 1 -- 172.21.15.28:0/3600774772 >> v1:172.21.15.22:6789/0 conn(0x5646f4016900 legacy=0x5646f4018d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.657+0000 7f242c754640 1 -- 172.21.15.28:0/3600774772 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f401b820 con 0x5646f4013fd0 2024-09-19T11:23:25.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.659+0000 7f242c754640 1 --2- 172.21.15.28:0/3600774772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f40163c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f241c00a520 tx=0x7f241c045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a680a573f68d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.659+0000 7f242b752640 1 -- 172.21.15.28:0/3600774772 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c055070 con 0x5646f4013fd0 2024-09-19T11:23:25.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.659+0000 7f242b752640 1 -- 172.21.15.28:0/3600774772 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c0515b0 con 0x5646f4013fd0 2024-09-19T11:23:25.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.659+0000 7f242b752640 1 -- 172.21.15.28:0/3600774772 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c0518d0 con 0x5646f4013fd0 2024-09-19T11:23:25.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.659+0000 7f242ec63800 1 -- 172.21.15.28:0/3600774772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 msgr2=0x5646f40163c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.659+0000 7f242ec63800 1 --2- 172.21.15.28:0/3600774772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f40163c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f241c00a520 tx=0x7f241c045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.660+0000 7f242ec63800 1 -- 172.21.15.28:0/3600774772 shutdown_connections 2024-09-19T11:23:25.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.660+0000 7f242ec63800 1 --2- 172.21.15.28:0/3600774772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f40163c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.660+0000 7f242ec63800 1 -- 172.21.15.28:0/3600774772 >> 172.21.15.28:0/3600774772 conn(0x5646f400da70 msgr2=0x5646f400fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.660+0000 7f242ec63800 1 -- 172.21.15.28:0/3600774772 shutdown_connections 2024-09-19T11:23:25.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.660+0000 7f242ec63800 1 -- 172.21.15.28:0/3600774772 wait complete. 2024-09-19T11:23:25.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242ec63800 1 Processor -- start 2024-09-19T11:23:25.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242ec63800 1 -- start start 2024-09-19T11:23:25.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242ec63800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f4103340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242ec63800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5646f3eabdd0 con 0x5646f4013fd0 2024-09-19T11:23:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242c754640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f4103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242c754640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f4103340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34046/0 (socket says 172.21.15.28:34046) 2024-09-19T11:23:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242c754640 1 -- 172.21.15.28:0/1035068530 learned_addr learned my addr 172.21.15.28:0/1035068530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.675+0000 7f242c754640 1 -- 172.21.15.28:0/1035068530 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f4103930 con 0x5646f4013fd0 2024-09-19T11:23:25.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.676+0000 7f242c754640 1 --2- 172.21.15.28:0/1035068530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f4103340 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f241c025da0 tx=0x7f241c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a402e5dce4998e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.676+0000 7f231a60c640 1 -- 172.21.15.28:0/1035068530 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c051950 con 0x5646f4013fd0 2024-09-19T11:23:25.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.676+0000 7f231a60c640 1 -- 172.21.15.28:0/1035068530 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c051af0 con 0x5646f4013fd0 2024-09-19T11:23:25.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.676+0000 7f231a60c640 1 -- 172.21.15.28:0/1035068530 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c051dd0 con 0x5646f4013fd0 2024-09-19T11:23:25.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.676+0000 7f242ec63800 1 -- 172.21.15.28:0/1035068530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 msgr2=0x5646f4103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.676+0000 7f242ec63800 1 --2- 172.21.15.28:0/1035068530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f4103340 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f241c025da0 tx=0x7f241c051000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 -- 172.21.15.28:0/1035068530 shutdown_connections 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 --2- 172.21.15.28:0/1035068530 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f4103340 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 -- 172.21.15.28:0/1035068530 >> 172.21.15.28:0/1035068530 conn(0x5646f400da70 msgr2=0x5646f40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 -- 172.21.15.28:0/1035068530 shutdown_connections 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 -- 172.21.15.28:0/1035068530 wait complete. 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 Processor -- start 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 -- start start 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f410acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242ec63800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5646f401aac0 con 0x5646f4013fd0 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242c754640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f410acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242c754640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f410acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34062/0 (socket says 172.21.15.28:34062) 2024-09-19T11:23:25.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.677+0000 7f242c754640 1 -- 172.21.15.28:0/2284432072 learned_addr learned my addr 172.21.15.28:0/2284432072 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.678+0000 7f242c754640 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f410b210 con 0x5646f4013fd0 2024-09-19T11:23:25.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.678+0000 7f242c754640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f410acb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f241c067040 tx=0x7f241c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.678+0000 7f2318e09640 1 -- 172.21.15.28:0/2284432072 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c073070 con 0x5646f4013fd0 2024-09-19T11:23:25.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.678+0000 7f2318e09640 1 -- 172.21.15.28:0/2284432072 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c05a450 con 0x5646f4013fd0 2024-09-19T11:23:25.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.678+0000 7f2318e09640 1 -- 172.21.15.28:0/2284432072 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f241c05a730 con 0x5646f4013fd0 2024-09-19T11:23:25.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.678+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646f410b4a0 con 0x5646f4013fd0 2024-09-19T11:23:25.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.678+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f410c0b0 con 0x5646f4013fd0 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.679+0000 7f2318e09640 1 -- 172.21.15.28:0/2284432072 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f241c076070 con 0x5646f4013fd0 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.679+0000 7f2318e09640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f231403d000 0x7f231403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.679+0000 7f2318e09640 1 -- 172.21.15.28:0/2284432072 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f241c062070 con 0x5646f4013fd0 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.679+0000 7f242bf53640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f231403d000 0x7f231403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.679+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f4016900 0x5646f4112300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.679+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4112840 con 0x5646f4016900 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.679+0000 7f242cf55640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f4016900 0x5646f4112300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242bf53640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f231403d000 0x7f231403f4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2420001e30 tx=0x7f2420001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242cf55640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f4016900 0x5646f4112300 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242cf55640 1 -- 172.21.15.28:0/2284432072 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f4016900 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4112be0 con 0x5646f4016900 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242cf55640 1 -- 172.21.15.28:0/2284432072 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f4016900 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f41146a0 0x5646f4116b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f41172d0 con 0x5646f41146a0 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.680+0000 7f242c754640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f41146a0 0x5646f4116b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.681+0000 7f242c754640 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f41146a0 0x5646f4116b40 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.681+0000 7f242c754640 1 -- 172.21.15.28:0/2284432072 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f241c07f070 con 0x5646f41146a0 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.681+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4113430 con 0x5646f4016900 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.681+0000 7f242cf55640 1 -- 172.21.15.28:0/2284432072 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f4016900 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.681+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4113850 con 0x5646f41146a0 2024-09-19T11:23:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.682+0000 7f242c754640 1 -- 172.21.15.28:0/2284432072 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f241c07f070 con 0x5646f41146a0 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.682+0000 7f242ec63800 1 Processor -- start 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.682+0000 7f242ec63800 1 -- start start 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.682+0000 7f242ec63800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013750 0x5646f4011fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.682+0000 7f242ec63800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5646f3f0a4a0 con 0x5646f4013750 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.682+0000 7f242cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013750 0x5646f4011fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f242cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013750 0x5646f4011fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34064/0 (socket says 172.21.15.28:34064) 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f242cf55640 1 -- 172.21.15.28:0/3188087235 learned_addr learned my addr 172.21.15.28:0/3188087235 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f242cf55640 1 -- 172.21.15.28:0/3188087235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f4013c70 con 0x5646f4013750 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f242cf55640 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013750 0x5646f4011fc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5646f410bb40 tx=0x7f24240167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f2311ffb640 1 -- 172.21.15.28:0/3188087235 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2424013640 con 0x5646f4013750 2024-09-19T11:23:25.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f2311ffb640 1 -- 172.21.15.28:0/3188087235 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24240137e0 con 0x5646f4013750 2024-09-19T11:23:25.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f2311ffb640 1 -- 172.21.15.28:0/3188087235 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2424013ae0 con 0x5646f4013750 2024-09-19T11:23:25.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646f4012500 con 0x5646f4013750 2024-09-19T11:23:25.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.683+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f4012aa0 con 0x5646f4013750 2024-09-19T11:23:25.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.684+0000 7f2311ffb640 1 -- 172.21.15.28:0/3188087235 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2424011040 con 0x5646f4013750 2024-09-19T11:23:25.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.684+0000 7f2311ffb640 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f230403d000 0x7f230403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.684+0000 7f2311ffb640 1 -- 172.21.15.28:0/3188087235 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f242400e040 con 0x5646f4013750 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.684+0000 7f242c754640 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f230403d000 0x7f230403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242ec63800 1 Processor -- start 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242c754640 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f230403d000 0x7f230403f4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x5646f41176a0 tx=0x7f241c06f8a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242ec63800 1 -- start start 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242ec63800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f4134ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242ec63800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5646f4103ca0 con 0x5646f41348e0 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f4134ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f4134ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34076/0 (socket says 172.21.15.28:34076) 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242cf55640 1 -- 172.21.15.28:0/1452049810 learned_addr learned my addr 172.21.15.28:0/1452049810 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242cf55640 1 -- 172.21.15.28:0/1452049810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f41352d0 con 0x5646f41348e0 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.685+0000 7f242cf55640 1 --2- 172.21.15.28:0/1452049810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f4134ce0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f24240547f0 tx=0x7f242401fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c67c8a731c2cae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f23037fe640 1 -- 172.21.15.28:0/1452049810 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2424051070 con 0x5646f41348e0 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f23037fe640 1 -- 172.21.15.28:0/1452049810 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2424053440 con 0x5646f41348e0 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f23037fe640 1 -- 172.21.15.28:0/1452049810 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2424021a90 con 0x5646f41348e0 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 -- 172.21.15.28:0/1452049810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 msgr2=0x5646f4134ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 --2- 172.21.15.28:0/1452049810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f4134ce0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f24240547f0 tx=0x7f242401fe70 comp rx=0 tx=0).stop 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 -- 172.21.15.28:0/1452049810 shutdown_connections 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 --2- 172.21.15.28:0/1452049810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f4134ce0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 -- 172.21.15.28:0/1452049810 >> 172.21.15.28:0/1452049810 conn(0x5646f4130ce0 msgr2=0x5646f412ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 -- 172.21.15.28:0/1452049810 shutdown_connections 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 -- 172.21.15.28:0/1452049810 wait complete. 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 Processor -- start 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 -- start start 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.686+0000 7f242ec63800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f413bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242ec63800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5646f41358f0 con 0x5646f41348e0 2024-09-19T11:23:25.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f413bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242cf55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f413bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34082/0 (socket says 172.21.15.28:34082) 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 learned_addr learned my addr 172.21.15.28:0/513354955 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f413c4c0 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242cf55640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f413bf60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24240547f0 tx=0x7f2424054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f2301ffb640 1 -- 172.21.15.28:0/513354955 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2424021440 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f2301ffb640 1 -- 172.21.15.28:0/513354955 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f242404c070 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646f413c750 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f2301ffb640 1 -- 172.21.15.28:0/513354955 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24240459c0 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.687+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f413d2e0 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.688+0000 7f2301ffb640 1 -- 172.21.15.28:0/513354955 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2424045b60 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.688+0000 7f2301ffb640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f22f403d000 0x7f22f403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.688+0000 7f2301ffb640 1 -- 172.21.15.28:0/513354955 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f24240539d0 con 0x5646f41348e0 2024-09-19T11:23:25.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.688+0000 7f242c754640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f22f403d000 0x7f22f403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.689+0000 7f242c754640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f22f403d000 0x7f22f403f4c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f241c051840 tx=0x7f241c08b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.689+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.689+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f41478a0 0x5646f4149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.689+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f414a4b0 con 0x5646f41478a0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.689+0000 7f242bf53640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f41478a0 0x5646f4149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.689+0000 7f242bf53640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f41478a0 0x5646f4149d60 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 realm 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4146bf0 con 0x5646f41478a0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f414afa0 con 0x5646f41478a0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f414b870 con 0x5646f41478a0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.690+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f414c290 0x5646f414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f414edc0 con 0x5646f414c290 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242cf55640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f414c290 0x5646f414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242cf55640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f414c290 0x5646f414e680 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242ec63800 20 searching for the correct realm 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.691+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.692+0000 7f242cf55640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f242404d2a0 0x7f2424033600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.692+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f242404d2a0 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.692+0000 7f242c754640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f242404d2a0 0x7f2424033600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.692+0000 7f242c754640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f242404d2a0 0x7f2424033600 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.692+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.692+0000 7f242c754640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f241c0a6040 0x7f241c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.692+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.693+0000 7f242bf53640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f241c0a6040 0x7f241c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.693+0000 7f242bf53640 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f241c0a6040 0x7f241c06e3c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.693+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.693+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.694+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f242404d2a0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.695+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f242404d2a0 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.696+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.697+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.697+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.697+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.697+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.697+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.697+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.698+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.698+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.698+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.698+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.698+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.698+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f242404d2a0 2024-09-19T11:23:25.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.699+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f242404d2a0 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f414c290 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.700+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f241c0a6040 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x5646f41478a0 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5646f41506f0 con 0x7f242404d2a0 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f414f780 con 0x5646f41478a0 2024-09-19T11:23:25.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.701+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f414ffd0 con 0x5646f41478a0 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4150b70 con 0x5646f414c290 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 zone default found 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 4 Realm: () 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:25.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 zonegroup default 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4150f10 con 0x7f241c0a6040 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.702+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f41531f0 con 0x5646f414c290 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242ec63800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f41535c0 con 0x5646f414c290 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242ec63800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242ec63800 20 started sync module instance, tier type = 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242ec63800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.703+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4152680 con 0x7f242404d2a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.704+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.704+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94862742727280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41522e0 con 0x7f242404d2a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.704+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41544f0 con 0x5646f41478a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.705+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94862742727280] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.705+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.705+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94862742732864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4154910 con 0x5646f41478a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.705+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41552d0 con 0x5646f414c290 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.706+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.706+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94862742736416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4155730 con 0x5646f414c290 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.706+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4156250 con 0x7f242404d2a0 2024-09-19T11:23:25.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.706+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94862742732864] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.707+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94862742736416] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.707+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.707+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94862742740256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4155240 con 0x7f242404d2a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.707+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4156c60 con 0x5646f41478a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.708+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.708+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94862742743440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41573a0 con 0x5646f41478a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.708+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4157ec0 con 0x7f241c0a6040 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.709+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94862742740256] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.709+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.709+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94862742747536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4157e80 con 0x7f241c0a6040 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.710+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41588f0 con 0x7f241c0a6040 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.710+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94862742743440] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.711+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.711+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94862742750240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41588f0 con 0x7f241c0a6040 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.711+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41592b0 con 0x7f241c0a6040 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.711+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94862742747536] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94862742750240] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94862742744560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4158e00 con 0x7f241c0a6040 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 20 add_watcher() i=0 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 20 add_watcher() i=1 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 20 add_watcher() i=2 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 20 add_watcher() i=3 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 20 add_watcher() i=4 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 20 add_watcher() i=5 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.712+0000 7f242ec63800 20 add_watcher() i=6 2024-09-19T11:23:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.713+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94862742744560] v31'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.713+0000 7f242ec63800 20 add_watcher() i=7 2024-09-19T11:23:25.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.713+0000 7f242ec63800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:25.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.713+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f415c2e0 con 0x5646f41478a0 2024-09-19T11:23:25.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.713+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.713+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94862742752768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f41596d0 con 0x5646f41478a0 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.714+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94862742752768] v31'104 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.714+0000 7f22e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.714+0000 7f242ec63800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.714+0000 7f242ec63800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.714+0000 7f242ec63800 5 note: GC not initialized 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f22e1fe3640 20 reqs_thread_entry: start 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 20 Filter name: none 2024-09-19T11:23:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f22daffd640 20 reqs_thread_entry: start 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f418f020 con 0x5646f41478a0 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.715+0000 7f242ec63800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f418ea40 con 0x7f242404d2a0 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f41927e0 con 0x7f241c0a6040 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(64 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-09-19T11:23:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.716+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4195f20 con 0x7f242404d2a0 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(65 foo.buckets [call] v31'17 uv17 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4194f10 con 0x7f242404d2a0 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.718+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f4199a70 con 0x7f242404d2a0 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.719+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v31'18 uv18 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.719+0000 7f242ec63800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-19T11:23:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.719+0000 7f242ec63800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.719+0000 7f242ec63800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.719+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94862743000144 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5646f4196d30 con 0x7f241c0a6040 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== watch-notify(notify (1) cookie 94862742750240 notify 133143986178 ret 0) ==== 380+0+0 (crc 0 0 0) 0x5646f4192bd0 con 0x7f241c0a6040 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f23037fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986178 cookie 94862742750240 notifier 4481 bl.length()=338 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(68 notify.6 [notify cookie 94862743000144 out=8b] v0'0 uv105 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f23037fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f23037fe640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f23037fe640 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f22f0003ea0 con 0x7f241c0a6040 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.720+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== watch-notify(notify_complete (2) cookie 94862743000144 notify 133143986178 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5646f4190ca0 con 0x7f241c0a6040 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.721+0000 7f22e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.721+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94862742752768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x5646f41478a0 2024-09-19T11:23:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.722+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94862742752768] v31'105 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.722+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94862742727280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x7f242404d2a0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.723+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94862742727280] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.723+0000 7f242ec63800 20 remove_watcher() i=0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.723+0000 7f242ec63800 2 removed watcher, disabling cache 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.723+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94862742732864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x5646f41478a0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.725+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94862742732864] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.725+0000 7f242ec63800 20 remove_watcher() i=1 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.725+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94862742736416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x5646f414c290 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.726+0000 7f242cf55640 1 -- 172.21.15.28:0/513354955 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94862742736416] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242400a4d0 con 0x5646f414c290 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.726+0000 7f242ec63800 20 remove_watcher() i=2 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.726+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94862742740256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x7f242404d2a0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.727+0000 7f242c754640 1 -- 172.21.15.28:0/513354955 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94862742740256] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f241c07f070 con 0x7f242404d2a0 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.727+0000 7f242ec63800 20 remove_watcher() i=3 2024-09-19T11:23:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.727+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94862742743440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x5646f41478a0 2024-09-19T11:23:25.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.728+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94862742743440] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x5646f41478a0 2024-09-19T11:23:25.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.728+0000 7f242ec63800 20 remove_watcher() i=4 2024-09-19T11:23:25.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.728+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94862742747536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x7f241c0a6040 2024-09-19T11:23:25.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.729+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94862742747536] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.729+0000 7f242ec63800 20 remove_watcher() i=5 2024-09-19T11:23:25.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.729+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94862742750240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x7f241c0a6040 2024-09-19T11:23:25.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.730+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94862742750240] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.730+0000 7f242ec63800 20 remove_watcher() i=6 2024-09-19T11:23:25.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.730+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94862742744560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5646f418cee0 con 0x7f241c0a6040 2024-09-19T11:23:25.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94862742744560] v31'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f242000f610 con 0x7f241c0a6040 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 20 remove_watcher() i=7 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f241c0a6040 msgr2=0x7f241c06e3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f241c0a6040 0x7f241c06e3c0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f41478a0 msgr2=0x5646f4149d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f41478a0 0x5646f4149d60 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f242404d2a0 msgr2=0x7f2424033600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f242404d2a0 0x7f2424033600 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f414c290 msgr2=0x5646f414e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f414c290 0x5646f414e680 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f22f403d000 msgr2=0x7f22f403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f22f403d000 0x7f22f403f4c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f241c051840 tx=0x7f241c08b000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 msgr2=0x5646f413bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242bf53640 1 -- 172.21.15.28:0/513354955 reap_dead start 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f413bf60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24240547f0 tx=0x7f2424054780 comp rx=0 tx=0).stop 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 shutdown_connections 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/513354955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f41348e0 0x5646f413bf60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 >> 172.21.15.28:0/513354955 conn(0x5646f4130ce0 msgr2=0x5646f412eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 shutdown_connections 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/513354955 wait complete. 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f230403d000 msgr2=0x7f230403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f230403d000 0x7f230403f4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x5646f41176a0 tx=0x7f241c06f8a0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013750 msgr2=0x5646f4011fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.731+0000 7f242ec63800 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013750 0x5646f4011fc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5646f410bb40 tx=0x7f24240167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 shutdown_connections 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f230403d000 0x7f230403f4c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/3188087235 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013750 0x5646f4011fc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 >> 172.21.15.28:0/3188087235 conn(0x5646f411a010 msgr2=0x5646f411a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 shutdown_connections 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/3188087235 wait complete. 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f4016900 msgr2=0x5646f4112300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f4016900 0x5646f4112300 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f41146a0 msgr2=0x5646f4116b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f41146a0 0x5646f4116b40 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f231403d000 msgr2=0x7f231403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f231403d000 0x7f231403f4c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2420001e30 tx=0x7f2420001390 comp rx=0 tx=0).stop 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 msgr2=0x5646f410acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f410acb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f241c067040 tx=0x7f241c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 shutdown_connections 2024-09-19T11:23:25.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5646f41146a0 0x5646f4116b40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f231403d000 0x7f231403f4c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5646f4016900 0x5646f4112300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 --2- 172.21.15.28:0/2284432072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5646f4013fd0 0x5646f410acb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 >> 172.21.15.28:0/2284432072 conn(0x5646f400da70 msgr2=0x5646f4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 shutdown_connections 2024-09-19T11:23:25.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.732+0000 7f242ec63800 1 -- 172.21.15.28:0/2284432072 wait complete. 2024-09-19T11:23:25.743 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-19T11:23:25.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-19T11:23:25.743 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-09-19T11:23:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:25.798 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:25.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.797+0000 7f5c1b663800 1 Processor -- start 2024-09-19T11:23:25.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.797+0000 7f5c1b663800 1 -- start start 2024-09-19T11:23:25.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.797+0000 7f5c1b663800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e5c2c66550 0x55e5c2cfa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.797+0000 7f5c1b663800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e5c2cfa930 con 0x55e5c2c65a40 2024-09-19T11:23:25.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.797+0000 7f5c1b663800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55e5c2b7c7d0 con 0x55e5c2c66550 2024-09-19T11:23:25.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.798+0000 7f5c19154640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55e5c2c65a40 0x55e5c2c65e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48296/0 (socket says 172.21.15.28:48296) 2024-09-19T11:23:25.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.798+0000 7f5c19154640 1 -- 172.21.15.28:0/2216814745 learned_addr learned my addr 172.21.15.28:0/2216814745 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:25.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.798+0000 7f5c18953640 1 --2- 172.21.15.28:0/2216814745 >> v2:172.21.15.22:3300/0 conn(0x55e5c2c66550 0x55e5c2cfa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.798+0000 7f5c13fff640 1 -- 172.21.15.28:0/2216814745 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1216527000 0 0) 0x55e5c2cfa930 con 0x55e5c2c65a40 2024-09-19T11:23:25.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.798+0000 7f5c13fff640 1 -- 172.21.15.28:0/2216814745 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c08001e10 con 0x55e5c2c65a40 2024-09-19T11:23:25.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.798+0000 7f5c18953640 1 -- 172.21.15.28:0/2216814745 >> v1:172.21.15.22:6789/0 conn(0x55e5c2c65a40 legacy=0x55e5c2c65e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.798+0000 7f5c18953640 1 -- 172.21.15.28:0/2216814745 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c2cfbb10 con 0x55e5c2c66550 2024-09-19T11:23:25.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.800+0000 7f5c18953640 1 --2- 172.21.15.28:0/2216814745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c66550 0x55e5c2cfa3f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400a470 tx=0x7f5c04045920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71db12c603f07c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.800+0000 7f5c13fff640 1 -- 172.21.15.28:0/2216814745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c04059070 con 0x55e5c2c66550 2024-09-19T11:23:25.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.800+0000 7f5c13fff640 1 -- 172.21.15.28:0/2216814745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c04054570 con 0x55e5c2c66550 2024-09-19T11:23:25.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.800+0000 7f5c13fff640 1 -- 172.21.15.28:0/2216814745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c04054870 con 0x55e5c2c66550 2024-09-19T11:23:25.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.800+0000 7f5c1b663800 1 -- 172.21.15.28:0/2216814745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c66550 msgr2=0x55e5c2cfa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.800+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2216814745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c66550 0x55e5c2cfa3f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400a470 tx=0x7f5c04045920 comp rx=0 tx=0).stop 2024-09-19T11:23:25.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.801+0000 7f5c1b663800 1 -- 172.21.15.28:0/2216814745 shutdown_connections 2024-09-19T11:23:25.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.801+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2216814745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c66550 0x55e5c2cfa3f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.801+0000 7f5c1b663800 1 -- 172.21.15.28:0/2216814745 >> 172.21.15.28:0/2216814745 conn(0x55e5c2cecca0 msgr2=0x55e5c2cef100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.801+0000 7f5c1b663800 1 -- 172.21.15.28:0/2216814745 shutdown_connections 2024-09-19T11:23:25.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.801+0000 7f5c1b663800 1 -- 172.21.15.28:0/2216814745 wait complete. 2024-09-19T11:23:25.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.816+0000 7f5c1b663800 1 Processor -- start 2024-09-19T11:23:25.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.816+0000 7f5c1b663800 1 -- start start 2024-09-19T11:23:25.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.816+0000 7f5c1b663800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2de3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.816+0000 7f5c1b663800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5c2b89dd0 con 0x55e5c2c65a40 2024-09-19T11:23:25.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5c19154640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2de3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5c19154640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2de3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34104/0 (socket says 172.21.15.28:34104) 2024-09-19T11:23:25.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5c19154640 1 -- 172.21.15.28:0/3342995989 learned_addr learned my addr 172.21.15.28:0/3342995989 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5c19154640 1 -- 172.21.15.28:0/3342995989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c2de3a70 con 0x55e5c2c65a40 2024-09-19T11:23:25.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5c19154640 1 --2- 172.21.15.28:0/3342995989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2de3480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c001e30 tx=0x7f5c0c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1f5eb50489b572 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5b07606640 1 -- 172.21.15.28:0/3342995989 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c012020 con 0x55e5c2c65a40 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5b07606640 1 -- 172.21.15.28:0/3342995989 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c00e470 con 0x55e5c2c65a40 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.817+0000 7f5b07606640 1 -- 172.21.15.28:0/3342995989 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c00e790 con 0x55e5c2c65a40 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 -- 172.21.15.28:0/3342995989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 msgr2=0x55e5c2de3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3342995989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2de3480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c001e30 tx=0x7f5c0c00b780 comp rx=0 tx=0).stop 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 -- 172.21.15.28:0/3342995989 shutdown_connections 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3342995989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2de3480 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 -- 172.21.15.28:0/3342995989 >> 172.21.15.28:0/3342995989 conn(0x55e5c2cecca0 msgr2=0x55e5c2ddee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 -- 172.21.15.28:0/3342995989 shutdown_connections 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 -- 172.21.15.28:0/3342995989 wait complete. 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 Processor -- start 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.818+0000 7f5c1b663800 1 -- start start 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.819+0000 7f5c1b663800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2deaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.819+0000 7f5c1b663800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5c2cfae00 con 0x55e5c2c65a40 2024-09-19T11:23:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.819+0000 7f5c19154640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2deaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.819+0000 7f5c19154640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2deaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34106/0 (socket says 172.21.15.28:34106) 2024-09-19T11:23:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.819+0000 7f5c19154640 1 -- 172.21.15.28:0/3753766812 learned_addr learned my addr 172.21.15.28:0/3753766812 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.819+0000 7f5c19154640 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c2deb4a0 con 0x55e5c2c65a40 2024-09-19T11:23:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.819+0000 7f5c19154640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2deaf40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b7c0 tx=0x7f5c0c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.820+0000 7f5b05e03640 1 -- 172.21.15.28:0/3753766812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c012030 con 0x55e5c2c65a40 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.820+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5c2deb730 con 0x55e5c2c65a40 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.820+0000 7f5b05e03640 1 -- 172.21.15.28:0/3753766812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c0c01a520 con 0x55e5c2c65a40 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.820+0000 7f5b05e03640 1 -- 172.21.15.28:0/3753766812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c01a800 con 0x55e5c2c65a40 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.820+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c2dec130 con 0x55e5c2c65a40 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.821+0000 7f5b05e03640 1 -- 172.21.15.28:0/3753766812 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5c0c015020 con 0x55e5c2c65a40 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.821+0000 7f5b05e03640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5b0003cfb0 0x7f5b0003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.821+0000 7f5b05e03640 1 -- 172.21.15.28:0/3753766812 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5c0c020070 con 0x55e5c2c65a40 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.821+0000 7f5c18953640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5b0003cfb0 0x7f5b0003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.821+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2c66550 0x55e5c2df2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.821+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2df2ad0 con 0x55e5c2c66550 2024-09-19T11:23:25.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.821+0000 7f5c19955640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2c66550 0x55e5c2df2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c18953640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5b0003cfb0 0x7f5b0003f470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e5c2c65880 tx=0x7f5c04050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c19955640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2c66550 0x55e5c2df2590 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c19955640 1 -- 172.21.15.28:0/3753766812 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5c1400a4d0 con 0x55e5c2c66550 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2df2e70 con 0x55e5c2c66550 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c19955640 1 -- 172.21.15.28:0/3753766812 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5c1400a4d0 con 0x55e5c2c66550 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2df4930 0x55e5c2df6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2df7530 con 0x55e5c2df4930 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.822+0000 7f5c19154640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2df4930 0x55e5c2df6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.823+0000 7f5c19154640 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2df4930 0x55e5c2df6da0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.823+0000 7f5c19154640 1 -- 172.21.15.28:0/3753766812 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5c0c054490 con 0x55e5c2df4930 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.823+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2df36c0 con 0x55e5c2c66550 2024-09-19T11:23:25.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.823+0000 7f5c19955640 1 -- 172.21.15.28:0/3753766812 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5c1400a4d0 con 0x55e5c2c66550 2024-09-19T11:23:25.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.823+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2df3a60 con 0x55e5c2df4930 2024-09-19T11:23:25.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.824+0000 7f5c19154640 1 -- 172.21.15.28:0/3753766812 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5c0c054490 con 0x55e5c2df4930 2024-09-19T11:23:25.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.824+0000 7f5c1b663800 1 Processor -- start 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.824+0000 7f5c1b663800 1 -- start start 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.824+0000 7f5c1b663800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e05e70 0x55e5c2e06290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.824+0000 7f5c1b663800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5c2be84a0 con 0x55e5c2e05e70 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.824+0000 7f5c19955640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e05e70 0x55e5c2e06290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5c19955640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e05e70 0x55e5c2e06290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34122/0 (socket says 172.21.15.28:34122) 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5c19955640 1 -- 172.21.15.28:0/2966620128 learned_addr learned my addr 172.21.15.28:0/2966620128 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5c19955640 1 -- 172.21.15.28:0/2966620128 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c2e068d0 con 0x55e5c2e05e70 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5c19955640 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e05e70 0x55e5c2e06290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55e5c2debf40 tx=0x7f5c140167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5afeffd640 1 -- 172.21.15.28:0/2966620128 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c14013640 con 0x55e5c2e05e70 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5afeffd640 1 -- 172.21.15.28:0/2966620128 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c140137e0 con 0x55e5c2e05e70 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5afeffd640 1 -- 172.21.15.28:0/2966620128 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c14013b00 con 0x55e5c2e05e70 2024-09-19T11:23:25.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5c2e07100 con 0x55e5c2e05e70 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.825+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c2e076a0 con 0x55e5c2e05e70 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.826+0000 7f5afeffd640 1 -- 172.21.15.28:0/2966620128 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5c14011040 con 0x55e5c2e05e70 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.826+0000 7f5afeffd640 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5af003d050 0x7f5af003f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.826+0000 7f5afeffd640 1 -- 172.21.15.28:0/2966620128 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5c1400e040 con 0x55e5c2e05e70 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.826+0000 7f5c19154640 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5af003d050 0x7f5af003f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c1b663800 1 Processor -- start 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c1b663800 1 -- start start 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c19154640 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5af003d050 0x7f5af003f510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c022ad0 tx=0x7f5c0c041040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c1b663800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e12cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c1b663800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5c2de3ea0 con 0x55e5c2e128d0 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c19955640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e12cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c19955640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e12cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34136/0 (socket says 172.21.15.28:34136) 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c19955640 1 -- 172.21.15.28:0/2296167262 learned_addr learned my addr 172.21.15.28:0/2296167262 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.827+0000 7f5c19955640 1 -- 172.21.15.28:0/2296167262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c2e132c0 con 0x55e5c2e128d0 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c19955640 1 --2- 172.21.15.28:0/2296167262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e12cd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5c14020000 tx=0x7f5c14021db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd05f6dd3114215 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5afcff9640 1 -- 172.21.15.28:0/2296167262 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c14044030 con 0x55e5c2e128d0 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5afcff9640 1 -- 172.21.15.28:0/2296167262 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c14013dd0 con 0x55e5c2e128d0 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5afcff9640 1 -- 172.21.15.28:0/2296167262 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c14021570 con 0x55e5c2e128d0 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c1b663800 1 -- 172.21.15.28:0/2296167262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 msgr2=0x55e5c2e12cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2296167262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e12cd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5c14020000 tx=0x7f5c14021db0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c1b663800 1 -- 172.21.15.28:0/2296167262 shutdown_connections 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2296167262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e12cd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c1b663800 1 -- 172.21.15.28:0/2296167262 >> 172.21.15.28:0/2296167262 conn(0x55e5c2e0ecf0 msgr2=0x55e5c2e0cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c1b663800 1 -- 172.21.15.28:0/2296167262 shutdown_connections 2024-09-19T11:23:25.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.828+0000 7f5c1b663800 1 -- 172.21.15.28:0/2296167262 wait complete. 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c1b663800 1 Processor -- start 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c1b663800 1 -- start start 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c1b663800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e19f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c1b663800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e5c2e138e0 con 0x55e5c2e128d0 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c19955640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e19f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c19955640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e19f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34144/0 (socket says 172.21.15.28:34144) 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 learned_addr learned my addr 172.21.15.28:0/657101291 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.829+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c2e1a460 con 0x55e5c2e128d0 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.830+0000 7f5c19955640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e19f00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c140547e0 tx=0x7f5c14054810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.830+0000 7f5aeeffd640 1 -- 172.21.15.28:0/657101291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c1401fa90 con 0x55e5c2e128d0 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.830+0000 7f5aeeffd640 1 -- 172.21.15.28:0/657101291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c1401fc30 con 0x55e5c2e128d0 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.830+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5c2e1a6f0 con 0x55e5c2e128d0 2024-09-19T11:23:25.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.830+0000 7f5aeeffd640 1 -- 172.21.15.28:0/657101291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c1401f500 con 0x55e5c2e128d0 2024-09-19T11:23:25.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.830+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c2e1b280 con 0x55e5c2e128d0 2024-09-19T11:23:25.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.831+0000 7f5aeeffd640 1 -- 172.21.15.28:0/657101291 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5c1401f6a0 con 0x55e5c2e128d0 2024-09-19T11:23:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.831+0000 7f5aeeffd640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ae403d000 0x7f5ae403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.831+0000 7f5aeeffd640 1 -- 172.21.15.28:0/657101291 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5c140537d0 con 0x55e5c2e128d0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.831+0000 7f5c19154640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ae403d000 0x7f5ae403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.831+0000 7f5c19154640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ae403d000 0x7f5ae403f4c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00efb0 tx=0x7f5c0c046070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2e25830 0x55e5c2e27cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e28440 con 0x55e5c2e25830 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c18953640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2e25830 0x55e5c2e27cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c18953640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2e25830 0x55e5c2e27cf0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.832+0000 7f5c1b663800 20 realm 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e28870 con 0x55e5c2e25830 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e29140 con 0x55e5c2e25830 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e294e0 con 0x55e5c2e25830 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2e2a240 0x55e5c2e2c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2cc40 con 0x55e5c2e2a240 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.833+0000 7f5c19955640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2e2a240 0x55e5c2e2c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c19955640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2e2a240 0x55e5c2e2c630 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c1b663800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c1b663800 20 searching for the correct realm 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e2a240 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c19955640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5c1404d2d0 0x7f5c14033600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x7f5c1404d2d0 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.834+0000 7f5c19154640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5c1404d2d0 0x7f5c14033600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.835+0000 7f5c19154640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5c1404d2d0 0x7f5c14033600 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.835+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.835+0000 7f5c19154640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c0c017050 0x7f5c0c051320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.835+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x7f5c0c017050 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.835+0000 7f5c18953640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c0c017050 0x7f5c0c051320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.836+0000 7f5c18953640 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c0c017050 0x7f5c0c051320 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.836+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.836+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e25830 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.836+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.836+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e25830 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x7f5c0c017050 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x7f5c0c017050 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x7f5c1404d2d0 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.837+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e2a240 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e25830 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x7f5c0c017050 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e25830 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.838+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x7f5c1404d2d0 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e570 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c0c017050 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.839+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e25830 2024-09-19T11:23:25.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e25830 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.840+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c0c017050 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.841+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.841+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e2a240 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.841+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.841+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c1404d2d0 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.841+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.841+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e25830 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c0c017050 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e2a240 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c0c017050 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.842+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c1404d2d0 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.843+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.843+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e2a240 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.843+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.843+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c0c017050 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.843+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.843+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x55e5c2e25830 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.843+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5c140620c0 con 0x7f5c1404d2d0 2024-09-19T11:23:25.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2d630 con 0x55e5c2e25830 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2de80 con 0x55e5c2e25830 2024-09-19T11:23:25.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.844+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2d270 con 0x55e5c2e2a240 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 zone default found 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 4 Realm: () 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 zonegroup default 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2e9f0 con 0x7f5c0c017050 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e30800 con 0x55e5c2e2a240 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.845+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e30c90 con 0x55e5c2e2a240 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.846+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.846+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.846+0000 7f5c1b663800 20 started sync module instance, tier type = 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.846+0000 7f5c1b663800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:25.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.846+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e2fca0 con 0x7f5c1404d2d0 2024-09-19T11:23:25.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.847+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.847+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94445305526256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e310f0 con 0x7f5c1404d2d0 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.847+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e31940 con 0x55e5c2e25830 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.848+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94445305526256] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.848+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.848+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94445305533584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e31d60 con 0x55e5c2e25830 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.848+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e325b0 con 0x55e5c2e2a240 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.849+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94445305533584] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.849+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.849+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94445305535120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e324a0 con 0x55e5c2e2a240 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.849+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e33000 con 0x7f5c1404d2d0 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.850+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94445305535120] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.850+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.850+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94445305536240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e333f0 con 0x7f5c1404d2d0 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.850+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e33e10 con 0x55e5c2e25830 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.851+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94445305536240] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.851+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.851+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94445305542912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e33dd0 con 0x55e5c2e25830 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.851+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e348c0 con 0x7f5c0c017050 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.852+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.852+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94445305542912] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.852+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94445305545632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e348c0 con 0x7f5c0c017050 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.852+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e352d0 con 0x7f5c0c017050 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.853+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94445305545632] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.854+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.854+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94445305539344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e352d0 con 0x7f5c0c017050 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.854+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e35c90 con 0x7f5c0c017050 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94445305539344] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94445305548800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e35c90 con 0x7f5c0c017050 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 20 add_watcher() i=0 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 20 add_watcher() i=1 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 20 add_watcher() i=2 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 20 add_watcher() i=3 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 20 add_watcher() i=4 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 20 add_watcher() i=5 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.855+0000 7f5c1b663800 20 add_watcher() i=6 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.856+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94445305548800] v31'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0405c040 con 0x7f5c0c017050 2024-09-19T11:23:25.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.856+0000 7f5c1b663800 20 add_watcher() i=7 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.856+0000 7f5c1b663800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.856+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e39090 con 0x55e5c2e25830 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.856+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5c0405c040 con 0x55e5c2e25830 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.856+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94445305551280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e365b0 con 0x55e5c2e25830 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.857+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94445305551280] v31'106 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x55e5c2e25830 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.857+0000 7f5acf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.857+0000 7f5c1b663800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.857+0000 7f5c1b663800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.857+0000 7f5c1b663800 5 note: GC not initialized 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5ace7e4640 20 reqs_thread_entry: start 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 20 Filter name: none 2024-09-19T11:23:25.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5a3ffff640 20 reqs_thread_entry: start 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e6bf70 con 0x7f5c0c017050 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.858+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e6a610 con 0x7f5c0c017050 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.859+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.859+0000 7f5c1b663800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.859+0000 7f5c1b663800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.859+0000 7f5c1b663800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.859+0000 7f5c1b663800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.859+0000 7f5c1b663800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.859+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e6aee0 con 0x7f5c0c017050 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v31'2 uv2 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5c1b663800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5c1b663800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5c1b663800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94445305768496 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e6f8b0 con 0x7f5c0c017050 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== watch-notify(notify (1) cookie 94445305545632 notify 133143986179 ret 0) ==== 598+0+0 (crc 0 0 0) 0x55e5c2e6b280 con 0x7f5c0c017050 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5afcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986179 cookie 94445305545632 notifier 4488 bl.length()=556 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5afcff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-19T11:23:25.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5afcff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(65 notify.5 [notify cookie 94445305768496 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.860+0000 7f5afcff9640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f5adc003ea0 con 0x7f5c0c017050 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.861+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.861+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== watch-notify(notify_complete (2) cookie 94445305768496 notify 133143986179 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e5c2e6b8c0 con 0x7f5c0c017050 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.861+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e6d3c0 con 0x7f5c0c017050 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v31'3 uv3 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c1b663800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c1b663800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c1b663800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94445305768416 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e6f890 con 0x55e5c2e2a240 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== watch-notify(notify (1) cookie 94445305535120 notify 133143986178 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55e5c2e1a6f0 con 0x55e5c2e2a240 2024-09-19T11:23:25.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5afcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986178 cookie 94445305535120 notifier 4488 bl.length()=177 2024-09-19T11:23:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5afcff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:23:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5afcff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 notify.2 [notify cookie 94445305768416 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5afcff9640 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f5adc003ea0 con 0x55e5c2e2a240 2024-09-19T11:23:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== watch-notify(notify_complete (2) cookie 94445305768416 notify 133143986178 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f5c14069020 con 0x55e5c2e2a240 2024-09-19T11:23:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.862+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.873 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-09-19T11:23:25.860029Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:25.860004Z","tags":[],"group_ids":[]} 2024-09-19T11:23:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.863+0000 7f5acf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.863+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94445305551280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x55e5c2e25830 2024-09-19T11:23:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.864+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94445305551280] v31'107 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x55e5c2e25830 2024-09-19T11:23:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.864+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94445305526256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x7f5c1404d2d0 2024-09-19T11:23:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.865+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94445305526256] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.866+0000 7f5c1b663800 20 remove_watcher() i=0 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.866+0000 7f5c1b663800 2 removed watcher, disabling cache 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.866+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94445305533584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x55e5c2e25830 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.866+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94445305533584] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x55e5c2e25830 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.866+0000 7f5c1b663800 20 remove_watcher() i=1 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.866+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94445305535120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x55e5c2e2a240 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.867+0000 7f5c19955640 1 -- 172.21.15.28:0/657101291 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94445305535120] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c14053020 con 0x55e5c2e2a240 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.867+0000 7f5c1b663800 20 remove_watcher() i=2 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.867+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94445305536240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x7f5c1404d2d0 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.868+0000 7f5c19154640 1 -- 172.21.15.28:0/657101291 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94445305536240] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0c054490 con 0x7f5c1404d2d0 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.868+0000 7f5c1b663800 20 remove_watcher() i=3 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.868+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94445305542912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x55e5c2e25830 2024-09-19T11:23:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.869+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94445305542912] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x55e5c2e25830 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.869+0000 7f5c1b663800 20 remove_watcher() i=4 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.869+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94445305545632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x7f5c0c017050 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.870+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94445305545632] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.870+0000 7f5c1b663800 20 remove_watcher() i=5 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.870+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94445305539344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x7f5c0c017050 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.871+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94445305539344] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.871+0000 7f5c1b663800 20 remove_watcher() i=6 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.871+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94445305548800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55e5c2e69d80 con 0x7f5c0c017050 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.872+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94445305548800] v31'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c04080040 con 0x7f5c0c017050 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.872+0000 7f5c1b663800 20 remove_watcher() i=7 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c0c017050 msgr2=0x7f5c0c051320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c0c017050 0x7f5c0c051320 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2e25830 msgr2=0x55e5c2e27cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2e25830 0x55e5c2e27cf0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5c1404d2d0 msgr2=0x7f5c14033600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5c1404d2d0 0x7f5c14033600 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2e2a240 msgr2=0x55e5c2e2c630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2e2a240 0x55e5c2e2c630 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ae403d000 msgr2=0x7f5ae403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ae403d000 0x7f5ae403f4c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00efb0 tx=0x7f5c0c046070 comp rx=0 tx=0).stop 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 msgr2=0x55e5c2e19f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c18953640 1 -- 172.21.15.28:0/657101291 reap_dead start 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e19f00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c140547e0 tx=0x7f5c14054810 comp rx=0 tx=0).stop 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 shutdown_connections 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/657101291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e128d0 0x55e5c2e19f00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 >> 172.21.15.28:0/657101291 conn(0x55e5c2e0ecf0 msgr2=0x55e5c2e0cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 shutdown_connections 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/657101291 wait complete. 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5af003d050 msgr2=0x7f5af003f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5af003d050 0x7f5af003f510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c022ad0 tx=0x7f5c0c041040 comp rx=0 tx=0).stop 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e05e70 msgr2=0x55e5c2e06290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e05e70 0x55e5c2e06290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55e5c2debf40 tx=0x7f5c140167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 shutdown_connections 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5af003d050 0x7f5af003f510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 --2- 172.21.15.28:0/2966620128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2e05e70 0x55e5c2e06290 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 >> 172.21.15.28:0/2966620128 conn(0x55e5c2dfa220 msgr2=0x55e5c2dfa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 shutdown_connections 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.873+0000 7f5c1b663800 1 -- 172.21.15.28:0/2966620128 wait complete. 2024-09-19T11:23:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2c66550 msgr2=0x55e5c2df2590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2c66550 0x55e5c2df2590 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2df4930 msgr2=0x55e5c2df6da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2df4930 0x55e5c2df6da0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5b0003cfb0 msgr2=0x7f5b0003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5b0003cfb0 0x7f5b0003f470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55e5c2c65880 tx=0x7f5c04050000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 msgr2=0x55e5c2deaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2deaf40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b7c0 tx=0x7f5c0c0040e0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 shutdown_connections 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e5c2df4930 0x55e5c2df6da0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5b0003cfb0 0x7f5b0003f470 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e5c2c66550 0x55e5c2df2590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 --2- 172.21.15.28:0/3753766812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e5c2c65a40 0x55e5c2deaf40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 >> 172.21.15.28:0/3753766812 conn(0x55e5c2cecca0 msgr2=0x55e5c2de4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 shutdown_connections 2024-09-19T11:23:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.874+0000 7f5c1b663800 1 -- 172.21.15.28:0/3753766812 wait complete. 2024-09-19T11:23:25.885 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-09-19T11:23:25.860029Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:25.860004Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:25.892 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-19T11:23:25.897 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:25.897 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-09-19T11:23:25.897 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-09-19T11:23:25.897 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-09-19T11:23:25.942 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:25.942 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:25.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.941+0000 7f5b0ac12800 1 Processor -- start 2024-09-19T11:23:25.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.941+0000 7f5b0ac12800 1 -- start start 2024-09-19T11:23:25.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.941+0000 7f5b0ac12800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564fa9939690 0x564fa993a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.941+0000 7f5b0ac12800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564fa9939d40 con 0x564fa993a8d0 2024-09-19T11:23:25.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.941+0000 7f5b0ac12800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x564fa97c57d0 con 0x564fa9939690 2024-09-19T11:23:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.942+0000 7f5b062cb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564fa9939690 0x564fa993a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.942+0000 7f5b062cb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564fa9939690 0x564fa993a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34158/0 (socket says 172.21.15.28:34158) 2024-09-19T11:23:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.942+0000 7f5b062cb640 1 -- 172.21.15.28:0/54148754 learned_addr learned my addr 172.21.15.28:0/54148754 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.942+0000 7f5b05aca640 1 --1- 172.21.15.28:0/54148754 >> v1:172.21.15.22:6789/0 conn(0x564fa993a8d0 0x564fa993f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48306/0 (socket says 172.21.15.28:48306) 2024-09-19T11:23:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.942+0000 7f5b062cb640 1 -- 172.21.15.28:0/54148754 >> v1:172.21.15.22:6789/0 conn(0x564fa993a8d0 legacy=0x564fa993f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.942+0000 7f5b062cb640 1 -- 172.21.15.28:0/54148754 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa993ad80 con 0x564fa9939690 2024-09-19T11:23:25.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.944+0000 7f5b062cb640 1 --2- 172.21.15.28:0/54148754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa993a390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5af800a520 tx=0x7f5af8045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c851e0d7cf9020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.944+0000 7f5b052c9640 1 -- 172.21.15.28:0/54148754 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8055070 con 0x564fa9939690 2024-09-19T11:23:25.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.944+0000 7f5b052c9640 1 -- 172.21.15.28:0/54148754 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af80515b0 con 0x564fa9939690 2024-09-19T11:23:25.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.944+0000 7f5b052c9640 1 -- 172.21.15.28:0/54148754 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af80518d0 con 0x564fa9939690 2024-09-19T11:23:25.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.944+0000 7f5b0ac12800 1 -- 172.21.15.28:0/54148754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 msgr2=0x564fa993a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.944+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/54148754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa993a390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5af800a520 tx=0x7f5af8045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.945+0000 7f5b0ac12800 1 -- 172.21.15.28:0/54148754 shutdown_connections 2024-09-19T11:23:25.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.945+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/54148754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa993a390 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.945+0000 7f5b0ac12800 1 -- 172.21.15.28:0/54148754 >> 172.21.15.28:0/54148754 conn(0x564fa9934a60 msgr2=0x564fa9936ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.945+0000 7f5b0ac12800 1 -- 172.21.15.28:0/54148754 shutdown_connections 2024-09-19T11:23:25.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.945+0000 7f5b0ac12800 1 -- 172.21.15.28:0/54148754 wait complete. 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.960+0000 7f5b0ac12800 1 Processor -- start 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.960+0000 7f5b0ac12800 1 -- start start 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.960+0000 7f5b0ac12800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa98ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.960+0000 7f5b0ac12800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa97d2dd0 con 0x564fa9939690 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.960+0000 7f5b062cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa98ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.960+0000 7f5b062cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa98ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34160/0 (socket says 172.21.15.28:34160) 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.960+0000 7f5b062cb640 1 -- 172.21.15.28:0/2798982964 learned_addr learned my addr 172.21.15.28:0/2798982964 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.961+0000 7f5b062cb640 1 -- 172.21.15.28:0/2798982964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa9a2cf10 con 0x564fa9939690 2024-09-19T11:23:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.961+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2798982964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa98ad860 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5af8025da0 tx=0x7f5af8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8992e30beea71e21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.961+0000 7f59f4608640 1 -- 172.21.15.28:0/2798982964 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8051890 con 0x564fa9939690 2024-09-19T11:23:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.961+0000 7f59f4608640 1 -- 172.21.15.28:0/2798982964 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af8051ac0 con 0x564fa9939690 2024-09-19T11:23:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.961+0000 7f59f4608640 1 -- 172.21.15.28:0/2798982964 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8051da0 con 0x564fa9939690 2024-09-19T11:23:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2798982964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 msgr2=0x564fa98ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2798982964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa98ad860 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5af8025da0 tx=0x7f5af8051000 comp rx=0 tx=0).stop 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2798982964 shutdown_connections 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2798982964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa98ad860 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2798982964 >> 172.21.15.28:0/2798982964 conn(0x564fa9934a60 msgr2=0x564fa9a24bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2798982964 shutdown_connections 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2798982964 wait complete. 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 Processor -- start 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 -- start start 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa9a31c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.962+0000 7f5b0ac12800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa9940d50 con 0x564fa9939690 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.963+0000 7f5b062cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa9a31c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.963+0000 7f5b062cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa9a31c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34176/0 (socket says 172.21.15.28:34176) 2024-09-19T11:23:25.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.963+0000 7f5b062cb640 1 -- 172.21.15.28:0/1617387148 learned_addr learned my addr 172.21.15.28:0/1617387148 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.963+0000 7f5b062cb640 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa9a321b0 con 0x564fa9939690 2024-09-19T11:23:25.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.963+0000 7f5b062cb640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa9a31c50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5af8000c00 tx=0x7f5af8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.963+0000 7f59f2e05640 1 -- 172.21.15.28:0/1617387148 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8073070 con 0x564fa9939690 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.964+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564fa9a32440 con 0x564fa9939690 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.963+0000 7f59f2e05640 1 -- 172.21.15.28:0/1617387148 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af805a450 con 0x564fa9939690 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.964+0000 7f59f2e05640 1 -- 172.21.15.28:0/1617387148 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af805a730 con 0x564fa9939690 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.964+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa9a33050 con 0x564fa9939690 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.964+0000 7f59f2e05640 1 -- 172.21.15.28:0/1617387148 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5af8076070 con 0x564fa9939690 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f59f2e05640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59ec03cfb0 0x7f59ec03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f59f2e05640 1 -- 172.21.15.28:0/1617387148 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5af8062070 con 0x564fa9939690 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f5b05aca640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59ec03cfb0 0x7f59ec03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa993a8d0 0x564fa9a392a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a397e0 con 0x564fa993a8d0 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f5b06acc640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa993a8d0 0x564fa9a392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f5b05aca640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59ec03cfb0 0x7f59ec03f470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001e30 tx=0x7f5afc001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.965+0000 7f5b06acc640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa993a8d0 0x564fa9a392a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.966+0000 7f5b06acc640 1 -- 172.21.15.28:0/1617387148 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5b0000a4d0 con 0x564fa993a8d0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.966+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a39b80 con 0x564fa993a8d0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.966+0000 7f5b06acc640 1 -- 172.21.15.28:0/1617387148 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5b0000a4d0 con 0x564fa993a8d0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.966+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a3b680 0x564fa9a3db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.966+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a3e2b0 con 0x564fa9a3b680 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.966+0000 7f5b062cb640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a3b680 0x564fa9a3db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.966+0000 7f5b062cb640 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a3b680 0x564fa9a3db20 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.967+0000 7f5b062cb640 1 -- 172.21.15.28:0/1617387148 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5af807f070 con 0x564fa9a3b680 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.967+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a3a3d0 con 0x564fa993a8d0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.967+0000 7f5b06acc640 1 -- 172.21.15.28:0/1617387148 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5b0000a4d0 con 0x564fa993a8d0 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.967+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a3a820 con 0x564fa9a3b680 2024-09-19T11:23:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.967+0000 7f5b062cb640 1 -- 172.21.15.28:0/1617387148 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5af807f070 con 0x564fa9a3b680 2024-09-19T11:23:25.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b0ac12800 1 Processor -- start 2024-09-19T11:23:25.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b0ac12800 1 -- start start 2024-09-19T11:23:25.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b0ac12800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a4cc70 0x564fa9a4d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b0ac12800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa98314a0 con 0x564fa9a4cc70 2024-09-19T11:23:25.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b06acc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a4cc70 0x564fa9a4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b06acc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a4cc70 0x564fa9a4d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34182/0 (socket says 172.21.15.28:34182) 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b06acc640 1 -- 172.21.15.28:0/2744909290 learned_addr learned my addr 172.21.15.28:0/2744909290 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.968+0000 7f5b06acc640 1 -- 172.21.15.28:0/2744909290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa9a4d6d0 con 0x564fa9a4cc70 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.969+0000 7f5b06acc640 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a4cc70 0x564fa9a4d090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x564fa9a32ae0 tx=0x7f5b000167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.969+0000 7f59ebfff640 1 -- 172.21.15.28:0/2744909290 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b00013640 con 0x564fa9a4cc70 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.969+0000 7f59ebfff640 1 -- 172.21.15.28:0/2744909290 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b000137e0 con 0x564fa9a4cc70 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.969+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564fa9a4df00 con 0x564fa9a4cc70 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.969+0000 7f59ebfff640 1 -- 172.21.15.28:0/2744909290 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b00013ac0 con 0x564fa9a4cc70 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.969+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa9a4e4a0 con 0x564fa9a4cc70 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f59ebfff640 1 -- 172.21.15.28:0/2744909290 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5b00011040 con 0x564fa9a4cc70 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f59ebfff640 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003d000 0x7f59e003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f59ebfff640 1 -- 172.21.15.28:0/2744909290 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5b0000e040 con 0x564fa9a4cc70 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003d000 0x7f59e003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f5b0ac12800 1 Processor -- start 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003d000 0x7f59e003f4c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5af8096e00 tx=0x7f5af8096cd0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f5b0ac12800 1 -- start start 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f5b0ac12800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.970+0000 7f5b0ac12800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa9a2d110 con 0x564fa9a596d0 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f5b06acc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f5b06acc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a59ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34192/0 (socket says 172.21.15.28:34192) 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f5b06acc640 1 -- 172.21.15.28:0/4150995630 learned_addr learned my addr 172.21.15.28:0/4150995630 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f5b06acc640 1 -- 172.21.15.28:0/4150995630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa9a5a0c0 con 0x564fa9a596d0 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f5b06acc640 1 --2- 172.21.15.28:0/4150995630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a59ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000afe0 tx=0x7f5b00013f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f789492cd1bbd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f59e9ffb640 1 -- 172.21.15.28:0/4150995630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b00044030 con 0x564fa9a596d0 2024-09-19T11:23:25.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f59e9ffb640 1 -- 172.21.15.28:0/4150995630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b00056070 con 0x564fa9a596d0 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.971+0000 7f59e9ffb640 1 -- 172.21.15.28:0/4150995630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b00056220 con 0x564fa9a596d0 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 -- 172.21.15.28:0/4150995630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 msgr2=0x564fa9a59ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/4150995630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a59ad0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000afe0 tx=0x7f5b00013f00 comp rx=0 tx=0).stop 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 -- 172.21.15.28:0/4150995630 shutdown_connections 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/4150995630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a59ad0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 -- 172.21.15.28:0/4150995630 >> 172.21.15.28:0/4150995630 conn(0x564fa9a55ad0 msgr2=0x564fa9a53d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 -- 172.21.15.28:0/4150995630 shutdown_connections 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 -- 172.21.15.28:0/4150995630 wait complete. 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 Processor -- start 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 -- start start 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a2a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.972+0000 7f5b0ac12800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564fa9a5a6e0 con 0x564fa9a596d0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f5b06acc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a2a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f5b06acc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a2a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34202/0 (socket says 172.21.15.28:34202) 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 learned_addr learned my addr 172.21.15.28:0/2940036269 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa9a2ab80 con 0x564fa9a596d0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f5b06acc640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a2a620 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5b000536e0 tx=0x7f5b000547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f59d7fff640 1 -- 172.21.15.28:0/2940036269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b00056220 con 0x564fa9a596d0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f59d7fff640 1 -- 172.21.15.28:0/2940036269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0001fca0 con 0x564fa9a596d0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f59d7fff640 1 -- 172.21.15.28:0/2940036269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b0001f500 con 0x564fa9a596d0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564fa9a28f30 con 0x564fa9a596d0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.973+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa9a294d0 con 0x564fa9a596d0 2024-09-19T11:23:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.974+0000 7f59d7fff640 1 -- 172.21.15.28:0/2940036269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5b0001f6a0 con 0x564fa9a596d0 2024-09-19T11:23:25.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.974+0000 7f59d7fff640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d003d000 0x7f59d003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.974+0000 7f59d7fff640 1 -- 172.21.15.28:0/2940036269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5b000174e0 con 0x564fa9a596d0 2024-09-19T11:23:25.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.974+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d003d000 0x7f59d003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.975+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.975+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa9a6e7b0 0x564fa9a70c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.975+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a713c0 con 0x564fa9a6e7b0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.975+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d003d000 0x7f59d003f4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5af8065fd0 tx=0x7f5af808c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.975+0000 7f5b05aca640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa9a6e7b0 0x564fa9a70c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.975+0000 7f5b05aca640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa9a6e7b0 0x564fa9a70c70 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5afc00f610 con 0x564fa9a6e7b0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 realm 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a6db00 con 0x564fa9a6e7b0 2024-09-19T11:23:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5afc00f610 con 0x564fa9a6e7b0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a71ea0 con 0x564fa9a6e7b0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5afc00f610 con 0x564fa9a6e7b0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a72770 con 0x564fa9a6e7b0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5afc00f610 con 0x564fa9a6e7b0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a730b0 0x564fa9a754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a75be0 con 0x564fa9a730b0 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.976+0000 7f5b06acc640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a730b0 0x564fa9a754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b06acc640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a730b0 0x564fa9a754a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b0ac12800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b0ac12800 20 searching for the correct realm 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b06acc640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b0003c3f0 0x7f5b00049980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5b0003c3f0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.977+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b0003c3f0 0x7f5b00049980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.978+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b0003c3f0 0x7f5b00049980 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.978+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.978+0000 7f5b062cb640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5af8078070 0x7f5af8095220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.978+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.978+0000 7f5b05aca640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5af8078070 0x7f5af8095220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.979+0000 7f5b05aca640 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5af8078070 0x7f5af8095220 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.979+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc00f610 con 0x7f5af8078070 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.979+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.979+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc00f610 con 0x564fa9a6e7b0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.979+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.979+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x564fa9a6e7b0 2024-09-19T11:23:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.979+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x7f5af8078070 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x7f5af8078070 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5b0003c3f0 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.980+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x564fa9a6e7b0 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x7f5af8078070 2024-09-19T11:23:25.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x564fa9a6e7b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5b0003c3f0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.981+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x7f5af8078070 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.982+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x564fa9a6e7b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5afc02b040 con 0x564fa9a6e7b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x7f5af8078070 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.983+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5b0003c3f0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.984+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.984+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.984+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc02b040 con 0x564fa9a6e7b0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.984+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.984+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.984+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5b0003c3f0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a730b0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.985+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5af8078070 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x564fa9a6e7b0 2024-09-19T11:23:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a77590 con 0x7f5b0003c3f0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a76660 con 0x564fa9a6e7b0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.986+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a76270 con 0x564fa9a6e7b0 2024-09-19T11:23:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a76f50 con 0x564fa9a730b0 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 zone default found 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 4 Realm: () 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 zonegroup default 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a76b00 con 0x7f5af8078070 2024-09-19T11:23:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a799d0 con 0x564fa9a730b0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.987+0000 7f5b0ac12800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.988+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.988+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a79e30 con 0x564fa9a730b0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.988+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.988+0000 7f5b0ac12800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.988+0000 7f5b0ac12800 20 started sync module instance, tier type = 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.988+0000 7f5b0ac12800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.988+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a79e30 con 0x7f5b0003c3f0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.989+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.989+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94900148734624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a79520 con 0x7f5b0003c3f0 2024-09-19T11:23:26.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.989+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7a930 con 0x564fa9a6e7b0 2024-09-19T11:23:26.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.990+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94900148734624] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.990+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.990+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94900148741120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7ad20 con 0x564fa9a6e7b0 2024-09-19T11:23:26.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.990+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7b760 con 0x564fa9a730b0 2024-09-19T11:23:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.991+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.991+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94900148744752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7b760 con 0x564fa9a730b0 2024-09-19T11:23:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.991+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7c760 con 0x7f5b0003c3f0 2024-09-19T11:23:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.991+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94900148741120] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.992+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94900148744752] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.992+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.992+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94900148748880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7bb50 con 0x7f5b0003c3f0 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.992+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7d000 con 0x564fa9a6e7b0 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.993+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94900148748880] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.993+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.993+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94900148751184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7cf70 con 0x564fa9a6e7b0 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.993+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7dec0 con 0x7f5af8078070 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.994+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94900148751184] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.994+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.994+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94900148753312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7dec0 con 0x7f5af8078070 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.994+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7e880 con 0x7f5af8078070 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.995+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94900148753312] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.996+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.996+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94900148755440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7e880 con 0x7f5af8078070 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.996+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7f240 con 0x7f5af8078070 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.996+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94900148755440] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94900148757936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7f240 con 0x7f5af8078070 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 20 add_watcher() i=0 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 20 add_watcher() i=1 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 20 add_watcher() i=2 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 20 add_watcher() i=3 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 20 add_watcher() i=4 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 20 add_watcher() i=5 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.997+0000 7f5b0ac12800 20 add_watcher() i=6 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.998+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94900148757936] v31'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.998+0000 7f5b0ac12800 20 add_watcher() i=7 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.998+0000 7f5b0ac12800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.998+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a82640 con 0x564fa9a6e7b0 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.998+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.998+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94900148760416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a7fb90 con 0x564fa9a6e7b0 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.999+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94900148760416] v31'108 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.999+0000 7f59bcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.999+0000 7f5b0ac12800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.999+0000 7f5b0ac12800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.999+0000 7f5b0ac12800 5 note: GC not initialized 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.999+0000 7f59377fe640 20 reqs_thread_entry: start 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:25.999+0000 7f5b0ac12800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.000+0000 7f5b0ac12800 20 Filter name: none 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.000+0000 7f59357fa640 20 reqs_thread_entry: start 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.000+0000 7f5b0ac12800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.000+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab5020 con 0x7f5b0003c3f0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.000+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv18 ondisk = 0) ==== 275+0+304 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.000+0000 7f5b0ac12800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.001+0000 7f59bcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.001+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94900148760416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9a6bbc0 con 0x564fa9a6e7b0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.002+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94900148760416] v31'109 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.002+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94900148734624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x7f5b0003c3f0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.003+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94900148734624] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.003+0000 7f5b0ac12800 20 remove_watcher() i=0 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.003+0000 7f5b0ac12800 2 removed watcher, disabling cache 2024-09-19T11:23:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.003+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94900148741120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x564fa9a6e7b0 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.004+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94900148741120] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.004+0000 7f5b0ac12800 20 remove_watcher() i=1 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.004+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94900148744752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x564fa9a730b0 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.005+0000 7f5b06acc640 1 -- 172.21.15.28:0/2940036269 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94900148744752] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b00053020 con 0x564fa9a730b0 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.005+0000 7f5b0ac12800 20 remove_watcher() i=2 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.005+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94900148748880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x7f5b0003c3f0 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.006+0000 7f5b062cb640 1 -- 172.21.15.28:0/2940036269 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94900148748880] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af8087050 con 0x7f5b0003c3f0 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.006+0000 7f5b0ac12800 20 remove_watcher() i=3 2024-09-19T11:23:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.006+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94900148751184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x564fa9a6e7b0 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.008+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94900148751184] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x564fa9a6e7b0 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.008+0000 7f5b0ac12800 20 remove_watcher() i=4 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.008+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94900148753312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x7f5af8078070 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.009+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94900148753312] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.009+0000 7f5b0ac12800 20 remove_watcher() i=5 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.009+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94900148755440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x7f5af8078070 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.009+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94900148755440] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.010+0000 7f5b0ac12800 20 remove_watcher() i=6 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.010+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94900148757936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x564fa9ab2fd0 con 0x7f5af8078070 2024-09-19T11:23:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94900148757936] v31'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5afc03b040 con 0x7f5af8078070 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 20 remove_watcher() i=7 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5af8078070 msgr2=0x7f5af8095220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5af8078070 0x7f5af8095220 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa9a6e7b0 msgr2=0x564fa9a70c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa9a6e7b0 0x564fa9a70c70 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b0003c3f0 msgr2=0x7f5b00049980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b0003c3f0 0x7f5b00049980 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a730b0 msgr2=0x564fa9a754a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a730b0 0x564fa9a754a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d003d000 msgr2=0x7f59d003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d003d000 0x7f59d003f4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5af8065fd0 tx=0x7f5af808c000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 msgr2=0x564fa9a2a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2940036269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a596d0 0x564fa9a2a620 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5b000536e0 tx=0x7f5b000547f0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 reap_dead start 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b05aca640 1 -- 172.21.15.28:0/2940036269 reap_dead start 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 shutdown_connections 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 >> 172.21.15.28:0/2940036269 conn(0x564fa9a55ad0 msgr2=0x564fa9a53ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 shutdown_connections 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2940036269 wait complete. 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003d000 msgr2=0x7f59e003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003d000 0x7f59e003f4c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5af8096e00 tx=0x7f5af8096cd0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a4cc70 msgr2=0x564fa9a4d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a4cc70 0x564fa9a4d090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x564fa9a32ae0 tx=0x7f5b000167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 shutdown_connections 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003d000 0x7f59e003f4c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/2744909290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9a4cc70 0x564fa9a4d090 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 >> 172.21.15.28:0/2744909290 conn(0x564fa9a41020 msgr2=0x564fa9a41400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 shutdown_connections 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.011+0000 7f5b0ac12800 1 -- 172.21.15.28:0/2744909290 wait complete. 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa993a8d0 msgr2=0x564fa9a392a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa993a8d0 0x564fa9a392a0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a3b680 msgr2=0x564fa9a3db20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a3b680 0x564fa9a3db20 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59ec03cfb0 msgr2=0x7f59ec03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59ec03cfb0 0x7f59ec03f470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5afc001e30 tx=0x7f5afc001390 comp rx=0 tx=0).stop 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 msgr2=0x564fa9a31c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa9a31c50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5af8000c00 tx=0x7f5af8002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 shutdown_connections 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564fa9a3b680 0x564fa9a3db20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59ec03cfb0 0x7f59ec03f470 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564fa993a8d0 0x564fa9a392a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 --2- 172.21.15.28:0/1617387148 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564fa9939690 0x564fa9a31c50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 >> 172.21.15.28:0/1617387148 conn(0x564fa9934a60 msgr2=0x564fa9a233c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 shutdown_connections 2024-09-19T11:23:26.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.012+0000 7f5b0ac12800 1 -- 172.21.15.28:0/1617387148 wait complete. 2024-09-19T11:23:26.022 INFO:teuthology.orchestra.run.smithi028.stdout:{"key":"bucket:myfoo","ver":{"tag":"_juFS-IoP_o7R49KvYDOGEfw","ver":2},"mtime":"2024-09-19T11:23:25.719168Z","data":{"bucket":{"name":"myfoo","marker":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","bucket_id":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","tenant":"","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"foo","creation_time":"2024-09-19T11:23:24.614937Z","linked":false,"has_bucket_info":false}} 2024-09-19T11:23:26.022 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_juFS-IoP_o7R49KvYDOGEfw', 'ver': 2}, 'mtime': '2024-09-19T11:23:25.719168Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'bucket_id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-09-19T11:23:24.614937Z', 'linked': False, 'has_bucket_info': False}} 2024-09-19T11:23:26.023 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1'] 2024-09-19T11:23:26.023 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1'] 2024-09-19T11:23:26.023 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:26.073 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:26.073 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:26.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.072+0000 7f6afe463800 1 Processor -- start 2024-09-19T11:23:26.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.072+0000 7f6afe463800 1 -- start start 2024-09-19T11:23:26.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.072+0000 7f6afe463800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561d0e990d40 0x561d0e991140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.072+0000 7f6afe463800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561d0e992240 con 0x561d0e991680 2024-09-19T11:23:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.072+0000 7f6afe463800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x561d0e81d7d0 con 0x561d0e990d40 2024-09-19T11:23:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.073+0000 7f6afbf54640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561d0e990d40 0x561d0e991140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.073+0000 7f6afbf54640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561d0e990d40 0x561d0e991140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34210/0 (socket says 172.21.15.28:34210) 2024-09-19T11:23:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.073+0000 7f6afb753640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x561d0e991680 0x561d0e991b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48314/0 (socket says 172.21.15.28:48314) 2024-09-19T11:23:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.073+0000 7f6afbf54640 1 -- 172.21.15.28:0/240866345 learned_addr learned my addr 172.21.15.28:0/240866345 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.073+0000 7f6afbf54640 1 -- 172.21.15.28:0/240866345 >> v1:172.21.15.22:6789/0 conn(0x561d0e991680 legacy=0x561d0e991b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.073+0000 7f6afbf54640 1 -- 172.21.15.28:0/240866345 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0e992410 con 0x561d0e990d40 2024-09-19T11:23:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afbf54640 1 --2- 172.21.15.28:0/240866345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0e991140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005fd0 tx=0x7f6aec045cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564f2e8a77b5b2aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afaf52640 1 -- 172.21.15.28:0/240866345 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6aec04e600 con 0x561d0e990d40 2024-09-19T11:23:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afaf52640 1 -- 172.21.15.28:0/240866345 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec04e7a0 con 0x561d0e990d40 2024-09-19T11:23:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afaf52640 1 -- 172.21.15.28:0/240866345 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6aec04eaa0 con 0x561d0e990d40 2024-09-19T11:23:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afe463800 1 -- 172.21.15.28:0/240866345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 msgr2=0x561d0e991140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afe463800 1 --2- 172.21.15.28:0/240866345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0e991140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005fd0 tx=0x7f6aec045cf0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afe463800 1 -- 172.21.15.28:0/240866345 shutdown_connections 2024-09-19T11:23:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afe463800 1 --2- 172.21.15.28:0/240866345 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0e991140 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afe463800 1 -- 172.21.15.28:0/240866345 >> 172.21.15.28:0/240866345 conn(0x561d0e98ca70 msgr2=0x561d0e98eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.075+0000 7f6afe463800 1 -- 172.21.15.28:0/240866345 shutdown_connections 2024-09-19T11:23:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.076+0000 7f6afe463800 1 -- 172.21.15.28:0/240866345 wait complete. 2024-09-19T11:23:26.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afe463800 1 Processor -- start 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afe463800 1 -- start start 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afe463800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea84610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afe463800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d0e82add0 con 0x561d0e990d40 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afbf54640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea84610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afbf54640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea84610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34214/0 (socket says 172.21.15.28:34214) 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afbf54640 1 -- 172.21.15.28:0/1425095509 learned_addr learned my addr 172.21.15.28:0/1425095509 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.091+0000 7f6afbf54640 1 -- 172.21.15.28:0/1425095509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0ea84c00 con 0x561d0e990d40 2024-09-19T11:23:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f6afbf54640 1 --2- 172.21.15.28:0/1425095509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea84610 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6aec045d30 tx=0x7f6aec050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b0f42bde1a8593 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f69e9e0b640 1 -- 172.21.15.28:0/1425095509 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6aec04eb40 con 0x561d0e990d40 2024-09-19T11:23:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f69e9e0b640 1 -- 172.21.15.28:0/1425095509 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec05c040 con 0x561d0e990d40 2024-09-19T11:23:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f69e9e0b640 1 -- 172.21.15.28:0/1425095509 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6aec04ee10 con 0x561d0e990d40 2024-09-19T11:23:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f6afe463800 1 -- 172.21.15.28:0/1425095509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 msgr2=0x561d0ea84610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f6afe463800 1 --2- 172.21.15.28:0/1425095509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea84610 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f6aec045d30 tx=0x7f6aec050000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f6afe463800 1 -- 172.21.15.28:0/1425095509 shutdown_connections 2024-09-19T11:23:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f6afe463800 1 --2- 172.21.15.28:0/1425095509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea84610 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.092+0000 7f6afe463800 1 -- 172.21.15.28:0/1425095509 >> 172.21.15.28:0/1425095509 conn(0x561d0e98ca70 msgr2=0x561d0ea7ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afe463800 1 -- 172.21.15.28:0/1425095509 shutdown_connections 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afe463800 1 -- 172.21.15.28:0/1425095509 wait complete. 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afe463800 1 Processor -- start 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afe463800 1 -- start start 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afe463800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea8c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afe463800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d0e9929a0 con 0x561d0e990d40 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afbf54640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afbf54640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea8c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34216/0 (socket says 172.21.15.28:34216) 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.093+0000 7f6afbf54640 1 -- 172.21.15.28:0/671914713 learned_addr learned my addr 172.21.15.28:0/671914713 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.094+0000 7f6afbf54640 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0ea8c5f0 con 0x561d0e990d40 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.094+0000 7f6afbf54640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea8c090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6aec045d30 tx=0x7f6aec003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.094+0000 7f69e3fff640 1 -- 172.21.15.28:0/671914713 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6aec061420 con 0x561d0e990d40 2024-09-19T11:23:26.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.094+0000 7f69e3fff640 1 -- 172.21.15.28:0/671914713 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec070070 con 0x561d0e990d40 2024-09-19T11:23:26.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.094+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d0ea8c880 con 0x561d0e990d40 2024-09-19T11:23:26.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.094+0000 7f69e3fff640 1 -- 172.21.15.28:0/671914713 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6aec0616f0 con 0x561d0e990d40 2024-09-19T11:23:26.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.094+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0ea8d200 con 0x561d0e990d40 2024-09-19T11:23:26.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.095+0000 7f69e3fff640 1 -- 172.21.15.28:0/671914713 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6aec0618e0 con 0x561d0e990d40 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.095+0000 7f69e3fff640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69e403cfb0 0x7f69e403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.095+0000 7f69e3fff640 1 -- 172.21.15.28:0/671914713 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f6aec066070 con 0x561d0e990d40 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.095+0000 7f6afb753640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69e403cfb0 0x7f69e403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.095+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0e991680 0x561d0ea931d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.095+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ea937f0 con 0x561d0e991680 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.096+0000 7f6afc755640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0e991680 0x561d0ea931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.096+0000 7f6afb753640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69e403cfb0 0x7f69e403f470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6af0001e30 tx=0x7f6af0001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.096+0000 7f6afc755640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0e991680 0x561d0ea931d0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.096+0000 7f6afc755640 1 -- 172.21.15.28:0/671914713 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6af40083e0 con 0x561d0e991680 2024-09-19T11:23:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.096+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ea940d0 con 0x561d0e991680 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.097+0000 7f6afc755640 1 -- 172.21.15.28:0/671914713 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6af40083e0 con 0x561d0e991680 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.097+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0ea959e0 0x561d0ea97e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.097+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ea98610 con 0x561d0ea959e0 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.097+0000 7f6afbf54640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0ea959e0 0x561d0ea97e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.097+0000 7f6afbf54640 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0ea959e0 0x561d0ea97e80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.097+0000 7f6afbf54640 1 -- 172.21.15.28:0/671914713 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6aec07c070 con 0x561d0ea959e0 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.097+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ea94920 con 0x561d0e991680 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.098+0000 7f6afc755640 1 -- 172.21.15.28:0/671914713 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6af40083e0 con 0x561d0e991680 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.098+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ea94cc0 con 0x561d0ea959e0 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.098+0000 7f6afbf54640 1 -- 172.21.15.28:0/671914713 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6aec07c070 con 0x561d0ea959e0 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.098+0000 7f6afe463800 1 Processor -- start 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.098+0000 7f6afe463800 1 -- start start 2024-09-19T11:23:26.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.098+0000 7f6afe463800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eaa6fa0 0x561d0eaa73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.098+0000 7f6afe463800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d0e8894a0 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f6afc755640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eaa6fa0 0x561d0eaa73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f6afc755640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eaa6fa0 0x561d0eaa73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34218/0 (socket says 172.21.15.28:34218) 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f6afc755640 1 -- 172.21.15.28:0/1868028281 learned_addr learned my addr 172.21.15.28:0/1868028281 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f6afc755640 1 -- 172.21.15.28:0/1868028281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0eaa7a00 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f6afc755640 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eaa6fa0 0x561d0eaa73c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x561d0ea8d600 tx=0x7f6af400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f69e17fa640 1 -- 172.21.15.28:0/1868028281 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af401c020 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f69e17fa640 1 -- 172.21.15.28:0/1868028281 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af4007c90 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f69e17fa640 1 -- 172.21.15.28:0/1868028281 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af4018440 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d0eaa8230 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.099+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0eaa87d0 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.100+0000 7f69e17fa640 1 -- 172.21.15.28:0/1868028281 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6af401f020 con 0x561d0eaa6fa0 2024-09-19T11:23:26.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.100+0000 7f69e17fa640 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69d403d000 0x7f69d403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f69e17fa640 1 -- 172.21.15.28:0/1868028281 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f6af4055a30 con 0x561d0eaa6fa0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afbf54640 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69d403d000 0x7f69d403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afe463800 1 Processor -- start 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afbf54640 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69d403d000 0x7f69d403f4c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x561d0ea989b0 tx=0x7f6aec046130 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afe463800 1 -- start start 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afe463800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eab3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afe463800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d0ea84f70 con 0x561d0eab3ac0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afc755640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eab3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afc755640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eab3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34228/0 (socket says 172.21.15.28:34228) 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afc755640 1 -- 172.21.15.28:0/2614415398 learned_addr learned my addr 172.21.15.28:0/2614415398 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.101+0000 7f6afc755640 1 -- 172.21.15.28:0/2614415398 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0eab44b0 con 0x561d0eab3ac0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afc755640 1 --2- 172.21.15.28:0/2614415398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eab3ec0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6af4053d80 tx=0x7f6af4054050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781f038a97c8a8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f69daffd640 1 -- 172.21.15.28:0/2614415398 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af4052070 con 0x561d0eab3ac0 2024-09-19T11:23:26.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f69daffd640 1 -- 172.21.15.28:0/2614415398 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af40156f0 con 0x561d0eab3ac0 2024-09-19T11:23:26.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f69daffd640 1 -- 172.21.15.28:0/2614415398 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af4016c50 con 0x561d0eab3ac0 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 -- 172.21.15.28:0/2614415398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 msgr2=0x561d0eab3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 --2- 172.21.15.28:0/2614415398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eab3ec0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6af4053d80 tx=0x7f6af4054050 comp rx=0 tx=0).stop 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 -- 172.21.15.28:0/2614415398 shutdown_connections 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 --2- 172.21.15.28:0/2614415398 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eab3ec0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 -- 172.21.15.28:0/2614415398 >> 172.21.15.28:0/2614415398 conn(0x561d0eaafec0 msgr2=0x561d0eaae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 -- 172.21.15.28:0/2614415398 shutdown_connections 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 -- 172.21.15.28:0/2614415398 wait complete. 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.102+0000 7f6afe463800 1 Processor -- start 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afe463800 1 -- start start 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afe463800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eabb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afe463800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561d0eab4aa0 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afc755640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eabb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afc755640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eabb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34244/0 (socket says 172.21.15.28:34244) 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 learned_addr learned my addr 172.21.15.28:0/3390815520 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0eabb560 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afc755640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eabb000 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6af4017f70 tx=0x7f6af4053d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f69d97fa640 1 -- 172.21.15.28:0/3390815520 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af4013530 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f69d97fa640 1 -- 172.21.15.28:0/3390815520 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af402e030 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d0eabb7f0 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f69d97fa640 1 -- 172.21.15.28:0/3390815520 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6af40139e0 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.103+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0eabc380 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.104+0000 7f69d97fa640 1 -- 172.21.15.28:0/3390815520 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6af4016040 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.104+0000 7f69d97fa640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69c403d000 0x7f69c403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.104+0000 7f69d97fa640 1 -- 172.21.15.28:0/3390815520 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f6af4033b30 con 0x561d0eab3ac0 2024-09-19T11:23:26.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.105+0000 7f6afbf54640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69c403d000 0x7f69c403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.105+0000 7f6afbf54640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69c403d000 0x7f69c403f4c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6aec04e5f0 tx=0x7f6aec08b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.105+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.105+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0eac69e0 0x561d0eac8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.105+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eac95f0 con 0x561d0eac69e0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.105+0000 7f6afb753640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0eac69e0 0x561d0eac8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afb753640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0eac69e0 0x561d0eac8ea0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6af000f610 con 0x561d0eac69e0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 20 realm 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eac5d30 con 0x561d0eac69e0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6af000f610 con 0x561d0eac69e0 2024-09-19T11:23:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eaca0e0 con 0x561d0eac69e0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6af000f610 con 0x561d0eac69e0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.106+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eaca9b0 con 0x561d0eac69e0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6af000f610 con 0x561d0eac69e0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afe463800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0eacb3d0 0x561d0eacd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacdf00 con 0x561d0eacb3d0 2024-09-19T11:23:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afc755640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0eacb3d0 0x561d0eacd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afc755640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0eacb3d0 0x561d0eacd7c0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afe463800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afe463800 20 searching for the correct realm 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.107+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afc755640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6af404e070 0x7f6af4052070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6af404e070 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afbf54640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6af404e070 0x7f6af4052070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afbf54640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6af404e070 0x7f6af4052070 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afbf54640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6aec0a5040 0x7f6aec098290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.108+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.109+0000 7f6afb753640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6aec0a5040 0x7f6aec098290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.109+0000 7f6afb753640 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6aec0a5040 0x7f6aec098290 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.109+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af000f610 con 0x7f6aec0a5040 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.109+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.109+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af000f610 con 0x561d0eac69e0 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.109+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x561d0eac69e0 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6af404e070 2024-09-19T11:23:26.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.110+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x561d0eac69e0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x561d0eac69e0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.111+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6af404e070 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.112+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x561d0eac69e0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6af002b040 con 0x561d0eac69e0 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.113+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6af404e070 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af002b040 con 0x561d0eac69e0 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.114+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6af404e070 2024-09-19T11:23:26.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.115+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eacb3d0 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6aec0a5040 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x561d0eac69e0 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf830 con 0x7f6af404e070 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:26.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.116+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eace8c0 con 0x561d0eac69e0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacf110 con 0x561d0eac69e0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eacfcb0 con 0x561d0eacb3d0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 zone default found 2024-09-19T11:23:26.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 4 Realm: () 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 zonegroup default 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead0090 con 0x7f6aec0a5040 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.117+0000 7f6afe463800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead1ec0 con 0x561d0eacb3d0 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead22e0 con 0x561d0eacb3d0 2024-09-19T11:23:26.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 20 started sync module instance, tier type = 2024-09-19T11:23:26.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:26.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.118+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead1360 con 0x7f6af404e070 2024-09-19T11:23:26.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.119+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.119+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94682800262832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead2740 con 0x7f6af404e070 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.119+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead2f90 con 0x561d0eac69e0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.120+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94682800262832] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.120+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.120+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94682800269120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead3010 con 0x561d0eac69e0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.120+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead38a0 con 0x561d0eacb3d0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.121+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.121+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94682800271376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead3490 con 0x561d0eacb3d0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.121+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead4840 con 0x7f6af404e070 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.122+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94682800269120] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.122+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94682800271376] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.123+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.123+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94682800276240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead4800 con 0x7f6af404e070 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.123+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead50e0 con 0x561d0eac69e0 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.124+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.124+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94682800278576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead5500 con 0x561d0eac69e0 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.124+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94682800276240] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.124+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead2b60 con 0x7f6aec0a5040 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.125+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94682800278576] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.125+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.125+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94682800279632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead5920 con 0x7f6aec0a5040 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.125+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead61f0 con 0x7f6aec0a5040 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.126+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94682800279632] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.126+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.126+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94682800280688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead63d0 con 0x7f6aec0a5040 2024-09-19T11:23:26.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.126+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead6c20 con 0x7f6aec0a5040 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94682800280688] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94682800285104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead6fc0 con 0x7f6aec0a5040 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 20 add_watcher() i=0 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 20 add_watcher() i=1 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 20 add_watcher() i=2 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 20 add_watcher() i=3 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 20 add_watcher() i=4 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 20 add_watcher() i=5 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.127+0000 7f6afe463800 20 add_watcher() i=6 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.128+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94682800285104] v31'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.128+0000 7f6afe463800 20 add_watcher() i=7 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.128+0000 7f6afe463800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:26.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.128+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead58d0 con 0x561d0eac69e0 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.129+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.129+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94682800286288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0ead5890 con 0x561d0eac69e0 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94682800286288] v31'110 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f69b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f6afe463800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f6afe463800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f6afe463800 5 note: GC not initialized 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f69b0fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f6afe463800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f6afe463800 20 Filter name: none 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f692a7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.130+0000 7f6afe463800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:23:26.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0cec0 con 0x7f6aec0a5040 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ee50 con 0x7f6af404e070 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv18 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.131+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb10930 con 0x7f6aec0a5040 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(64 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.132+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb14ed0 con 0x7f6af404e070 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.133+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(65 foo.buckets [call] v31'19 uv17 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.133+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~387 in=387b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb1aca0 con 0x7f6aec0a5040 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(66 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [delete,create,call,call,writefull 0~387,setxattr (123)] v31'3 uv3 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afe463800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afe463800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afe463800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afe463800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94682800525264 in=772b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb15300 con 0x7f6aec0a5040 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== watch-notify(notify (1) cookie 94682800279632 notify 133143986180 ret 0) ==== 802+0+0 (crc 0 0 0) 0x561d0eb10d20 con 0x7f6aec0a5040 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f69daffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986180 cookie 94682800279632 notifier 4502 bl.length()=760 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f69daffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f69daffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f69daffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(67 notify.5 [notify cookie 94682800525264 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.134+0000 7f69daffd640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f69c0003ea0 con 0x7f6aec0a5040 2024-09-19T11:23:26.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.135+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.135+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== watch-notify(notify_complete (2) cookie 94682800525264 notify 133143986180 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561d0eb0fe70 con 0x7f6aec0a5040 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.135+0000 7f6afe463800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_sources= new_sources= 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.135+0000 7f6afe463800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential sources added=[] removed=[] 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.135+0000 7f6afe463800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_dests= new_dests= 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.135+0000 7f6afe463800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential dests added=[] removed=[] 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.135+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb10fb0 con 0x7f6aec0a5040 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.136+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(69 fud.buckets [call] v31'1 uv1 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.136+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb10fb0 con 0x7f6af404e070 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.137+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v31'20 uv20 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.137+0000 7f6afe463800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.137+0000 7f6afe463800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.137+0000 7f6afe463800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:23:26.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.137+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94682800525232 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb19aa0 con 0x7f6aec0a5040 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.137+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== watch-notify(notify (1) cookie 94682800280688 notify 133143986181 ret 0) ==== 356+0+0 (crc 0 0 0) 0x7f6af0018040 con 0x7f6aec0a5040 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.137+0000 7f69da7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986181 cookie 94682800280688 notifier 4502 bl.length()=314 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.138+0000 7f69da7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.138+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(71 notify.6 [notify cookie 94682800525232 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.138+0000 7f69da7fc640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.138+0000 7f69da7fc640 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f6930003400 con 0x7f6aec0a5040 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.138+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== watch-notify(notify_complete (2) cookie 94682800525232 notify 133143986181 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f6af0009050 con 0x7f6aec0a5040 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.138+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.139+0000 7f69b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.139+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94682800286288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x561d0eac69e0 2024-09-19T11:23:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.140+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94682800286288] v31'111 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.140+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94682800262832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x7f6af404e070 2024-09-19T11:23:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.141+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94682800262832] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.141+0000 7f6afe463800 20 remove_watcher() i=0 2024-09-19T11:23:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.141+0000 7f6afe463800 2 removed watcher, disabling cache 2024-09-19T11:23:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.141+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94682800269120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x561d0eac69e0 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.142+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94682800269120] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.142+0000 7f6afe463800 20 remove_watcher() i=1 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.142+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94682800271376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x561d0eacb3d0 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.143+0000 7f6afc755640 1 -- 172.21.15.28:0/3390815520 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94682800271376] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af403a070 con 0x561d0eacb3d0 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.143+0000 7f6afe463800 20 remove_watcher() i=2 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.143+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94682800276240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x7f6af404e070 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.144+0000 7f6afbf54640 1 -- 172.21.15.28:0/3390815520 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94682800276240] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6aec07c070 con 0x7f6af404e070 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.144+0000 7f6afe463800 20 remove_watcher() i=3 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.144+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94682800278576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x561d0eac69e0 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.145+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94682800278576] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x561d0eac69e0 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.145+0000 7f6afe463800 20 remove_watcher() i=4 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.145+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94682800279632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x7f6aec0a5040 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.146+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94682800279632] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.146+0000 7f6afe463800 20 remove_watcher() i=5 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.146+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94682800280688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x7f6aec0a5040 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.147+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94682800280688] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.147+0000 7f6afe463800 20 remove_watcher() i=6 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.147+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94682800285104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561d0eb0ad20 con 0x7f6aec0a5040 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94682800285104] v31'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6af003b040 con 0x7f6aec0a5040 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 20 remove_watcher() i=7 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6aec0a5040 msgr2=0x7f6aec098290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6aec0a5040 0x7f6aec098290 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0eac69e0 msgr2=0x561d0eac8ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0eac69e0 0x561d0eac8ea0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6af404e070 msgr2=0x7f6af4052070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6af404e070 0x7f6af4052070 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0eacb3d0 msgr2=0x561d0eacd7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0eacb3d0 0x561d0eacd7c0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69c403d000 msgr2=0x7f69c403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69c403d000 0x7f69c403f4c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6aec04e5f0 tx=0x7f6aec08b000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 msgr2=0x561d0eabb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afe463800 1 --2- 172.21.15.28:0/3390815520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eab3ac0 0x561d0eabb000 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6af4017f70 tx=0x7f6af4053d10 comp rx=0 tx=0).stop 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 reap_dead start 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.148+0000 7f6afb753640 1 -- 172.21.15.28:0/3390815520 reap_dead start 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 shutdown_connections 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 >> 172.21.15.28:0/3390815520 conn(0x561d0eaafec0 msgr2=0x561d0eab5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 shutdown_connections 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/3390815520 wait complete. 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69d403d000 msgr2=0x7f69d403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69d403d000 0x7f69d403f4c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x561d0ea989b0 tx=0x7f6aec046130 comp rx=0 tx=0).stop 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eaa6fa0 msgr2=0x561d0eaa73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eaa6fa0 0x561d0eaa73c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x561d0ea8d600 tx=0x7f6af400bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 shutdown_connections 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69d403d000 0x7f69d403f4c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/1868028281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0eaa6fa0 0x561d0eaa73c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 >> 172.21.15.28:0/1868028281 conn(0x561d0ea9b350 msgr2=0x561d0ea9b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 shutdown_connections 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/1868028281 wait complete. 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0e991680 msgr2=0x561d0ea931d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0e991680 0x561d0ea931d0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0ea959e0 msgr2=0x561d0ea97e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0ea959e0 0x561d0ea97e80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69e403cfb0 msgr2=0x7f69e403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69e403cfb0 0x7f69e403f470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6af0001e30 tx=0x7f6af0001390 comp rx=0 tx=0).stop 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 msgr2=0x561d0ea8c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.149+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea8c090 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6aec045d30 tx=0x7f6aec003330 comp rx=0 tx=0).stop 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 shutdown_connections 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561d0ea959e0 0x561d0ea97e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f69e403cfb0 0x7f69e403f470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561d0e991680 0x561d0ea931d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 --2- 172.21.15.28:0/671914713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561d0e990d40 0x561d0ea8c090 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 >> 172.21.15.28:0/671914713 conn(0x561d0e98ca70 msgr2=0x561d0ea85a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 shutdown_connections 2024-09-19T11:23:26.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.150+0000 7f6afe463800 1 -- 172.21.15.28:0/671914713 wait complete. 2024-09-19T11:23:26.160 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-09-19T11:23:26.160 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-09-19T11:23:26.160 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-09-19T11:23:26.206 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:26.206 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:26.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.205+0000 7f38f290c800 1 Processor -- start 2024-09-19T11:23:26.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38f290c800 1 -- start start 2024-09-19T11:23:26.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38f290c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f87c04d840 0x55f87c056360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38f290c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f87c0568a0 con 0x55f87c04bd20 2024-09-19T11:23:26.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38f290c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55f87bed87d0 con 0x55f87c04d840 2024-09-19T11:23:26.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38efbf5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f87c04d840 0x55f87c056360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38f03f6640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55f87c04bd20 0x55f87c04c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48322/0 (socket says 172.21.15.28:48322) 2024-09-19T11:23:26.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38efbf5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f87c04d840 0x55f87c056360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34248/0 (socket says 172.21.15.28:34248) 2024-09-19T11:23:26.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38f03f6640 1 -- 172.21.15.28:0/2090353442 learned_addr learned my addr 172.21.15.28:0/2090353442 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:26.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38efbf5640 1 -- 172.21.15.28:0/2090353442 >> v1:172.21.15.22:6789/0 conn(0x55f87c04bd20 legacy=0x55f87c04c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.206+0000 7f38efbf5640 1 -- 172.21.15.28:0/2090353442 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c057a80 con 0x55f87c04d840 2024-09-19T11:23:26.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.208+0000 7f38efbf5640 1 --2- 172.21.15.28:0/2090353442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04d840 0x55f87c056360 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f38e400a470 tx=0x7f38e4045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f8f49941224a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.208+0000 7f38ef3f4640 1 -- 172.21.15.28:0/2090353442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e4053570 con 0x55f87c04d840 2024-09-19T11:23:26.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.208+0000 7f38ef3f4640 1 -- 172.21.15.28:0/2090353442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e4058070 con 0x55f87c04d840 2024-09-19T11:23:26.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.208+0000 7f38ef3f4640 1 -- 172.21.15.28:0/2090353442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e4053880 con 0x55f87c04d840 2024-09-19T11:23:26.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.209+0000 7f38f290c800 1 -- 172.21.15.28:0/2090353442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04d840 msgr2=0x55f87c056360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.209+0000 7f38f290c800 1 --2- 172.21.15.28:0/2090353442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04d840 0x55f87c056360 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f38e400a470 tx=0x7f38e4045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:26.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.209+0000 7f38f290c800 1 -- 172.21.15.28:0/2090353442 shutdown_connections 2024-09-19T11:23:26.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.209+0000 7f38f290c800 1 --2- 172.21.15.28:0/2090353442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04d840 0x55f87c056360 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.209+0000 7f38f290c800 1 -- 172.21.15.28:0/2090353442 >> 172.21.15.28:0/2090353442 conn(0x55f87c047a50 msgr2=0x55f87c049eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.209+0000 7f38f290c800 1 -- 172.21.15.28:0/2090353442 shutdown_connections 2024-09-19T11:23:26.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.209+0000 7f38f290c800 1 -- 172.21.15.28:0/2090353442 wait complete. 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.223+0000 7f38f290c800 1 Processor -- start 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f290c800 1 -- start start 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f290c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c13f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f290c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f87bee5dd0 con 0x55f87c04bd20 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f03f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c13f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f03f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c13f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34258/0 (socket says 172.21.15.28:34258) 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f03f6640 1 -- 172.21.15.28:0/3195141024 learned_addr learned my addr 172.21.15.28:0/3195141024 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f03f6640 1 -- 172.21.15.28:0/3195141024 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c13fad0 con 0x55f87c04bd20 2024-09-19T11:23:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.224+0000 7f38f03f6640 1 --2- 172.21.15.28:0/3195141024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c13f4e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001df0 tx=0x7f38e000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b621f43405d022a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f37de60c640 1 -- 172.21.15.28:0/3195141024 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e0012020 con 0x55f87c04bd20 2024-09-19T11:23:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f37de60c640 1 -- 172.21.15.28:0/3195141024 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e000e470 con 0x55f87c04bd20 2024-09-19T11:23:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f37de60c640 1 -- 172.21.15.28:0/3195141024 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e000e7f0 con 0x55f87c04bd20 2024-09-19T11:23:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f38f290c800 1 -- 172.21.15.28:0/3195141024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 msgr2=0x55f87c13f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f38f290c800 1 --2- 172.21.15.28:0/3195141024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c13f4e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001df0 tx=0x7f38e000b6c0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f38f290c800 1 -- 172.21.15.28:0/3195141024 shutdown_connections 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f38f290c800 1 --2- 172.21.15.28:0/3195141024 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c13f4e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f38f290c800 1 -- 172.21.15.28:0/3195141024 >> 172.21.15.28:0/3195141024 conn(0x55f87c047a50 msgr2=0x55f87c13ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f38f290c800 1 -- 172.21.15.28:0/3195141024 shutdown_connections 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.225+0000 7f38f290c800 1 -- 172.21.15.28:0/3195141024 wait complete. 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f290c800 1 Processor -- start 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f290c800 1 -- start start 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f290c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f290c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f87c056d40 con 0x55f87c04bd20 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f03f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f03f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34262/0 (socket says 172.21.15.28:34262) 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f03f6640 1 -- 172.21.15.28:0/2124371921 learned_addr learned my addr 172.21.15.28:0/2124371921 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f03f6640 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c147520 con 0x55f87c04bd20 2024-09-19T11:23:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.226+0000 7f38f03f6640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c146fc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55f87c140470 tx=0x7f38e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.227+0000 7f37dce09640 1 -- 172.21.15.28:0/2124371921 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e0012030 con 0x55f87c04bd20 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.227+0000 7f37dce09640 1 -- 172.21.15.28:0/2124371921 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e001a6d0 con 0x55f87c04bd20 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.227+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f87c1477b0 con 0x55f87c04bd20 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.227+0000 7f37dce09640 1 -- 172.21.15.28:0/2124371921 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e001a9d0 con 0x55f87c04bd20 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.227+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87c1481b0 con 0x55f87c04bd20 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f37dce09640 1 -- 172.21.15.28:0/2124371921 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f38e0015020 con 0x55f87c04bd20 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f37dce09640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37d803cfb0 0x7f37d803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f37dce09640 1 -- 172.21.15.28:0/2124371921 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f38e0020030 con 0x55f87c04bd20 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f38efbf5640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37d803cfb0 0x7f37d803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c04d840 0x55f87c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c14eb50 con 0x55f87c04d840 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c04d840 0x55f87c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.228+0000 7f38efbf5640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37d803cfb0 0x7f37d803f470 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38e4057000 tx=0x7f38e4052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.229+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c04d840 0x55f87c14e610 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.229+0000 7f38f0bf7640 1 -- 172.21.15.28:0/2124371921 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38e800a4d0 con 0x55f87c04d840 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.229+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c14eef0 con 0x55f87c04d840 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.229+0000 7f38f0bf7640 1 -- 172.21.15.28:0/2124371921 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f38e800a4d0 con 0x55f87c04d840 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.229+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c150a30 0x55f87c152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.229+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c153630 con 0x55f87c150a30 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.229+0000 7f38f03f6640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c150a30 0x55f87c152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.230+0000 7f38f03f6640 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c150a30 0x55f87c152ea0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.230+0000 7f38f03f6640 1 -- 172.21.15.28:0/2124371921 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f38e001ad50 con 0x55f87c150a30 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.230+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c14f740 con 0x55f87c04d840 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.230+0000 7f38f0bf7640 1 -- 172.21.15.28:0/2124371921 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f38e800a4d0 con 0x55f87c04d840 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.230+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c14fb60 con 0x55f87c150a30 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f03f6640 1 -- 172.21.15.28:0/2124371921 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f38e001ad50 con 0x55f87c150a30 2024-09-19T11:23:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f290c800 1 Processor -- start 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f290c800 1 -- start start 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f290c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c161f80 0x55f87c1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f290c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f87bf444a0 con 0x55f87c161f80 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f0bf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c161f80 0x55f87c1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f0bf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c161f80 0x55f87c1623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34268/0 (socket says 172.21.15.28:34268) 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.231+0000 7f38f0bf7640 1 -- 172.21.15.28:0/634913733 learned_addr learned my addr 172.21.15.28:0/634913733 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.232+0000 7f38f0bf7640 1 -- 172.21.15.28:0/634913733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c1629e0 con 0x55f87c161f80 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.232+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c161f80 0x55f87c1623a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55f87c148380 tx=0x7f38e80167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.232+0000 7f37d5ffb640 1 -- 172.21.15.28:0/634913733 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8013640 con 0x55f87c161f80 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.232+0000 7f37d5ffb640 1 -- 172.21.15.28:0/634913733 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e80137e0 con 0x55f87c161f80 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.232+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f87c163210 con 0x55f87c161f80 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.232+0000 7f37d5ffb640 1 -- 172.21.15.28:0/634913733 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8013ae0 con 0x55f87c161f80 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.232+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87c1637b0 con 0x55f87c161f80 2024-09-19T11:23:26.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.233+0000 7f37d5ffb640 1 -- 172.21.15.28:0/634913733 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f38e8011040 con 0x55f87c161f80 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.233+0000 7f37d5ffb640 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c803d000 0x7f37c803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.233+0000 7f37d5ffb640 1 -- 172.21.15.28:0/634913733 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f38e800e040 con 0x55f87c161f80 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.233+0000 7f38f03f6640 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c803d000 0x7f37c803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.233+0000 7f38f290c800 1 Processor -- start 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f290c800 1 -- start start 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f03f6640 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c803d000 0x7f37c803f4c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f38e0052a50 tx=0x7f38e0041040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f290c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c16ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f290c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f87c13ff00 con 0x55f87c16ea40 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f0bf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c16ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f0bf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c16ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34272/0 (socket says 172.21.15.28:34272) 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f0bf7640 1 -- 172.21.15.28:0/2075858880 learned_addr learned my addr 172.21.15.28:0/2075858880 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f0bf7640 1 -- 172.21.15.28:0/2075858880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c16f430 con 0x55f87c16ea40 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/2075858880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c16ee40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f38e8054860 tx=0x7f38e801f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bceb0ec5e4846635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f37cf7fe640 1 -- 172.21.15.28:0/2075858880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8044030 con 0x55f87c16ea40 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f37cf7fe640 1 -- 172.21.15.28:0/2075858880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e8021e20 con 0x55f87c16ea40 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.234+0000 7f37cf7fe640 1 -- 172.21.15.28:0/2075858880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e804de60 con 0x55f87c16ea40 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 -- 172.21.15.28:0/2075858880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 msgr2=0x55f87c16ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 --2- 172.21.15.28:0/2075858880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c16ee40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f38e8054860 tx=0x7f38e801f400 comp rx=0 tx=0).stop 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 -- 172.21.15.28:0/2075858880 shutdown_connections 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 --2- 172.21.15.28:0/2075858880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c16ee40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 -- 172.21.15.28:0/2075858880 >> 172.21.15.28:0/2075858880 conn(0x55f87c16ae60 msgr2=0x55f87c1690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 -- 172.21.15.28:0/2075858880 shutdown_connections 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 -- 172.21.15.28:0/2075858880 wait complete. 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 Processor -- start 2024-09-19T11:23:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 -- start start 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c176020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.235+0000 7f38f290c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f87c16fa50 con 0x55f87c16ea40 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f38f0bf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c176020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f38f0bf7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c176020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34282/0 (socket says 172.21.15.28:34282) 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 learned_addr learned my addr 172.21.15.28:0/3787326081 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87c176580 con 0x55f87c16ea40 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c176020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f38e8016cd0 tx=0x7f38e80547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f37cdffb640 1 -- 172.21.15.28:0/3787326081 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e80143c0 con 0x55f87c16ea40 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f37cdffb640 1 -- 172.21.15.28:0/3787326081 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e8014560 con 0x55f87c16ea40 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f87c176810 con 0x55f87c16ea40 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f37cdffb640 1 -- 172.21.15.28:0/3787326081 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8020d40 con 0x55f87c16ea40 2024-09-19T11:23:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.236+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87c1773a0 con 0x55f87c16ea40 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.237+0000 7f37cdffb640 1 -- 172.21.15.28:0/3787326081 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f38e801fa90 con 0x55f87c16ea40 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.237+0000 7f37cdffb640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c003d000 0x7f37c003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.237+0000 7f37cdffb640 1 -- 172.21.15.28:0/3787326081 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f38e804c030 con 0x55f87c16ea40 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.237+0000 7f38f03f6640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c003d000 0x7f37c003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.238+0000 7f38f03f6640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c003d000 0x7f37c003f4c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f38e0054010 tx=0x7f38e0054420 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.238+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.238+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c181940 0x55f87c183e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.238+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c184550 con 0x55f87c181940 2024-09-19T11:23:26.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.238+0000 7f38efbf5640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c181940 0x55f87c183e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.238+0000 7f38efbf5640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c181940 0x55f87c183e00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 realm 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c184980 con 0x55f87c181940 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c185250 con 0x55f87c181940 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c1855f0 con 0x55f87c181940 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c186350 0x55f87c188740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c188d90 con 0x55f87c186350 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.239+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c186350 0x55f87c188740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c186350 0x55f87c188740 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f290c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f290c800 20 searching for the correct realm 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f0bf7640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f38e8031b80 0x7f38e80127c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e8031b80 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.240+0000 7f38f03f6640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f38e8031b80 0x7f38e80127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.241+0000 7f38f03f6640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f38e8031b80 0x7f38e80127c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.241+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.241+0000 7f38f03f6640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f38e0017050 0x7f38e004b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.241+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.241+0000 7f38efbf5640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f38e0017050 0x7f38e004b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.241+0000 7f38efbf5640 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f38e0017050 0x7f38e004b950 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.242+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e8031b80 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.243+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e8031b80 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.244+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.245+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.246+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e8031b80 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.247+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.247+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.247+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.247+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.247+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.247+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e8031b80 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c186350 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.248+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e0017050 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x55f87c181940 2024-09-19T11:23:26.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18a6c0 con 0x7f38e8031b80 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c189750 con 0x55f87c181940 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.249+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c189fa0 con 0x55f87c181940 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18ab40 con 0x55f87c186350 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 zone default found 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 4 Realm: () 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 zonegroup default 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18aee0 con 0x7f38e0017050 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.250+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18d190 con 0x55f87c186350 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18d530 con 0x55f87c186350 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f290c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f290c800 20 started sync module instance, tier type = 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f290c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.251+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18c620 con 0x7f38e8031b80 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.252+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.252+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94525722249792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18c280 con 0x7f38e8031b80 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.252+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18e460 con 0x55f87c181940 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.253+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94525722249792] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.253+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.253+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94525722255280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18e880 con 0x55f87c181940 2024-09-19T11:23:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.253+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18f240 con 0x55f87c186350 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.254+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.254+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94525722258832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18f6a0 con 0x55f87c186350 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.254+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c1901c0 con 0x7f38e8031b80 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.255+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94525722255280] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.255+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94525722258832] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.256+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.256+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94525722262672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c190180 con 0x7f38e8031b80 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.256+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c190a60 con 0x55f87c181940 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.257+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.257+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94525722265008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c190e80 con 0x55f87c181940 2024-09-19T11:23:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.257+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c1916d0 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.257+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94525722262672] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.258+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.258+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94525722267264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c191ac0 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.258+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c192310 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.258+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94525722265008] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.259+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.259+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94525722269888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c192190 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.259+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c192b50 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.259+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94525722267264] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94525722268320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c192b50 con 0x7f38e0017050 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=0 2024-09-19T11:23:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=1 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=2 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=3 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=4 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=5 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94525722269888] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=6 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94525722268320] v31'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 20 add_watcher() i=7 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.260+0000 7f38f290c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.261+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c195b80 con 0x55f87c181940 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.261+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.261+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94525722273440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c192f70 con 0x55f87c181940 2024-09-19T11:23:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.262+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94525722273440] v31'112 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.262+0000 7f37a67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.262+0000 7f38f290c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.262+0000 7f38f290c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.262+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19e300 con 0x55f87c181940 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.263+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.263+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19e300 con 0x7f38e0017050 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.264+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.264+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19ed20 con 0x55f87c186350 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.265+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.265+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19ed20 con 0x55f87c181940 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.266+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'113 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.266+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19ed20 con 0x7f38e8031b80 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.267+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'32 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.267+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f390 con 0x55f87c186350 2024-09-19T11:23:26.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.268+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'21 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.268+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f300 con 0x55f87c181940 2024-09-19T11:23:26.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.269+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.269+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f300 con 0x7f38e0017050 2024-09-19T11:23:26.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.270+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'21 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.271+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f390 con 0x7f38e8031b80 2024-09-19T11:23:26.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.271+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.271+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.272+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'20 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.272+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x7f38e8031b80 2024-09-19T11:23:26.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.274+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'33 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.274+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x7f38e0017050 2024-09-19T11:23:26.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.275+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.275+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x7f38e8031b80 2024-09-19T11:23:26.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.276+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.276+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x7f38e0017050 2024-09-19T11:23:26.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.277+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'28 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.277+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c181940 2024-09-19T11:23:26.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.278+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'114 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.278+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.279+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'26 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.279+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x7f38e8031b80 2024-09-19T11:23:26.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.280+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'34 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.280+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.281+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.281+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c181940 2024-09-19T11:23:26.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.282+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.282+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c181940 2024-09-19T11:23:26.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.283+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'115 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.283+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.284+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'27 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.284+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x7f38e8031b80 2024-09-19T11:23:26.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.285+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.285+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.286+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.286+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.287+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.287+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.288+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.288+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f840 con 0x55f87c186350 2024-09-19T11:23:26.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.289+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'22 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.289+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f880 con 0x7f38e0017050 2024-09-19T11:23:26.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.291+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.291+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f8c0 con 0x55f87c186350 2024-09-19T11:23:26.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.291+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.292+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f8c0 con 0x7f38e0017050 2024-09-19T11:23:26.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.293+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'22 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.293+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f8c0 con 0x55f87c186350 2024-09-19T11:23:26.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.294+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'28 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.294+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f8c0 con 0x7f38e0017050 2024-09-19T11:23:26.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.295+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'29 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.295+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c19f900 con 0x7f38e8031b80 2024-09-19T11:23:26.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.296+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.296+0000 7f37a57e2640 20 reqs_thread_entry: start 2024-09-19T11:23:26.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:26.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 20 Filter name: none 2024-09-19T11:23:26.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f3716ffd640 20 reqs_thread_entry: start 2024-09-19T11:23:26.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:23:26.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c1ca480 con 0x7f38e0017050 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.297+0000 7f38f290c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.298+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55f87c1ccdb0 con 0x7f38e0017050 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.298+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(95 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-09-19T11:23:26.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.299+0000 7f37a67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.299+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94525722273440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c17e040 con 0x55f87c181940 2024-09-19T11:23:26.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.300+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94525722273440] v31'116 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.300+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94525722249792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c17e040 con 0x7f38e8031b80 2024-09-19T11:23:26.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.301+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94525722249792] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.301+0000 7f38f290c800 20 remove_watcher() i=0 2024-09-19T11:23:26.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.301+0000 7f38f290c800 2 removed watcher, disabling cache 2024-09-19T11:23:26.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.301+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94525722255280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18ca40 con 0x55f87c181940 2024-09-19T11:23:26.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.302+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94525722255280] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.302+0000 7f38f290c800 20 remove_watcher() i=1 2024-09-19T11:23:26.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.302+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94525722258832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18ca40 con 0x55f87c186350 2024-09-19T11:23:26.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.303+0000 7f38f0bf7640 1 -- 172.21.15.28:0/3787326081 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94525722258832] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e803a070 con 0x55f87c186350 2024-09-19T11:23:26.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.303+0000 7f38f290c800 20 remove_watcher() i=2 2024-09-19T11:23:26.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.303+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94525722262672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18ed90 con 0x7f38e8031b80 2024-09-19T11:23:26.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.304+0000 7f38f03f6640 1 -- 172.21.15.28:0/3787326081 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94525722262672] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e001ad50 con 0x7f38e8031b80 2024-09-19T11:23:26.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.304+0000 7f38f290c800 20 remove_watcher() i=3 2024-09-19T11:23:26.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.304+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94525722265008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18ca40 con 0x55f87c181940 2024-09-19T11:23:26.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.305+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94525722265008] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x55f87c181940 2024-09-19T11:23:26.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.305+0000 7f38f290c800 20 remove_watcher() i=4 2024-09-19T11:23:26.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.305+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94525722267264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18fc90 con 0x7f38e0017050 2024-09-19T11:23:26.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.306+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94525722267264] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.306+0000 7f38f290c800 20 remove_watcher() i=5 2024-09-19T11:23:26.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.306+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94525722269888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c18ed90 con 0x7f38e0017050 2024-09-19T11:23:26.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.307+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94525722269888] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.307+0000 7f38f290c800 20 remove_watcher() i=6 2024-09-19T11:23:26.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.307+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94525722268320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55f87c1905b0 con 0x7f38e0017050 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94525722268320] v31'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38e4051070 con 0x7f38e0017050 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 20 remove_watcher() i=7 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f38e0017050 msgr2=0x7f38e004b950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f38e0017050 0x7f38e004b950 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c181940 msgr2=0x55f87c183e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c181940 0x55f87c183e00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f38e8031b80 msgr2=0x7f38e80127c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f38e8031b80 0x7f38e80127c0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c186350 msgr2=0x55f87c188740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.308+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c186350 0x55f87c188740 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c003d000 msgr2=0x7f37c003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c003d000 0x7f37c003f4c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f38e0054010 tx=0x7f38e0054420 comp rx=0 tx=0).stop 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 msgr2=0x55f87c176020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 --2- 172.21.15.28:0/3787326081 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c16ea40 0x55f87c176020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f38e8016cd0 tx=0x7f38e80547c0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 reap_dead start 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38efbf5640 1 -- 172.21.15.28:0/3787326081 reap_dead start 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 shutdown_connections 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 >> 172.21.15.28:0/3787326081 conn(0x55f87c16ae60 msgr2=0x55f87c169070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 shutdown_connections 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/3787326081 wait complete. 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c803d000 msgr2=0x7f37c803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c803d000 0x7f37c803f4c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f38e0052a50 tx=0x7f38e0041040 comp rx=0 tx=0).stop 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c161f80 msgr2=0x55f87c1623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c161f80 0x55f87c1623a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55f87c148380 tx=0x7f38e80167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 shutdown_connections 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37c803d000 0x7f37c803f4c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 --2- 172.21.15.28:0/634913733 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c161f80 0x55f87c1623a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 >> 172.21.15.28:0/634913733 conn(0x55f87c156330 msgr2=0x55f87c156710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 shutdown_connections 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.309+0000 7f38f290c800 1 -- 172.21.15.28:0/634913733 wait complete. 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c04d840 msgr2=0x55f87c14e610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c04d840 0x55f87c14e610 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c150a30 msgr2=0x55f87c152ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c150a30 0x55f87c152ea0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37d803cfb0 msgr2=0x7f37d803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37d803cfb0 0x7f37d803f470 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38e4057000 tx=0x7f38e4052000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 msgr2=0x55f87c146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c146fc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55f87c140470 tx=0x7f38e0000bc0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 shutdown_connections 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f87c150a30 0x55f87c152ea0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f37d803cfb0 0x7f37d803f470 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f87c04d840 0x55f87c14e610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 --2- 172.21.15.28:0/2124371921 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f87c04bd20 0x55f87c146fc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 >> 172.21.15.28:0/2124371921 conn(0x55f87c047a50 msgr2=0x55f87c1409d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 shutdown_connections 2024-09-19T11:23:26.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.310+0000 7f38f290c800 1 -- 172.21.15.28:0/2124371921 wait complete. 2024-09-19T11:23:26.320 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-19T11:23:26.320 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-09-19T11:23:26.320 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-09-19T11:23:26.321 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-09-19T11:23:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b7a130800 1 Processor -- start 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b7a130800 1 -- start start 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b7a130800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562619982ab0 0x562619984f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b7a130800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5626199855f0 con 0x56261997eeb0 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b7a130800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5626198097d0 con 0x562619982ab0 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b755bb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562619982ab0 0x562619984f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b75dbc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x56261997eeb0 0x5626199812a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48326/0 (socket says 172.21.15.28:48326) 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b755bb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562619982ab0 0x562619984f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34286/0 (socket says 172.21.15.28:34286) 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.368+0000 7f5b75dbc640 1 -- 172.21.15.28:0/72674811 learned_addr learned my addr 172.21.15.28:0/72674811 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.369+0000 7f5b755bb640 1 -- 172.21.15.28:0/72674811 >> v1:172.21.15.22:6789/0 conn(0x56261997eeb0 legacy=0x5626199812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.369+0000 7f5b755bb640 1 -- 172.21.15.28:0/72674811 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562619986820 con 0x562619982ab0 2024-09-19T11:23:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.370+0000 7f5b755bb640 1 --2- 172.21.15.28:0/72674811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619982ab0 0x562619984f70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00a470 tx=0x7f5b6c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655d56141008ce00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b74dba640 1 -- 172.21.15.28:0/72674811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b6c053570 con 0x562619982ab0 2024-09-19T11:23:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b74dba640 1 -- 172.21.15.28:0/72674811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c058070 con 0x562619982ab0 2024-09-19T11:23:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b74dba640 1 -- 172.21.15.28:0/72674811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b6c053880 con 0x562619982ab0 2024-09-19T11:23:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b7a130800 1 -- 172.21.15.28:0/72674811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619982ab0 msgr2=0x562619984f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b7a130800 1 --2- 172.21.15.28:0/72674811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619982ab0 0x562619984f70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00a470 tx=0x7f5b6c045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b7a130800 1 -- 172.21.15.28:0/72674811 shutdown_connections 2024-09-19T11:23:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b7a130800 1 --2- 172.21.15.28:0/72674811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619982ab0 0x562619984f70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b7a130800 1 -- 172.21.15.28:0/72674811 >> 172.21.15.28:0/72674811 conn(0x562619978a70 msgr2=0x56261997aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b7a130800 1 -- 172.21.15.28:0/72674811 shutdown_connections 2024-09-19T11:23:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.371+0000 7f5b7a130800 1 -- 172.21.15.28:0/72674811 wait complete. 2024-09-19T11:23:26.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.386+0000 7f5b7a130800 1 Processor -- start 2024-09-19T11:23:26.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.386+0000 7f5b7a130800 1 -- start start 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.386+0000 7f5b7a130800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a6e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5b7a130800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562619816dd0 con 0x56261997eeb0 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5b75dbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a6e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5b75dbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a6e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34298/0 (socket says 172.21.15.28:34298) 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5b75dbc640 1 -- 172.21.15.28:0/912819984 learned_addr learned my addr 172.21.15.28:0/912819984 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5b75dbc640 1 -- 172.21.15.28:0/912819984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562619a6e930 con 0x56261997eeb0 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/912819984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a6e340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5b68001800 tx=0x7f5b68001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c54e540e1de5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5a63e07640 1 -- 172.21.15.28:0/912819984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b68004520 con 0x56261997eeb0 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5a63e07640 1 -- 172.21.15.28:0/912819984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b680046c0 con 0x56261997eeb0 2024-09-19T11:23:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.387+0000 7f5a63e07640 1 -- 172.21.15.28:0/912819984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b680049e0 con 0x56261997eeb0 2024-09-19T11:23:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 -- 172.21.15.28:0/912819984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 msgr2=0x562619a6e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 --2- 172.21.15.28:0/912819984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a6e340 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5b68001800 tx=0x7f5b68001cc0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 -- 172.21.15.28:0/912819984 shutdown_connections 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 --2- 172.21.15.28:0/912819984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a6e340 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 -- 172.21.15.28:0/912819984 >> 172.21.15.28:0/912819984 conn(0x562619978a70 msgr2=0x562619a69cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 -- 172.21.15.28:0/912819984 shutdown_connections 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 -- 172.21.15.28:0/912819984 wait complete. 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 Processor -- start 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.388+0000 7f5b7a130800 1 -- start start 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b7a130800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a75e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b7a130800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562619985ac0 con 0x56261997eeb0 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b75dbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a75e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b75dbc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a75e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34300/0 (socket says 172.21.15.28:34300) 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b75dbc640 1 -- 172.21.15.28:0/2777804225 learned_addr learned my addr 172.21.15.28:0/2777804225 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b75dbc640 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562619a763f0 con 0x56261997eeb0 2024-09-19T11:23:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a75e90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b68000f80 tx=0x7f5b68001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5a62604640 1 -- 172.21.15.28:0/2777804225 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b68022020 con 0x56261997eeb0 2024-09-19T11:23:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5a62604640 1 -- 172.21.15.28:0/2777804225 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68014890 con 0x56261997eeb0 2024-09-19T11:23:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562619a76680 con 0x56261997eeb0 2024-09-19T11:23:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.389+0000 7f5a62604640 1 -- 172.21.15.28:0/2777804225 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b68014bb0 con 0x56261997eeb0 2024-09-19T11:23:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.390+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562619a77080 con 0x56261997eeb0 2024-09-19T11:23:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.390+0000 7f5a62604640 1 -- 172.21.15.28:0/2777804225 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5b68025020 con 0x56261997eeb0 2024-09-19T11:23:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5a62604640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5c03cfb0 0x7f5a5c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5a62604640 1 -- 172.21.15.28:0/2777804225 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5b68018030 con 0x56261997eeb0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5b755bb640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5c03cfb0 0x7f5a5c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619982ab0 0x562619a7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619a7da20 con 0x562619982ab0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5b765bd640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619982ab0 0x562619a7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5b755bb640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5c03cfb0 0x7f5a5c03f470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c057000 tx=0x7f5b6c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.391+0000 7f5b765bd640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619982ab0 0x562619a7d4e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b765bd640 1 -- 172.21.15.28:0/2777804225 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5b700083e0 con 0x562619982ab0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619a7ddc0 con 0x562619982ab0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b765bd640 1 -- 172.21.15.28:0/2777804225 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5b700083e0 con 0x562619982ab0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619a7f8c0 0x562619a81d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619a824d0 con 0x562619a7f8c0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619a7f8c0 0x562619a81d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619a7f8c0 0x562619a81d40 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.392+0000 7f5b75dbc640 1 -- 172.21.15.28:0/2777804225 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5b6801b070 con 0x562619a7f8c0 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.393+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619a7e610 con 0x562619982ab0 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.393+0000 7f5b765bd640 1 -- 172.21.15.28:0/2777804225 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5b700083e0 con 0x562619982ab0 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.393+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619a7ea60 con 0x562619a7f8c0 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.393+0000 7f5b75dbc640 1 -- 172.21.15.28:0/2777804225 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5b6801b070 con 0x562619a7f8c0 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.393+0000 7f5b7a130800 1 Processor -- start 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.393+0000 7f5b7a130800 1 -- start start 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b7a130800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997e650 0x56261997cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b7a130800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5626198754a0 con 0x56261997e650 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b765bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997e650 0x56261997cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b765bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997e650 0x56261997cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34312/0 (socket says 172.21.15.28:34312) 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b765bd640 1 -- 172.21.15.28:0/353609202 learned_addr learned my addr 172.21.15.28:0/353609202 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b765bd640 1 -- 172.21.15.28:0/353609202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56261997eb70 con 0x56261997e650 2024-09-19T11:23:26.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b765bd640 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997e650 0x56261997cec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x562619a76e90 tx=0x7f5b7000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5a5b7fe640 1 -- 172.21.15.28:0/353609202 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b7001c020 con 0x56261997e650 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5a5b7fe640 1 -- 172.21.15.28:0/353609202 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b70007c90 con 0x56261997e650 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56261997d400 con 0x56261997e650 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.394+0000 7f5a5b7fe640 1 -- 172.21.15.28:0/353609202 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b70018440 con 0x56261997e650 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.395+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56261997d950 con 0x56261997e650 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.395+0000 7f5a5b7fe640 1 -- 172.21.15.28:0/353609202 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5b7001f020 con 0x56261997e650 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.395+0000 7f5a5b7fe640 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5003d000 0x7f5a5003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5a5b7fe640 1 -- 172.21.15.28:0/353609202 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5b700559f0 con 0x56261997e650 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5003d000 0x7f5a5003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b7a130800 1 Processor -- start 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b7a130800 1 -- start start 2024-09-19T11:23:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5003d000 0x7f5a5003f4c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b68052b50 tx=0x7f5b68011f40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b7a130800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619a9fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b7a130800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562619a6ed60 con 0x562619a9faf0 2024-09-19T11:23:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b765bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619a9fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b765bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619a9fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34314/0 (socket says 172.21.15.28:34314) 2024-09-19T11:23:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.396+0000 7f5b765bd640 1 -- 172.21.15.28:0/2690224212 learned_addr learned my addr 172.21.15.28:0/2690224212 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b765bd640 1 -- 172.21.15.28:0/2690224212 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562619aa04e0 con 0x562619a9faf0 2024-09-19T11:23:26.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b765bd640 1 --2- 172.21.15.28:0/2690224212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619a9fef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b70053d70 tx=0x7f5b70053e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6581a9603128c61d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5a597fa640 1 -- 172.21.15.28:0/2690224212 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b7004a070 con 0x562619a9faf0 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5a597fa640 1 -- 172.21.15.28:0/2690224212 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b700174d0 con 0x562619a9faf0 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5a597fa640 1 -- 172.21.15.28:0/2690224212 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b70052070 con 0x562619a9faf0 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b7a130800 1 -- 172.21.15.28:0/2690224212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 msgr2=0x562619a9fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2690224212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619a9fef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b70053d70 tx=0x7f5b70053e80 comp rx=0 tx=0).stop 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b7a130800 1 -- 172.21.15.28:0/2690224212 shutdown_connections 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2690224212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619a9fef0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b7a130800 1 -- 172.21.15.28:0/2690224212 >> 172.21.15.28:0/2690224212 conn(0x562619a9bef0 msgr2=0x562619a9a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.397+0000 7f5b7a130800 1 -- 172.21.15.28:0/2690224212 shutdown_connections 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b7a130800 1 -- 172.21.15.28:0/2690224212 wait complete. 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b7a130800 1 Processor -- start 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b7a130800 1 -- start start 2024-09-19T11:23:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b7a130800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619aa7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b7a130800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562619aa0b00 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b765bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619aa7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b765bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619aa7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34328/0 (socket says 172.21.15.28:34328) 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 learned_addr learned my addr 172.21.15.28:0/1995197325 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.398+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562619aa7690 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.399+0000 7f5b765bd640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619aa7130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b70048040 tx=0x7f5b70053cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.399+0000 7f5a477fe640 1 -- 172.21.15.28:0/1995197325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b7004a040 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.399+0000 7f5a477fe640 1 -- 172.21.15.28:0/1995197325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b70013e80 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.399+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562619aa7920 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.399+0000 7f5a477fe640 1 -- 172.21.15.28:0/1995197325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b70020d30 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.399+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562619aa84b0 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.399+0000 7f5a477fe640 1 -- 172.21.15.28:0/1995197325 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5b70016040 con 0x562619a9faf0 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5a477fe640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a4003d000 0x7f5a4003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5a477fe640 1 -- 172.21.15.28:0/1995197325 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5b7003cc20 con 0x562619a9faf0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a4003d000 0x7f5a4003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a4003d000 0x7f5a4003f4c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b68052db0 tx=0x7f5b68046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619ab2a90 0x562619ab4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619ab56a0 con 0x562619ab2a90 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.400+0000 7f5b755bb640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619ab2a90 0x562619ab4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b755bb640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619ab2a90 0x562619ab4f50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 20 realm 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619ab5ad0 con 0x562619ab2a90 2024-09-19T11:23:26.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.401+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619ab63a0 con 0x562619ab2a90 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b7a130800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619ab6740 con 0x562619ab2a90 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619ab74a0 0x562619ab9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619ab9ea0 con 0x562619ab74a0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b765bd640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619ab74a0 0x562619ab9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b765bd640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619ab74a0 0x562619ab9890 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.402+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b7a130800 20 searching for the correct realm 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b765bd640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b70032020 0x7f5b7002a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b70032020 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b70032020 0x7f5b7002a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b70032020 0x7f5b7002a8b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b75dbc640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5b68027020 0x7f5b6801e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.403+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.404+0000 7f5b755bb640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5b68027020 0x7f5b6801e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.404+0000 7f5b755bb640 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5b68027020 0x7f5b6801e3f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.404+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.404+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab2a90 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.404+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.404+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab2a90 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b70032020 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.405+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab2a90 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab2a90 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.406+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b70032020 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.407+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.407+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.407+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.407+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.407+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.407+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab2a90 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.408+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab2a90 2024-09-19T11:23:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.409+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.409+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.409+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.409+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.409+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.409+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b70032020 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab2a90 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.410+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b68027020 2024-09-19T11:23:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x7f5b70032020 2024-09-19T11:23:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x562619abb7d0 con 0x562619ab74a0 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5b7006b040 con 0x7f5b68027020 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.411+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5b7006b040 con 0x562619ab2a90 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x7f5b7006b040 con 0x7f5b70032020 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619aba760 con 0x562619ab2a90 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619abab00 con 0x562619ab2a90 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.412+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619abaf20 con 0x562619ab74a0 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:26.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 zone default found 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 4 Realm: () 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 zonegroup default 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619abbc90 con 0x7f5b68027020 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619abdf70 con 0x562619ab74a0 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.413+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619abdf70 con 0x562619ab74a0 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.414+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.414+0000 7f5b7a130800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.414+0000 7f5b7a130800 20 started sync module instance, tier type = 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.414+0000 7f5b7a130800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.414+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619abd400 con 0x7f5b70032020 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.415+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.415+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94721639438368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619abe380 con 0x7f5b70032020 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.415+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619abebd0 con 0x562619ab2a90 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.416+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94721639438368] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.416+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.416+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94721639442208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619abeff0 con 0x562619ab2a90 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.416+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619abf840 con 0x562619ab74a0 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.417+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.417+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94721639444944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619abfcd0 con 0x562619ab74a0 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.417+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac07f0 con 0x7f5b70032020 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.417+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94721639442208] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.418+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94721639444944] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.418+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.418+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94721639449280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619abf6f0 con 0x7f5b70032020 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.418+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac1200 con 0x562619ab2a90 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.419+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94721639449280] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.419+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.419+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94721639443264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac1200 con 0x562619ab2a90 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.419+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac1bc0 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.420+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94721639443264] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.420+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.420+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94721639454960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac1200 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.420+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac2610 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.421+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94721639454960] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.421+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.422+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94721639457056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac1200 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.422+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac3020 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94721639457056] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94721639452464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac3020 con 0x7f5b68027020 2024-09-19T11:23:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=0 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=1 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=2 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=3 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=4 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=5 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=6 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94721639452464] v31'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 20 add_watcher() i=7 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.423+0000 7f5b7a130800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.424+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619ac6450 con 0x562619ab2a90 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.424+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.424+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94721639460160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619ac3970 con 0x562619ab2a90 2024-09-19T11:23:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.425+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94721639460160] v31'117 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.425+0000 7f5a2bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.425+0000 7f5b7a130800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.425+0000 7f5b7a130800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.425+0000 7f5b7a130800 5 note: GC not initialized 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.426+0000 7f5a2afe5640 20 reqs_thread_entry: start 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.426+0000 7f5b7a130800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.426+0000 7f5b7a130800 20 Filter name: none 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.426+0000 7f5a28fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.426+0000 7f5b7a130800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.426+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.426+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619af92a0 con 0x7f5b68027020 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:23:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619afb230 con 0x7f5b70032020 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619afccd0 con 0x7f5b68027020 2024-09-19T11:23:26.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(64 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.427+0000 7f5b7a130800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.428+0000 7f5b7a130800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.428+0000 7f5b7a130800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.428+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619b003f0 con 0x7f5b68027020 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(65 fud.buckets [call] v31'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619aff420 con 0x7f5b70032020 2024-09-19T11:23:26.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:26.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-09-19T11:23:26.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.429+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619b03f40 con 0x7f5b70032020 2024-09-19T11:23:26.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.430+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v31'21 uv21 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.430+0000 7f5b7a130800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.430+0000 7f5b7a130800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.430+0000 7f5b7a130800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.430+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94721639709024 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x562619b01240 con 0x7f5b68027020 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.430+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== watch-notify(notify (1) cookie 94721639457056 notify 133143986182 ret 0) ==== 380+0+0 (crc 0 0 0) 0x562619afd0c0 con 0x7f5b68027020 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.431+0000 7f5a597fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986182 cookie 94721639457056 notifier 4516 bl.length()=338 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.431+0000 7f5a597fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.431+0000 7f5a597fa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.431+0000 7f5a597fa640 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f5a38003ea0 con 0x7f5b68027020 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.431+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(68 notify.6 [notify cookie 94721639709024 out=8b] v0'0 uv120 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.431+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.431+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== watch-notify(notify_complete (2) cookie 94721639709024 notify 133143986182 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562619afc210 con 0x7f5b68027020 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.432+0000 7f5a2bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.432+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94721639460160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x562619ab2a90 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.433+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94721639460160] v31'118 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.433+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94721639438368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x7f5b70032020 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.434+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94721639438368] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.434+0000 7f5b7a130800 20 remove_watcher() i=0 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.434+0000 7f5b7a130800 2 removed watcher, disabling cache 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.434+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94721639442208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x562619ab2a90 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.435+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94721639442208] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.435+0000 7f5b7a130800 20 remove_watcher() i=1 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.435+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94721639444944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x562619ab74a0 2024-09-19T11:23:26.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.436+0000 7f5b765bd640 1 -- 172.21.15.28:0/1995197325 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94721639444944] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b70039070 con 0x562619ab74a0 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.436+0000 7f5b7a130800 20 remove_watcher() i=2 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.436+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94721639449280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x7f5b70032020 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.437+0000 7f5b75dbc640 1 -- 172.21.15.28:0/1995197325 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94721639449280] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6801b070 con 0x7f5b70032020 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.437+0000 7f5b7a130800 20 remove_watcher() i=3 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.437+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94721639443264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x562619ab2a90 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.438+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94721639443264] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x562619ab2a90 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.438+0000 7f5b7a130800 20 remove_watcher() i=4 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.438+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94721639454960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x7f5b68027020 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.439+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94721639454960] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.439+0000 7f5b7a130800 20 remove_watcher() i=5 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.439+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94721639457056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x7f5b68027020 2024-09-19T11:23:26.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.440+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94721639457056] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.440+0000 7f5b7a130800 20 remove_watcher() i=6 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.440+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94721639452464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x562619af7160 con 0x7f5b68027020 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.441+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94721639452464] v31'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5b6c051070 con 0x7f5b68027020 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.441+0000 7f5b7a130800 20 remove_watcher() i=7 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.441+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5b68027020 msgr2=0x7f5b6801e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.441+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5b68027020 0x7f5b6801e3f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619ab2a90 msgr2=0x562619ab4f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619ab2a90 0x562619ab4f50 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b70032020 msgr2=0x7f5b7002a8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5b70032020 0x7f5b7002a8b0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619ab74a0 msgr2=0x562619ab9890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619ab74a0 0x562619ab9890 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a4003d000 msgr2=0x7f5a4003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a4003d000 0x7f5a4003f4c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b68052db0 tx=0x7f5b68046000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 msgr2=0x562619aa7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b755bb640 1 -- 172.21.15.28:0/1995197325 reap_dead start 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619aa7130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b70048040 tx=0x7f5b70053cd0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 shutdown_connections 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/1995197325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562619a9faf0 0x562619aa7130 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 >> 172.21.15.28:0/1995197325 conn(0x562619a9bef0 msgr2=0x562619a9a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 shutdown_connections 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/1995197325 wait complete. 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5003d000 msgr2=0x7f5a5003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5003d000 0x7f5a5003f4c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b68052b50 tx=0x7f5b68011f40 comp rx=0 tx=0).stop 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997e650 msgr2=0x56261997cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997e650 0x56261997cec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x562619a76e90 tx=0x7f5b7000bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 shutdown_connections 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5003d000 0x7f5a5003f4c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 --2- 172.21.15.28:0/353609202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997e650 0x56261997cec0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 >> 172.21.15.28:0/353609202 conn(0x562619a85250 msgr2=0x562619a85630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 shutdown_connections 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.442+0000 7f5b7a130800 1 -- 172.21.15.28:0/353609202 wait complete. 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619982ab0 msgr2=0x562619a7d4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619982ab0 0x562619a7d4e0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619a7f8c0 msgr2=0x562619a81d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619a7f8c0 0x562619a81d40 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5c03cfb0 msgr2=0x7f5a5c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5c03cfb0 0x7f5a5c03f470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c057000 tx=0x7f5b6c052000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 msgr2=0x562619a75e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a75e90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5b68000f80 tx=0x7f5b68001250 comp rx=0 tx=0).stop 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 shutdown_connections 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562619a7f8c0 0x562619a81d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5a5c03cfb0 0x7f5a5c03f470 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562619982ab0 0x562619a7d4e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 --2- 172.21.15.28:0/2777804225 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56261997eeb0 0x562619a75e90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 >> 172.21.15.28:0/2777804225 conn(0x562619978a70 msgr2=0x562619a6f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 shutdown_connections 2024-09-19T11:23:26.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.443+0000 7f5b7a130800 1 -- 172.21.15.28:0/2777804225 wait complete. 2024-09-19T11:23:26.454 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1'] 2024-09-19T11:23:26.454 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1'] 2024-09-19T11:23:26.455 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:26.505 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:26.505 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:26.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.504+0000 7f9d5bd1f800 1 Processor -- start 2024-09-19T11:23:26.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.504+0000 7f9d5bd1f800 1 -- start start 2024-09-19T11:23:26.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d5bd1f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c9afc76d40 0x55c9afc77140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d5bd1f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c9afc78240 con 0x55c9afc77680 2024-09-19T11:23:26.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d5bd1f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55c9afb037d0 con 0x55c9afc76d40 2024-09-19T11:23:26.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d57aa9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c9afc76d40 0x55c9afc77140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d572a8640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55c9afc77680 0x55c9afc77b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48342/0 (socket says 172.21.15.28:48342) 2024-09-19T11:23:26.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d572a8640 1 -- 172.21.15.28:0/2945224882 learned_addr learned my addr 172.21.15.28:0/2945224882 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:26.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d57aa9640 1 -- 172.21.15.28:0/2945224882 >> v1:172.21.15.22:6789/0 conn(0x55c9afc77680 legacy=0x55c9afc77b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.505+0000 7f9d57aa9640 1 -- 172.21.15.28:0/2945224882 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9afc78410 con 0x55c9afc76d40 2024-09-19T11:23:26.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.507+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/2945224882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afc77140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800a470 tx=0x7f9d48045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359b978e9d8874fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.507+0000 7f9d56aa7640 1 -- 172.21.15.28:0/2945224882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d48053570 con 0x55c9afc76d40 2024-09-19T11:23:26.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.507+0000 7f9d56aa7640 1 -- 172.21.15.28:0/2945224882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d48058070 con 0x55c9afc76d40 2024-09-19T11:23:26.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.507+0000 7f9d56aa7640 1 -- 172.21.15.28:0/2945224882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d48053880 con 0x55c9afc76d40 2024-09-19T11:23:26.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.508+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/2945224882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 msgr2=0x55c9afc77140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.508+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/2945224882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afc77140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800a470 tx=0x7f9d48045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:26.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.508+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/2945224882 shutdown_connections 2024-09-19T11:23:26.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.508+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/2945224882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afc77140 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.508+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/2945224882 >> 172.21.15.28:0/2945224882 conn(0x55c9afc72a70 msgr2=0x55c9afc74ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.508+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/2945224882 shutdown_connections 2024-09-19T11:23:26.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.508+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/2945224882 wait complete. 2024-09-19T11:23:26.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.523+0000 7f9d5bd1f800 1 Processor -- start 2024-09-19T11:23:26.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.523+0000 7f9d5bd1f800 1 -- start start 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.523+0000 7f9d5bd1f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd6a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.523+0000 7f9d5bd1f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c9afb10dd0 con 0x55c9afc76d40 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9d57aa9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd6a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9d57aa9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd6a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34338/0 (socket says 172.21.15.28:34338) 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3508178150 learned_addr learned my addr 172.21.15.28:0/3508178150 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3508178150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9afd6ac00 con 0x55c9afc76d40 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/3508178150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd6a610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d48025c20 tx=0x7f9d48051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb883280e77257d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9c45e0b640 1 -- 172.21.15.28:0/3508178150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d4805c580 con 0x55c9afc76d40 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9c45e0b640 1 -- 172.21.15.28:0/3508178150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d48063030 con 0x55c9afc76d40 2024-09-19T11:23:26.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.524+0000 7f9c45e0b640 1 -- 172.21.15.28:0/3508178150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d4805c850 con 0x55c9afc76d40 2024-09-19T11:23:26.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3508178150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 msgr2=0x55c9afd6a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3508178150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd6a610 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9d48025c20 tx=0x7f9d48051000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3508178150 shutdown_connections 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3508178150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd6a610 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3508178150 >> 172.21.15.28:0/3508178150 conn(0x55c9afc72a70 msgr2=0x55c9afd65f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3508178150 shutdown_connections 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3508178150 wait complete. 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 Processor -- start 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 -- start start 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd72000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.525+0000 7f9d5bd1f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c9afc789a0 con 0x55c9afc76d40 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.526+0000 7f9d57aa9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd72000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.526+0000 7f9d57aa9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd72000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34340/0 (socket says 172.21.15.28:34340) 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.526+0000 7f9d57aa9640 1 -- 172.21.15.28:0/126891269 learned_addr learned my addr 172.21.15.28:0/126891269 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.526+0000 7f9d57aa9640 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9afd72560 con 0x55c9afc76d40 2024-09-19T11:23:26.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.526+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd72000 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800a330 tx=0x7f9d48002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.527+0000 7f9c3ffff640 1 -- 172.21.15.28:0/126891269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d4806c030 con 0x55c9afc76d40 2024-09-19T11:23:26.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.527+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c9afd727f0 con 0x55c9afc76d40 2024-09-19T11:23:26.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.527+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9afd73400 con 0x55c9afc76d40 2024-09-19T11:23:26.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.527+0000 7f9c3ffff640 1 -- 172.21.15.28:0/126891269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d4805c580 con 0x55c9afc76d40 2024-09-19T11:23:26.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.527+0000 7f9c3ffff640 1 -- 172.21.15.28:0/126891269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d48079060 con 0x55c9afc76d40 2024-09-19T11:23:26.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9c3ffff640 1 -- 172.21.15.28:0/126891269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9d48053440 con 0x55c9afc76d40 2024-09-19T11:23:26.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9c3ffff640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c4003d000 0x7f9c4003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9c3ffff640 1 -- 172.21.15.28:0/126891269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f9d48070020 con 0x55c9afc76d40 2024-09-19T11:23:26.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9d572a8640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c4003d000 0x7f9c4003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afc77680 0x55c9afd79650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afd79b90 con 0x55c9afc77680 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9d582aa640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afc77680 0x55c9afd79650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9d572a8640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c4003d000 0x7f9c4003f4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c001d70 tx=0x7f9d4c0012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.528+0000 7f9d582aa640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afc77680 0x55c9afd79650 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.529+0000 7f9d582aa640 1 -- 172.21.15.28:0/126891269 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d5000a4d0 con 0x55c9afc77680 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.529+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afd79f30 con 0x55c9afc77680 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.529+0000 7f9d582aa640 1 -- 172.21.15.28:0/126891269 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9d5000a4d0 con 0x55c9afc77680 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.529+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afd7ba30 0x55c9afd7ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.529+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afd7e660 con 0x55c9afd7ba30 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.529+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afd7ba30 0x55c9afd7ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.529+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afd7ba30 0x55c9afd7ded0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.530+0000 7f9d57aa9640 1 -- 172.21.15.28:0/126891269 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9d48051070 con 0x55c9afd7ba30 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.530+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afd7a780 con 0x55c9afc77680 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.530+0000 7f9d582aa640 1 -- 172.21.15.28:0/126891269 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9d5000a4d0 con 0x55c9afc77680 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.530+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afd7aba0 con 0x55c9afd7ba30 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.530+0000 7f9d57aa9640 1 -- 172.21.15.28:0/126891269 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9d48051070 con 0x55c9afd7ba30 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d5bd1f800 1 Processor -- start 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d5bd1f800 1 -- start start 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d5bd1f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd8d030 0x55c9afd8d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d5bd1f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c9afb6f4a0 con 0x55c9afd8d030 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d582aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd8d030 0x55c9afd8d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d582aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd8d030 0x55c9afd8d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34342/0 (socket says 172.21.15.28:34342) 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d582aa640 1 -- 172.21.15.28:0/1471608249 learned_addr learned my addr 172.21.15.28:0/1471608249 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.531+0000 7f9d582aa640 1 -- 172.21.15.28:0/1471608249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9afd8da90 con 0x55c9afd8d030 2024-09-19T11:23:26.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.532+0000 7f9d582aa640 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd8d030 0x55c9afd8d450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55c9afd73000 tx=0x7f9d500167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.532+0000 7f9c3d7fa640 1 -- 172.21.15.28:0/1471608249 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50013640 con 0x55c9afd8d030 2024-09-19T11:23:26.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.532+0000 7f9c3d7fa640 1 -- 172.21.15.28:0/1471608249 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d500137e0 con 0x55c9afd8d030 2024-09-19T11:23:26.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.532+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c9afd8e2c0 con 0x55c9afd8d030 2024-09-19T11:23:26.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.532+0000 7f9c3d7fa640 1 -- 172.21.15.28:0/1471608249 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50013ae0 con 0x55c9afd8d030 2024-09-19T11:23:26.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.532+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9afd8e860 con 0x55c9afd8d030 2024-09-19T11:23:26.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.533+0000 7f9c3d7fa640 1 -- 172.21.15.28:0/1471608249 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9d50011040 con 0x55c9afd8d030 2024-09-19T11:23:26.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.533+0000 7f9c3d7fa640 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c3003d000 0x7f9c3003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.533+0000 7f9c3d7fa640 1 -- 172.21.15.28:0/1471608249 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f9d5000e040 con 0x55c9afd8d030 2024-09-19T11:23:26.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.533+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c3003d000 0x7f9c3003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.533+0000 7f9d5bd1f800 1 Processor -- start 2024-09-19T11:23:26.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.533+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c3003d000 0x7f9c3003f4c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c9afd7ea00 tx=0x7f9d48068890 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.533+0000 7f9d5bd1f800 1 -- start start 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9d5bd1f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afd99ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9d5bd1f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c9afd6afd0 con 0x55c9afd99ad0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9d582aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afd99ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9d582aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afd99ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34356/0 (socket says 172.21.15.28:34356) 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9d582aa640 1 -- 172.21.15.28:0/1233644458 learned_addr learned my addr 172.21.15.28:0/1233644458 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9d582aa640 1 -- 172.21.15.28:0/1233644458 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9afd9a4c0 con 0x55c9afd99ad0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9d582aa640 1 --2- 172.21.15.28:0/1233644458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afd99ed0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d50054860 tx=0x7f9d50020ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1097a8ede748f178 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9c36ffd640 1 -- 172.21.15.28:0/1233644458 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50044030 con 0x55c9afd99ad0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9c36ffd640 1 -- 172.21.15.28:0/1233644458 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d50021a90 con 0x55c9afd99ad0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.534+0000 7f9c36ffd640 1 -- 172.21.15.28:0/1233644458 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d5004de60 con 0x55c9afd99ad0 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1233644458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 msgr2=0x55c9afd99ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/1233644458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afd99ed0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9d50054860 tx=0x7f9d50020ad0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1233644458 shutdown_connections 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/1233644458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afd99ed0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1233644458 >> 172.21.15.28:0/1233644458 conn(0x55c9afd95f10 msgr2=0x55c9afd94150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1233644458 shutdown_connections 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1233644458 wait complete. 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 Processor -- start 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 -- start start 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afda1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d5bd1f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c9afd9aae0 con 0x55c9afd99ad0 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d582aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afda1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d582aa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afda1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34368/0 (socket says 172.21.15.28:34368) 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.535+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 learned_addr learned my addr 172.21.15.28:0/3447041796 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.536+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9afda1660 con 0x55c9afd99ad0 2024-09-19T11:23:26.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.536+0000 7f9d582aa640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afda1100 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9d50022ff0 tx=0x7f9d500547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.536+0000 7f9c357fa640 1 -- 172.21.15.28:0/3447041796 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d500532b0 con 0x55c9afd99ad0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.536+0000 7f9c357fa640 1 -- 172.21.15.28:0/3447041796 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d50053450 con 0x55c9afd99ad0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.536+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c9afda18f0 con 0x55c9afd99ad0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.536+0000 7f9c357fa640 1 -- 172.21.15.28:0/3447041796 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d50020d20 con 0x55c9afd99ad0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.536+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9afda2480 con 0x55c9afd99ad0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.537+0000 7f9c357fa640 1 -- 172.21.15.28:0/3447041796 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9d5001fa90 con 0x55c9afd99ad0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.537+0000 7f9c357fa640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c2003d000 0x7f9c2003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.537+0000 7f9c357fa640 1 -- 172.21.15.28:0/3447041796 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f9d5004b030 con 0x55c9afd99ad0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.537+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c2003d000 0x7f9c2003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.538+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c2003d000 0x7f9c2003f4c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d48074fd0 tx=0x7f9d48082000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.538+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.538+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afdaca20 0x55c9afdaeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.538+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdaf630 con 0x55c9afdaca20 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.538+0000 7f9d572a8640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afdaca20 0x55c9afdaeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.538+0000 7f9d572a8640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afdaca20 0x55c9afdaeee0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d4c009050 con 0x55c9afdaca20 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 20 realm 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdabd70 con 0x55c9afdaca20 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d4c009050 con 0x55c9afdaca20 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb0120 con 0x55c9afdaca20 2024-09-19T11:23:26.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d4c009050 con 0x55c9afdaca20 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.539+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb09f0 con 0x55c9afdaca20 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9d4c009050 con 0x55c9afdaca20 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d5bd1f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afdb1410 0x55c9afdb3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb3f40 con 0x55c9afdb1410 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d582aa640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afdb1410 0x55c9afdb3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d582aa640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afdb1410 0x55c9afdb3800 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d5bd1f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:26.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d5bd1f800 20 searching for the correct realm 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.540+0000 7f9d582aa640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9d500455b0 0x7f9d5001aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.541+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d500455b0 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.541+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9d500455b0 0x7f9d5001aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.541+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9d500455b0 0x7f9d5001aa70 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.541+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.541+0000 7f9d57aa9640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9d4808c070 0x7f9d48053730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.541+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.541+0000 7f9d572a8640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9d4808c070 0x7f9d48053730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.542+0000 7f9d572a8640 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9d4808c070 0x7f9d48053730 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.542+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c009050 con 0x7f9d4808c070 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.542+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.542+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c009050 con 0x55c9afdaca20 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.542+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.542+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.542+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d500455b0 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.543+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d500455b0 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.544+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.545+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.546+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d500455b0 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.547+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d500455b0 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdb1410 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d4808c070 2024-09-19T11:23:26.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.548+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x55c9afdaca20 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5870 con 0x7f9d500455b0 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb4900 con 0x55c9afdaca20 2024-09-19T11:23:26.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5150 con 0x55c9afdaca20 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.549+0000 7f9d5bd1f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb5cf0 con 0x55c9afdb1410 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 zone default found 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 4 Realm: () 2024-09-19T11:23:26.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 zonegroup default 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb60b0 con 0x7f9d4808c070 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb8390 con 0x55c9afdb1410 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.550+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb8730 con 0x55c9afdb1410 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.551+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.551+0000 7f9d5bd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.551+0000 7f9d5bd1f800 20 started sync module instance, tier type = 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.551+0000 7f9d5bd1f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.551+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb77d0 con 0x7f9d500455b0 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.552+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.552+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94324727182112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb8b70 con 0x7f9d500455b0 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.552+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb93c0 con 0x55c9afdaca20 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.553+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94324727182112] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.553+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.553+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94324727189264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb97e0 con 0x55c9afdaca20 2024-09-19T11:23:26.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.553+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdba030 con 0x55c9afdb1410 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.554+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94324727189264] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.554+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.554+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94324727190800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdb9f40 con 0x55c9afdb1410 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.554+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbaa00 con 0x7f9d500455b0 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.555+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94324727190800] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.555+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.555+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94324727194832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdba9c0 con 0x7f9d500455b0 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.555+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbb510 con 0x55c9afdaca20 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.556+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.556+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94324727197760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbb510 con 0x55c9afdaca20 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.556+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbc4c0 con 0x7f9d4808c070 2024-09-19T11:23:26.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.557+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94324727194832] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.557+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94324727197760] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.557+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.557+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94324727201744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbb7d0 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.557+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbc9d0 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.558+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94324727201744] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.559+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.559+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94324727203008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbc990 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.559+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbd3b0 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94324727203008] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94324727205536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbd370 con 0x7f9d4808c070 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 20 add_watcher() i=0 2024-09-19T11:23:26.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 20 add_watcher() i=1 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 20 add_watcher() i=2 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 20 add_watcher() i=3 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 20 add_watcher() i=4 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 20 add_watcher() i=5 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.560+0000 7f9d5bd1f800 20 add_watcher() i=6 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.561+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94324727205536] v31'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.561+0000 7f9d5bd1f800 20 add_watcher() i=7 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.561+0000 7f9d5bd1f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:26.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.561+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdc0860 con 0x55c9afdaca20 2024-09-19T11:23:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.561+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.561+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94324727191952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdbdd80 con 0x55c9afdaca20 2024-09-19T11:23:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.562+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94324727191952] v31'119 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.562+0000 7f9c0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.562+0000 7f9d5bd1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.562+0000 7f9d5bd1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.562+0000 7f9d5bd1f800 5 note: GC not initialized 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.562+0000 7f9c0cfe1640 20 reqs_thread_entry: start 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.562+0000 7f9d5bd1f800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 20 Filter name: none 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9b867fc640 20 reqs_thread_entry: start 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf36e0 con 0x55c9afdaca20 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf5670 con 0x7f9d500455b0 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv21 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.563+0000 7f9d5bd1f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf6f60 con 0x7f9d4808c070 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(64 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.564+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdfb5a0 con 0x7f9d4808c070 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.565+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(65 fud.buckets [call] v31'3 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.565+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~387 in=387b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afe01320 con 0x7f9d4808c070 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.566+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(66 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [delete,create,call,call,writefull 0~387,setxattr (123)] v31'4 uv4 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.566+0000 7f9d5bd1f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.566+0000 7f9d5bd1f800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.566+0000 7f9d5bd1f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.566+0000 7f9d5bd1f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.566+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94324727445024 in=772b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdfb9d0 con 0x7f9d4808c070 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== watch-notify(notify (1) cookie 94324727201744 notify 133143986183 ret 0) ==== 802+0+0 (crc 0 0 0) 0x55c9afdf7350 con 0x7f9d4808c070 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9c36ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986183 cookie 94324727201744 notifier 4523 bl.length()=760 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9c36ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9c36ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9c36ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(67 notify.5 [notify cookie 94324727445024 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9c36ffd640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f9c1c003ea0 con 0x7f9d4808c070 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== watch-notify(notify_complete (2) cookie 94324727445024 notify 133143986183 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c9afdf6480 con 0x7f9d4808c070 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d5bd1f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_sources= new_sources= 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d5bd1f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential sources added=[] removed=[] 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d5bd1f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_dests= new_dests= 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d5bd1f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential dests added=[] removed=[] 2024-09-19T11:23:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.567+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf7600 con 0x7f9d500455b0 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.568+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(69 foo.buckets [call] v31'22 uv22 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.568+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf7600 con 0x7f9d500455b0 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.569+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v31'23 uv23 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.569+0000 7f9d5bd1f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.569+0000 7f9d5bd1f800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.569+0000 7f9d5bd1f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.569+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94324727444992 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c9afe00170 con 0x7f9d4808c070 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== watch-notify(notify (1) cookie 94324727203008 notify 133143986184 ret 0) ==== 356+0+0 (crc 0 0 0) 0x55c9afc78240 con 0x7f9d4808c070 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9c367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986184 cookie 94324727203008 notifier 4523 bl.length()=314 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9c367fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9c367fc640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9c367fc640 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f9b8c003400 con 0x7f9d4808c070 2024-09-19T11:23:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(71 notify.6 [notify cookie 94324727444992 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== watch-notify(notify_complete (2) cookie 94324727444992 notify 133143986184 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f9d4c02d760 con 0x7f9d4808c070 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.570+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.571+0000 7f9c0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.571+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94324727191952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x55c9afdaca20 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.572+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94324727191952] v31'120 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.572+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94324727182112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x7f9d500455b0 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.573+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94324727182112] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.573+0000 7f9d5bd1f800 20 remove_watcher() i=0 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.573+0000 7f9d5bd1f800 2 removed watcher, disabling cache 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.573+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94324727189264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x55c9afdaca20 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.574+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94324727189264] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.574+0000 7f9d5bd1f800 20 remove_watcher() i=1 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.574+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94324727190800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x55c9afdb1410 2024-09-19T11:23:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.575+0000 7f9d582aa640 1 -- 172.21.15.28:0/3447041796 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94324727190800] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d50053020 con 0x55c9afdb1410 2024-09-19T11:23:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.575+0000 7f9d5bd1f800 20 remove_watcher() i=2 2024-09-19T11:23:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.575+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94324727194832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x7f9d500455b0 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.576+0000 7f9d57aa9640 1 -- 172.21.15.28:0/3447041796 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94324727194832] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d48051070 con 0x7f9d500455b0 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.576+0000 7f9d5bd1f800 20 remove_watcher() i=3 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.576+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94324727197760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x55c9afdaca20 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.577+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94324727197760] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x55c9afdaca20 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.577+0000 7f9d5bd1f800 20 remove_watcher() i=4 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.577+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94324727201744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x7f9d4808c070 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.578+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94324727201744] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.578+0000 7f9d5bd1f800 20 remove_watcher() i=5 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.578+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94324727203008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x7f9d4808c070 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.579+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94324727203008] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.579+0000 7f9d5bd1f800 20 remove_watcher() i=6 2024-09-19T11:23:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.579+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94324727205536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c9afdf15a0 con 0x7f9d4808c070 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.580+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94324727205536] v31'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d4c029040 con 0x7f9d4808c070 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.580+0000 7f9d5bd1f800 20 remove_watcher() i=7 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9d4808c070 msgr2=0x7f9d48053730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9d4808c070 0x7f9d48053730 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afdaca20 msgr2=0x55c9afdaeee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afdaca20 0x55c9afdaeee0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9d500455b0 msgr2=0x7f9d5001aa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9d500455b0 0x7f9d5001aa70 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afdb1410 msgr2=0x55c9afdb3800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afdb1410 0x55c9afdb3800 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c2003d000 msgr2=0x7f9c2003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c2003d000 0x7f9c2003f4c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9d48074fd0 tx=0x7f9d48082000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 msgr2=0x55c9afda1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/3447041796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd99ad0 0x55c9afda1100 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9d50022ff0 tx=0x7f9d500547c0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 reap_dead start 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d572a8640 1 -- 172.21.15.28:0/3447041796 reap_dead start 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 shutdown_connections 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 >> 172.21.15.28:0/3447041796 conn(0x55c9afd95f10 msgr2=0x55c9afd94120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 shutdown_connections 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/3447041796 wait complete. 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c3003d000 msgr2=0x7f9c3003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c3003d000 0x7f9c3003f4c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55c9afd7ea00 tx=0x7f9d48068890 comp rx=0 tx=0).stop 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd8d030 msgr2=0x55c9afd8d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd8d030 0x55c9afd8d450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55c9afd73000 tx=0x7f9d500167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 shutdown_connections 2024-09-19T11:23:26.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c3003d000 0x7f9c3003f4c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/1471608249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afd8d030 0x55c9afd8d450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 >> 172.21.15.28:0/1471608249 conn(0x55c9afd813e0 msgr2=0x55c9afd817c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 shutdown_connections 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.581+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/1471608249 wait complete. 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afc77680 msgr2=0x55c9afd79650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afc77680 0x55c9afd79650 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afd7ba30 msgr2=0x55c9afd7ded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afd7ba30 0x55c9afd7ded0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c4003d000 msgr2=0x7f9c4003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c4003d000 0x7f9c4003f4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c001d70 tx=0x7f9d4c0012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 msgr2=0x55c9afd72000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd72000 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800a330 tx=0x7f9d48002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 shutdown_connections 2024-09-19T11:23:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c9afd7ba30 0x55c9afd7ded0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9c4003d000 0x7f9c4003f4c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c9afc77680 0x55c9afd79650 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 --2- 172.21.15.28:0/126891269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c9afc76d40 0x55c9afd72000 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 >> 172.21.15.28:0/126891269 conn(0x55c9afc72a70 msgr2=0x55c9afd64780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 shutdown_connections 2024-09-19T11:23:26.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.582+0000 7f9d5bd1f800 1 -- 172.21.15.28:0/126891269 wait complete. 2024-09-19T11:23:26.593 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-09-19T11:23:26.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-09-19T11:23:26.593 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-09-19T11:23:26.639 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:26.639 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:26.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.638+0000 7f0227f26800 1 Processor -- start 2024-09-19T11:23:26.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.638+0000 7f0227f26800 1 -- start start 2024-09-19T11:23:26.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0227f26800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561a986851c0 0x561a986855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0227f26800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561a9871a9c0 con 0x561a98685b00 2024-09-19T11:23:26.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0227f26800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x561a9859c7d0 con 0x561a986851c0 2024-09-19T11:23:26.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0223b09640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561a986851c0 0x561a986855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0223b09640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x561a986851c0 0x561a986855c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34374/0 (socket says 172.21.15.28:34374) 2024-09-19T11:23:26.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0223b09640 1 -- 172.21.15.28:0/2052364402 learned_addr learned my addr 172.21.15.28:0/2052364402 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0223308640 1 --1- 172.21.15.28:0/2052364402 >> v1:172.21.15.22:6789/0 conn(0x561a98685b00 0x561a987192a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48350/0 (socket says 172.21.15.28:48350) 2024-09-19T11:23:26.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0223b09640 1 -- 172.21.15.28:0/2052364402 >> v1:172.21.15.22:6789/0 conn(0x561a98685b00 legacy=0x561a987192a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.639+0000 7f0223b09640 1 -- 172.21.15.28:0/2052364402 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9871bba0 con 0x561a986851c0 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.641+0000 7f0223b09640 1 --2- 172.21.15.28:0/2052364402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a986855c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f021400a520 tx=0x7f0214045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83c5cade4e34297 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.641+0000 7f0222b07640 1 -- 172.21.15.28:0/2052364402 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214055070 con 0x561a986851c0 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.641+0000 7f0222b07640 1 -- 172.21.15.28:0/2052364402 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02140515b0 con 0x561a986851c0 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.641+0000 7f0222b07640 1 -- 172.21.15.28:0/2052364402 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02140518d0 con 0x561a986851c0 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.642+0000 7f0227f26800 1 -- 172.21.15.28:0/2052364402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 msgr2=0x561a986855c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.642+0000 7f0227f26800 1 --2- 172.21.15.28:0/2052364402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a986855c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f021400a520 tx=0x7f0214045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.642+0000 7f0227f26800 1 -- 172.21.15.28:0/2052364402 shutdown_connections 2024-09-19T11:23:26.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.642+0000 7f0227f26800 1 --2- 172.21.15.28:0/2052364402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a986855c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.642+0000 7f0227f26800 1 -- 172.21.15.28:0/2052364402 >> 172.21.15.28:0/2052364402 conn(0x561a9870cba0 msgr2=0x561a9870efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.642+0000 7f0227f26800 1 -- 172.21.15.28:0/2052364402 shutdown_connections 2024-09-19T11:23:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.642+0000 7f0227f26800 1 -- 172.21.15.28:0/2052364402 wait complete. 2024-09-19T11:23:26.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.657+0000 7f0227f26800 1 Processor -- start 2024-09-19T11:23:26.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.657+0000 7f0227f26800 1 -- start start 2024-09-19T11:23:26.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0227f26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a988034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0227f26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a985a9dd0 con 0x561a986851c0 2024-09-19T11:23:26.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0223b09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a988034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0223b09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a988034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34388/0 (socket says 172.21.15.28:34388) 2024-09-19T11:23:26.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0223b09640 1 -- 172.21.15.28:0/620507538 learned_addr learned my addr 172.21.15.28:0/620507538 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0223b09640 1 -- 172.21.15.28:0/620507538 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a98803ad0 con 0x561a986851c0 2024-09-19T11:23:26.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0223b09640 1 --2- 172.21.15.28:0/620507538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a988034e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0214025da0 tx=0x7f0214051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a3a2865d7ae7f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0111e0b640 1 -- 172.21.15.28:0/620507538 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214051950 con 0x561a986851c0 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0111e0b640 1 -- 172.21.15.28:0/620507538 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0214051af0 con 0x561a986851c0 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.658+0000 7f0111e0b640 1 -- 172.21.15.28:0/620507538 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214051dd0 con 0x561a986851c0 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 -- 172.21.15.28:0/620507538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 msgr2=0x561a988034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 --2- 172.21.15.28:0/620507538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a988034e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0214025da0 tx=0x7f0214051000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 -- 172.21.15.28:0/620507538 shutdown_connections 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 --2- 172.21.15.28:0/620507538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a988034e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 -- 172.21.15.28:0/620507538 >> 172.21.15.28:0/620507538 conn(0x561a9870cba0 msgr2=0x561a987fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 -- 172.21.15.28:0/620507538 shutdown_connections 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 -- 172.21.15.28:0/620507538 wait complete. 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.659+0000 7f0227f26800 1 Processor -- start 2024-09-19T11:23:26.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.660+0000 7f0227f26800 1 -- start start 2024-09-19T11:23:26.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.660+0000 7f0227f26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a9880aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.660+0000 7f0227f26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a9871add0 con 0x561a986851c0 2024-09-19T11:23:26.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.660+0000 7f0223b09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a9880aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.660+0000 7f0223b09640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a9880aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34394/0 (socket says 172.21.15.28:34394) 2024-09-19T11:23:26.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.660+0000 7f0223b09640 1 -- 172.21.15.28:0/4049508996 learned_addr learned my addr 172.21.15.28:0/4049508996 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.660+0000 7f0223b09640 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9880b410 con 0x561a986851c0 2024-09-19T11:23:26.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.661+0000 7f0223b09640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a9880aeb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0214067040 tx=0x7f0214002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.661+0000 7f010bfff640 1 -- 172.21.15.28:0/4049508996 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214073070 con 0x561a986851c0 2024-09-19T11:23:26.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.661+0000 7f010bfff640 1 -- 172.21.15.28:0/4049508996 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021405a450 con 0x561a986851c0 2024-09-19T11:23:26.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.661+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561a9880b6a0 con 0x561a986851c0 2024-09-19T11:23:26.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.661+0000 7f010bfff640 1 -- 172.21.15.28:0/4049508996 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021405a730 con 0x561a986851c0 2024-09-19T11:23:26.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.661+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a9880c2b0 con 0x561a986851c0 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.662+0000 7f010bfff640 1 -- 172.21.15.28:0/4049508996 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0214076070 con 0x561a986851c0 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.662+0000 7f010bfff640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f010c03d000 0x7f010c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.662+0000 7f010bfff640 1 -- 172.21.15.28:0/4049508996 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0214062070 con 0x561a986851c0 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.662+0000 7f0223308640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f010c03d000 0x7f010c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.662+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98685b00 0x561a98812500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.662+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98812a40 con 0x561a98685b00 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.662+0000 7f022430a640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98685b00 0x561a98812500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f0223308640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f010c03d000 0x7f010c03f4c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f020c001e30 tx=0x7f020c001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f022430a640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98685b00 0x561a98812500 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f022430a640 1 -- 172.21.15.28:0/4049508996 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a98685b00 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98812de0 con 0x561a98685b00 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f022430a640 1 -- 172.21.15.28:0/4049508996 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a98685b00 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a988148a0 0x561a98816d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a988174d0 con 0x561a988148a0 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.663+0000 7f0223b09640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a988148a0 0x561a98816d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.664+0000 7f0223b09640 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a988148a0 0x561a98816d40 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.664+0000 7f0223b09640 1 -- 172.21.15.28:0/4049508996 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0214075030 con 0x561a988148a0 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.664+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98813630 con 0x561a98685b00 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.664+0000 7f022430a640 1 -- 172.21.15.28:0/4049508996 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a98685b00 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.664+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a988139d0 con 0x561a988148a0 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.665+0000 7f0223b09640 1 -- 172.21.15.28:0/4049508996 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0214075030 con 0x561a988148a0 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.665+0000 7f0227f26800 1 Processor -- start 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.665+0000 7f0227f26800 1 -- start start 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.665+0000 7f0227f26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a98825e30 0x561a98826250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.665+0000 7f0227f26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a986084a0 con 0x561a98825e30 2024-09-19T11:23:26.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.665+0000 7f022430a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a98825e30 0x561a98826250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f022430a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a98825e30 0x561a98826250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34396/0 (socket says 172.21.15.28:34396) 2024-09-19T11:23:26.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f022430a640 1 -- 172.21.15.28:0/210909318 learned_addr learned my addr 172.21.15.28:0/210909318 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f022430a640 1 -- 172.21.15.28:0/210909318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a98826890 con 0x561a98825e30 2024-09-19T11:23:26.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f022430a640 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a98825e30 0x561a98826250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x561a9880bd40 tx=0x7f021c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f01097fa640 1 -- 172.21.15.28:0/210909318 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c013640 con 0x561a98825e30 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f01097fa640 1 -- 172.21.15.28:0/210909318 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021c0137e0 con 0x561a98825e30 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f01097fa640 1 -- 172.21.15.28:0/210909318 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c013ae0 con 0x561a98825e30 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561a988270c0 con 0x561a98825e30 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.666+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a98827660 con 0x561a98825e30 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.667+0000 7f01097fa640 1 -- 172.21.15.28:0/210909318 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f021c011040 con 0x561a98825e30 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.667+0000 7f01097fa640 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00fc03d000 0x7f00fc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.667+0000 7f01097fa640 1 -- 172.21.15.28:0/210909318 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f021c00e040 con 0x561a98825e30 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.667+0000 7f0223b09640 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00fc03d000 0x7f00fc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f0227f26800 1 Processor -- start 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f0223b09640 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00fc03d000 0x7f00fc03f4c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x561a9880cfb0 tx=0x7f0214096fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f0227f26800 1 -- start start 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f0227f26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98832cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f0227f26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a98803e40 con 0x561a988328b0 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f022430a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98832cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f022430a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98832cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34408/0 (socket says 172.21.15.28:34408) 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f022430a640 1 -- 172.21.15.28:0/3442094618 learned_addr learned my addr 172.21.15.28:0/3442094618 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.668+0000 7f022430a640 1 -- 172.21.15.28:0/3442094618 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a988332a0 con 0x561a988328b0 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f022430a640 1 --2- 172.21.15.28:0/3442094618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98832cb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f021c016cd0 tx=0x7f021c020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e522caa90d5da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0102ffd640 1 -- 172.21.15.28:0/3442094618 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c051070 con 0x561a988328b0 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0102ffd640 1 -- 172.21.15.28:0/3442094618 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021c01fdb0 con 0x561a988328b0 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0102ffd640 1 -- 172.21.15.28:0/3442094618 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c053440 con 0x561a988328b0 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 -- 172.21.15.28:0/3442094618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 msgr2=0x561a98832cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 --2- 172.21.15.28:0/3442094618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98832cb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f021c016cd0 tx=0x7f021c020460 comp rx=0 tx=0).stop 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 -- 172.21.15.28:0/3442094618 shutdown_connections 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 --2- 172.21.15.28:0/3442094618 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98832cb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 -- 172.21.15.28:0/3442094618 >> 172.21.15.28:0/3442094618 conn(0x561a9882ecb0 msgr2=0x561a9882cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 -- 172.21.15.28:0/3442094618 shutdown_connections 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 -- 172.21.15.28:0/3442094618 wait complete. 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.669+0000 7f0227f26800 1 Processor -- start 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f0227f26800 1 -- start start 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f0227f26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98839f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f0227f26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x561a988338c0 con 0x561a988328b0 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f022430a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98839f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f022430a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98839f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34412/0 (socket says 172.21.15.28:34412) 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 learned_addr learned my addr 172.21.15.28:0/1273165432 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a9883a460 con 0x561a988328b0 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f022430a640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98839f00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f021c056420 tx=0x7f021c054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f01017fa640 1 -- 172.21.15.28:0/1273165432 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c021440 con 0x561a988328b0 2024-09-19T11:23:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f01017fa640 1 -- 172.21.15.28:0/1273165432 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021c04c070 con 0x561a988328b0 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561a9883a6f0 con 0x561a988328b0 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.670+0000 7f01017fa640 1 -- 172.21.15.28:0/1273165432 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f021c01f340 con 0x561a988328b0 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.671+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a9883a6f0 con 0x561a988328b0 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.671+0000 7f01017fa640 1 -- 172.21.15.28:0/1273165432 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f021c0459d0 con 0x561a988328b0 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.671+0000 7f01017fa640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00ec03d000 0x7f00ec03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.672+0000 7f01017fa640 1 -- 172.21.15.28:0/1273165432 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f021c004720 con 0x561a988328b0 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.672+0000 7f0223b09640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00ec03d000 0x7f00ec03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.672+0000 7f0223b09640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00ec03d000 0x7f00ec03f4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02140873f0 tx=0x7f0214093000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.672+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.672+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98845630 0x561a98847af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.672+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98848240 con 0x561a98845630 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.672+0000 7f0223308640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98845630 0x561a98847af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0223308640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98845630 0x561a98847af0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f020c00f610 con 0x561a98845630 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 20 realm 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98844980 con 0x561a98845630 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f020c00f610 con 0x561a98845630 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:26.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98848d30 con 0x561a98845630 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f020c00f610 con 0x561a98845630 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.673+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98849600 con 0x561a98845630 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f020c00f610 con 0x561a98845630 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a9884a020 0x561a9884c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a9884cb50 con 0x561a9884a020 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f022430a640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a9884a020 0x561a9884c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f022430a640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a9884a020 0x561a9884c410 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 20 searching for the correct realm 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.674+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.675+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.675+0000 7f022430a640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f021c014ac0 0x7f021c0123d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.675+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f021c014ac0 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.675+0000 7f0223b09640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f021c014ac0 0x7f021c0123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.675+0000 7f0223b09640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f021c014ac0 0x7f021c0123d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.676+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.676+0000 7f0223b09640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02140a6040 0x7f0214050610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.676+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.676+0000 7f0223308640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02140a6040 0x7f0214050610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.676+0000 7f0223308640 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02140a6040 0x7f0214050610 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.676+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c00f610 con 0x7f02140a6040 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.676+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c00f610 con 0x561a98845630 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x561a98845630 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x7f02140a6040 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x7f02140a6040 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.677+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f021c014ac0 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x561a98845630 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x7f02140a6040 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.678+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x561a98845630 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f021c014ac0 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.679+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x7f02140a6040 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x561a98845630 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.680+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.681+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f020c02b040 con 0x561a98845630 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.681+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.681+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x7f02140a6040 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.681+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.681+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.681+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f021c014ac0 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c02b040 con 0x561a98845630 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.682+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f021c014ac0 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a9884a020 2024-09-19T11:23:26.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f02140a6040 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.683+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x561a98845630 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e480 con 0x7f021c014ac0 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:26.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a9884d510 con 0x561a98845630 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.684+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a9884dd60 con 0x561a98845630 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a9884e900 con 0x561a9884a020 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 zone default found 2024-09-19T11:23:26.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 4 Realm: () 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 zonegroup default 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a9884ecc0 con 0x7f02140a6040 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98850b20 con 0x561a9884a020 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.685+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f0227f26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98850fb0 con 0x561a9884a020 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f0227f26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f0227f26800 20 started sync module instance, tier type = 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f0227f26800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.686+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a9884ff90 con 0x7f021c014ac0 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.687+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.687+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94672227992448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a9884fbf0 con 0x7f021c014ac0 2024-09-19T11:23:26.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.687+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98851ee0 con 0x561a98845630 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.688+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94672227992448] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.688+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.688+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94672227998256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98852300 con 0x561a98845630 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.688+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98852cc0 con 0x561a9884a020 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.689+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94672227998256] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.689+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.689+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94672227999792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98852cc0 con 0x561a9884a020 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.689+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98853700 con 0x7f021c014ac0 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.690+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94672227999792] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.690+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.690+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94672228001808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98852c30 con 0x7f021c014ac0 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.690+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988545c0 con 0x561a98845630 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.691+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.691+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94672228008208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98854530 con 0x561a98845630 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.692+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988555b0 con 0x7f02140a6040 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.692+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94672228001808] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.692+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.692+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94672228008208] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.692+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94672228012160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98855570 con 0x7f02140a6040 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.693+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988549d0 con 0x7f02140a6040 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.693+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94672228012160] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.694+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.694+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94672228006640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98855b40 con 0x7f02140a6040 2024-09-19T11:23:26.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.694+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98856500 con 0x7f02140a6040 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94672228006640] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94672228016208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98856920 con 0x7f02140a6040 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 20 add_watcher() i=0 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 20 add_watcher() i=1 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 20 add_watcher() i=2 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 20 add_watcher() i=3 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 20 add_watcher() i=4 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 20 add_watcher() i=5 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.695+0000 7f0227f26800 20 add_watcher() i=6 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.696+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94672228016208] v31'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.696+0000 7f0227f26800 20 add_watcher() i=7 2024-09-19T11:23:26.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.696+0000 7f0227f26800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.696+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98856920 con 0x561a98845630 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.696+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.696+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94672228017264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98856900 con 0x561a98845630 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.697+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94672228017264] v31'121 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.697+0000 7f00d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.697+0000 7f0227f26800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.697+0000 7f0227f26800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.697+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988621c0 con 0x561a98845630 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.698+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.698+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988621c0 con 0x7f02140a6040 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.699+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.699+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98862c60 con 0x561a9884a020 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.700+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.700+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863040 con 0x561a98845630 2024-09-19T11:23:26.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.701+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'122 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.701+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98862d20 con 0x7f021c014ac0 2024-09-19T11:23:26.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.702+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'35 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.702+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863160 con 0x561a9884a020 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.704+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'23 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.704+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863560 con 0x561a98845630 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.705+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.705+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863540 con 0x7f02140a6040 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.706+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'23 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.706+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988634e0 con 0x7f021c014ac0 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.707+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.707+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988634e0 con 0x561a9884a020 2024-09-19T11:23:26.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.708+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'21 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.708+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988634e0 con 0x7f021c014ac0 2024-09-19T11:23:26.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.709+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'36 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.709+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988634e0 con 0x7f02140a6040 2024-09-19T11:23:26.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.710+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.710+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863570 con 0x7f021c014ac0 2024-09-19T11:23:26.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.711+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.711+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863650 con 0x7f02140a6040 2024-09-19T11:23:26.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.712+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'30 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.712+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863650 con 0x561a98845630 2024-09-19T11:23:26.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.713+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'123 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.713+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863650 con 0x561a9884a020 2024-09-19T11:23:26.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.715+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'29 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.715+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988636e0 con 0x7f021c014ac0 2024-09-19T11:23:26.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.716+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'37 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.716+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988636e0 con 0x561a9884a020 2024-09-19T11:23:26.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.717+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'15 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.717+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863700 con 0x561a98845630 2024-09-19T11:23:26.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.718+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.718+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863700 con 0x561a98845630 2024-09-19T11:23:26.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.719+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'124 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.719+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863700 con 0x561a9884a020 2024-09-19T11:23:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.720+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'30 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.720+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863700 con 0x7f021c014ac0 2024-09-19T11:23:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.721+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.721+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863760 con 0x561a9884a020 2024-09-19T11:23:26.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.722+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.722+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863790 con 0x561a9884a020 2024-09-19T11:23:26.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.723+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.723+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988637f0 con 0x561a9884a020 2024-09-19T11:23:26.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.724+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.724+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988637b0 con 0x561a9884a020 2024-09-19T11:23:26.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.725+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'24 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.725+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988637b0 con 0x7f02140a6040 2024-09-19T11:23:26.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.726+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.726+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863810 con 0x561a9884a020 2024-09-19T11:23:26.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.727+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.727+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863810 con 0x7f02140a6040 2024-09-19T11:23:26.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.728+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'24 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.728+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863810 con 0x561a9884a020 2024-09-19T11:23:26.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.729+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'31 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.729+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863810 con 0x7f02140a6040 2024-09-19T11:23:26.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.730+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'31 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.730+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98863810 con 0x7f021c014ac0 2024-09-19T11:23:26.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.731+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f00d8fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:26.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:26.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 20 Filter name: none 2024-09-19T11:23:26.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f00527fc640 20 reqs_thread_entry: start 2024-09-19T11:23:26.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:23:26.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98861e00 con 0x7f02140a6040 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.732+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98890d30 con 0x7f02140a6040 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.733+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(95 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 155+0+15 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.733+0000 7f0227f26800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.733+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a9888ed10 con 0x7f02140a6040 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v31'4 uv4 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0227f26800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:23:26.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 94672228251680 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a988913a0 con 0x561a9884a020 2024-09-19T11:23:26.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== watch-notify(notify (1) cookie 94672227999792 notify 133143986179 ret 0) ==== 212+0+0 (crc 0 0 0) 0x561a9885bcd0 con 0x561a9884a020 2024-09-19T11:23:26.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0102ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986179 cookie 94672227999792 notifier 4530 bl.length()=170 2024-09-19T11:23:26.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(97 notify.2 [notify cookie 94672228251680 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0102ffd640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f00e8003d60 con 0x561a9884a020 2024-09-19T11:23:26.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== watch-notify(notify_complete (2) cookie 94672228251680 notify 133143986179 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561a988332a0 con 0x561a9884a020 2024-09-19T11:23:26.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0227f26800 10 removing email index: 2024-09-19T11:23:26.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0227f26800 10 removing user buckets index 2024-09-19T11:23:26.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.734+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98891320 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.735+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(99 fud.buckets [delete] v31'4 uv4 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.735+0000 7f0227f26800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.735+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94672228258592 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a98890830 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== watch-notify(notify (1) cookie 94672228006640 notify 133143986185 ret 0) ==== 202+0+0 (crc 0 0 0) 0x561a988627b0 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0102ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986185 cookie 94672228006640 notifier 4530 bl.length()=160 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(100 notify.6 [notify cookie 94672228258592 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0102ffd640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f00e8003d60 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(101 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== watch-notify(notify_complete (2) cookie 94672228258592 notify 133143986185 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561a9885ba70 con 0x7f02140a6040 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0227f26800 10 removing user index: fud 2024-09-19T11:23:26.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.736+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98890830 con 0x7f02140a6040 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(102 fud [call,call,delete] v31'5 uv5 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0227f26800 10 removing default.rgw.meta+users.uid+fud from cache 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0227f26800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.3 7:c609908c:::notify.5:head [notify cookie 94672228257232 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x561a9888fcd0 con 0x7f02140a6040 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== watch-notify(notify (1) cookie 94672228012160 notify 133143986186 ret 0) ==== 194+0+0 (crc 0 0 0) 0x561a9885ba70 con 0x7f02140a6040 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0102ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986186 cookie 94672228012160 notifier 4530 bl.length()=152 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(103 notify.5 [notify cookie 94672228257232 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.737+0000 7f0102ffd640 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:104 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f00e8003d60 con 0x7f02140a6040 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.738+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(104 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.738+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== watch-notify(notify_complete (2) cookie 94672228257232 notify 133143986186 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561a98862560 con 0x7f02140a6040 2024-09-19T11:23:26.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.739+0000 7f00d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.739+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94672228017264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a988612e0 con 0x561a98845630 2024-09-19T11:23:26.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.740+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94672228017264] v31'125 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.740+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94672227992448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98856470 con 0x7f021c014ac0 2024-09-19T11:23:26.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.741+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94672227992448] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.741+0000 7f0227f26800 20 remove_watcher() i=0 2024-09-19T11:23:26.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.741+0000 7f0227f26800 2 removed watcher, disabling cache 2024-09-19T11:23:26.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.741+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94672227998256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98856470 con 0x561a98845630 2024-09-19T11:23:26.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.742+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94672227998256] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.742+0000 7f0227f26800 20 remove_watcher() i=1 2024-09-19T11:23:26.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.742+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94672227999792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98851a30 con 0x561a9884a020 2024-09-19T11:23:26.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.743+0000 7f022430a640 1 -- 172.21.15.28:0/1273165432 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94672227999792] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f021c00a4d0 con 0x561a9884a020 2024-09-19T11:23:26.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.743+0000 7f0227f26800 20 remove_watcher() i=2 2024-09-19T11:23:26.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.743+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94672228001808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98856470 con 0x7f021c014ac0 2024-09-19T11:23:26.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.744+0000 7f0223b09640 1 -- 172.21.15.28:0/1273165432 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94672228001808] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0214075030 con 0x7f021c014ac0 2024-09-19T11:23:26.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.744+0000 7f0227f26800 20 remove_watcher() i=3 2024-09-19T11:23:26.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.744+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94672228008208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98852030 con 0x561a98845630 2024-09-19T11:23:26.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.745+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94672228008208] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x561a98845630 2024-09-19T11:23:26.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.745+0000 7f0227f26800 20 remove_watcher() i=4 2024-09-19T11:23:26.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.745+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94672228012160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98851a30 con 0x7f02140a6040 2024-09-19T11:23:26.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.746+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94672228012160] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.746+0000 7f0227f26800 20 remove_watcher() i=5 2024-09-19T11:23:26.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.746+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94672228006640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98852810 con 0x7f02140a6040 2024-09-19T11:23:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.746+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94672228006640] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.747+0000 7f0227f26800 20 remove_watcher() i=6 2024-09-19T11:23:26.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.747+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94672228016208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x561a98856470 con 0x7f02140a6040 2024-09-19T11:23:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.747+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94672228016208] v31'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f020c03b040 con 0x7f02140a6040 2024-09-19T11:23:26.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.747+0000 7f0227f26800 20 remove_watcher() i=7 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02140a6040 msgr2=0x7f0214050610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02140a6040 0x7f0214050610 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98845630 msgr2=0x561a98847af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98845630 0x561a98847af0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f021c014ac0 msgr2=0x7f021c0123d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f021c014ac0 0x7f021c0123d0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a9884a020 msgr2=0x561a9884c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a9884a020 0x561a9884c410 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00ec03d000 msgr2=0x7f00ec03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00ec03d000 0x7f00ec03f4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02140873f0 tx=0x7f0214093000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 msgr2=0x561a98839f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0223308640 1 -- 172.21.15.28:0/1273165432 reap_dead start 2024-09-19T11:23:26.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98839f00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f021c056420 tx=0x7f021c054790 comp rx=0 tx=0).stop 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 shutdown_connections 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/1273165432 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a988328b0 0x561a98839f00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 >> 172.21.15.28:0/1273165432 conn(0x561a9882ecb0 msgr2=0x561a9882cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 shutdown_connections 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/1273165432 wait complete. 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00fc03d000 msgr2=0x7f00fc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00fc03d000 0x7f00fc03f4c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x561a9880cfb0 tx=0x7f0214096fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a98825e30 msgr2=0x561a98826250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a98825e30 0x561a98826250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x561a9880bd40 tx=0x7f021c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 shutdown_connections 2024-09-19T11:23:26.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f00fc03d000 0x7f00fc03f4c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 --2- 172.21.15.28:0/210909318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a98825e30 0x561a98826250 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 >> 172.21.15.28:0/210909318 conn(0x561a9881a1e0 msgr2=0x561a9881a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 shutdown_connections 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.748+0000 7f0227f26800 1 -- 172.21.15.28:0/210909318 wait complete. 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98685b00 msgr2=0x561a98812500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98685b00 0x561a98812500 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a988148a0 msgr2=0x561a98816d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a988148a0 0x561a98816d40 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f010c03d000 msgr2=0x7f010c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f010c03d000 0x7f010c03f4c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f020c001e30 tx=0x7f020c001390 comp rx=0 tx=0).stop 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 msgr2=0x561a9880aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a9880aeb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0214067040 tx=0x7f0214002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 shutdown_connections 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x561a988148a0 0x561a98816d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f010c03d000 0x7f010c03f4c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x561a98685b00 0x561a98812500 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 --2- 172.21.15.28:0/4049508996 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x561a986851c0 0x561a9880aeb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 >> 172.21.15.28:0/4049508996 conn(0x561a9870cba0 msgr2=0x561a98804810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 shutdown_connections 2024-09-19T11:23:26.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.749+0000 7f0227f26800 1 -- 172.21.15.28:0/4049508996 wait complete. 2024-09-19T11:23:26.759 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-19T11:23:26.759 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-19T11:23:26.760 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-09-19T11:23:26.816 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:26.816 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89daf44800 1 Processor -- start 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89daf44800 1 -- start start 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89daf44800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558121e71af0 0x558121e77690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89daf44800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558121e70d90 con 0x558121e706e0 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89daf44800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x558121cfc7d0 con 0x558121e71af0 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89d64f3640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558121e71af0 0x558121e77690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89d6cf4640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x558121e706e0 0x558121e713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48354/0 (socket says 172.21.15.28:48354) 2024-09-19T11:23:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.816+0000 7f89d6cf4640 1 -- 172.21.15.28:0/826986258 learned_addr learned my addr 172.21.15.28:0/826986258 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:26.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.817+0000 7f89d5cf2640 1 -- 172.21.15.28:0/826986258 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814229760 0 0) 0x558121e70d90 con 0x558121e706e0 2024-09-19T11:23:26.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.817+0000 7f89d5cf2640 1 -- 172.21.15.28:0/826986258 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89c4001e10 con 0x558121e706e0 2024-09-19T11:23:26.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.817+0000 7f89d64f3640 1 -- 172.21.15.28:0/826986258 >> v1:172.21.15.22:6789/0 conn(0x558121e706e0 legacy=0x558121e713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.817+0000 7f89d64f3640 1 -- 172.21.15.28:0/826986258 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558121e71fa0 con 0x558121e71af0 2024-09-19T11:23:26.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.818+0000 7f89d64f3640 1 --2- 172.21.15.28:0/826986258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e71af0 0x558121e77690 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89c000a470 tx=0x7f89c0045920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69f8389ea6e6c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89d5cf2640 1 -- 172.21.15.28:0/826986258 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c0059070 con 0x558121e71af0 2024-09-19T11:23:26.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89d5cf2640 1 -- 172.21.15.28:0/826986258 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c0054570 con 0x558121e71af0 2024-09-19T11:23:26.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89d5cf2640 1 -- 172.21.15.28:0/826986258 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c0054870 con 0x558121e71af0 2024-09-19T11:23:26.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89daf44800 1 -- 172.21.15.28:0/826986258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e71af0 msgr2=0x558121e77690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89daf44800 1 --2- 172.21.15.28:0/826986258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e71af0 0x558121e77690 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f89c000a470 tx=0x7f89c0045920 comp rx=0 tx=0).stop 2024-09-19T11:23:26.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89daf44800 1 -- 172.21.15.28:0/826986258 shutdown_connections 2024-09-19T11:23:26.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89daf44800 1 --2- 172.21.15.28:0/826986258 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e71af0 0x558121e77690 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89daf44800 1 -- 172.21.15.28:0/826986258 >> 172.21.15.28:0/826986258 conn(0x558121e6bab0 msgr2=0x558121e6df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89daf44800 1 -- 172.21.15.28:0/826986258 shutdown_connections 2024-09-19T11:23:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.819+0000 7f89daf44800 1 -- 172.21.15.28:0/826986258 wait complete. 2024-09-19T11:23:26.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.834+0000 7f89daf44800 1 Processor -- start 2024-09-19T11:23:26.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.834+0000 7f89daf44800 1 -- start start 2024-09-19T11:23:26.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.835+0000 7f89daf44800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121de48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.835+0000 7f89daf44800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558121d09dd0 con 0x558121e706e0 2024-09-19T11:23:26.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.835+0000 7f89d6cf4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121de48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.835+0000 7f89d6cf4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121de48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34428/0 (socket says 172.21.15.28:34428) 2024-09-19T11:23:26.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.835+0000 7f89d6cf4640 1 -- 172.21.15.28:0/748355897 learned_addr learned my addr 172.21.15.28:0/748355897 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.835+0000 7f89d6cf4640 1 -- 172.21.15.28:0/748355897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558121f63e70 con 0x558121e706e0 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.835+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/748355897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121de48b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f89c8001e30 tx=0x7f89c800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8164e7880e5e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f88c4e09640 1 -- 172.21.15.28:0/748355897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c8012020 con 0x558121e706e0 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f88c4e09640 1 -- 172.21.15.28:0/748355897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c800e470 con 0x558121e706e0 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f88c4e09640 1 -- 172.21.15.28:0/748355897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c800e790 con 0x558121e706e0 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f89daf44800 1 -- 172.21.15.28:0/748355897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 msgr2=0x558121de48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f89daf44800 1 --2- 172.21.15.28:0/748355897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121de48b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f89c8001e30 tx=0x7f89c800b780 comp rx=0 tx=0).stop 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f89daf44800 1 -- 172.21.15.28:0/748355897 shutdown_connections 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f89daf44800 1 --2- 172.21.15.28:0/748355897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121de48b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f89daf44800 1 -- 172.21.15.28:0/748355897 >> 172.21.15.28:0/748355897 conn(0x558121e6bab0 msgr2=0x558121f5bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f89daf44800 1 -- 172.21.15.28:0/748355897 shutdown_connections 2024-09-19T11:23:26.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.836+0000 7f89daf44800 1 -- 172.21.15.28:0/748355897 wait complete. 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89daf44800 1 Processor -- start 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89daf44800 1 -- start start 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89daf44800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121f61580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89daf44800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558121e77de0 con 0x558121e706e0 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89d6cf4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121f61580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89d6cf4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121f61580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34438/0 (socket says 172.21.15.28:34438) 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89d6cf4640 1 -- 172.21.15.28:0/194913328 learned_addr learned my addr 172.21.15.28:0/194913328 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.837+0000 7f89d6cf4640 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558121f61ae0 con 0x558121e706e0 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.838+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121f61580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x558121de43e0 tx=0x7f89c80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.838+0000 7f88c3606640 1 -- 172.21.15.28:0/194913328 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c8012030 con 0x558121e706e0 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.838+0000 7f88c3606640 1 -- 172.21.15.28:0/194913328 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c801a7a0 con 0x558121e706e0 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.838+0000 7f88c3606640 1 -- 172.21.15.28:0/194913328 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c801aaa0 con 0x558121e706e0 2024-09-19T11:23:26.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.838+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558121f5fe90 con 0x558121e706e0 2024-09-19T11:23:26.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.838+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558121f60450 con 0x558121e706e0 2024-09-19T11:23:26.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f88c3606640 1 -- 172.21.15.28:0/194913328 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f89c8015020 con 0x558121e706e0 2024-09-19T11:23:26.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f88c3606640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88bc03d000 0x7f88bc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f88c3606640 1 -- 172.21.15.28:0/194913328 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f89c8020070 con 0x558121e706e0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f89d64f3640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88bc03d000 0x7f88bc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121e71af0 0x558121f722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121f72800 con 0x558121e71af0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f89d74f5640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121e71af0 0x558121f722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.839+0000 7f89d64f3640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88bc03d000 0x7f88bc03f4c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x558121e6fe20 tx=0x7f89c0050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.840+0000 7f89d74f5640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121e71af0 0x558121f722c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.840+0000 7f89d74f5640 1 -- 172.21.15.28:0/194913328 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f89d000a4d0 con 0x558121e71af0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.840+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121f60960 con 0x558121e71af0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.840+0000 7f89d74f5640 1 -- 172.21.15.28:0/194913328 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f89d000a4d0 con 0x558121e71af0 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.840+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121f74ad0 0x558121f76ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.840+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121f77590 con 0x558121f74ad0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.840+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121f74ad0 0x558121f76ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.841+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121f74ad0 0x558121f76ec0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.841+0000 7f89d6cf4640 1 -- 172.21.15.28:0/194913328 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f89c8018950 con 0x558121f74ad0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.841+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121f73970 con 0x558121e71af0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.841+0000 7f89d74f5640 1 -- 172.21.15.28:0/194913328 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f89d000a4d0 con 0x558121e71af0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.841+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121f73dd0 con 0x558121f74ad0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89d6cf4640 1 -- 172.21.15.28:0/194913328 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f89c8018950 con 0x558121f74ad0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89daf44800 1 Processor -- start 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89daf44800 1 -- start start 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89daf44800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f85eb0 0x558121f862d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89daf44800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558121d684a0 con 0x558121f85eb0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89d74f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f85eb0 0x558121f862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89d74f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f85eb0 0x558121f862d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34444/0 (socket says 172.21.15.28:34444) 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.842+0000 7f89d74f5640 1 -- 172.21.15.28:0/484299613 learned_addr learned my addr 172.21.15.28:0/484299613 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.843+0000 7f89d74f5640 1 -- 172.21.15.28:0/484299613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558121f86910 con 0x558121f85eb0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.843+0000 7f89d74f5640 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f85eb0 0x558121f862d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x558121f60670 tx=0x7f89d00167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.843+0000 7f88c0e01640 1 -- 172.21.15.28:0/484299613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0013640 con 0x558121f85eb0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.843+0000 7f88c0e01640 1 -- 172.21.15.28:0/484299613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d00137e0 con 0x558121f85eb0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.843+0000 7f88c0e01640 1 -- 172.21.15.28:0/484299613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0013ac0 con 0x558121f85eb0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.843+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558121f87140 con 0x558121f85eb0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.843+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558121f876e0 con 0x558121f85eb0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.844+0000 7f88c0e01640 1 -- 172.21.15.28:0/484299613 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f89d0011040 con 0x558121f85eb0 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.844+0000 7f88c0e01640 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88b003d050 0x7f88b003f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.844+0000 7f88c0e01640 1 -- 172.21.15.28:0/484299613 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f89d000e040 con 0x558121f85eb0 2024-09-19T11:23:26.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.844+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88b003d050 0x7f88b003f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89daf44800 1 Processor -- start 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88b003d050 0x7f88b003f510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89c8052c40 tx=0x7f89c800beb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89daf44800 1 -- start start 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89daf44800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f92d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89daf44800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558121f64150 con 0x558121f92930 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89d74f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f92d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89d74f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f92d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34454/0 (socket says 172.21.15.28:34454) 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89d74f5640 1 -- 172.21.15.28:0/3454408396 learned_addr learned my addr 172.21.15.28:0/3454408396 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.845+0000 7f89d74f5640 1 -- 172.21.15.28:0/3454408396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558121f93320 con 0x558121f92930 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89d74f5640 1 --2- 172.21.15.28:0/3454408396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f92d30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f89d0054890 tx=0x7f89d0013e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2834fec13daf1354 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f88ba7fc640 1 -- 172.21.15.28:0/3454408396 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0051020 con 0x558121f92930 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f88ba7fc640 1 -- 172.21.15.28:0/3454408396 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d0021a90 con 0x558121f92930 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f88ba7fc640 1 -- 172.21.15.28:0/3454408396 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d0021570 con 0x558121f92930 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 -- 172.21.15.28:0/3454408396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 msgr2=0x558121f92d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 --2- 172.21.15.28:0/3454408396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f92d30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f89d0054890 tx=0x7f89d0013e40 comp rx=0 tx=0).stop 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 -- 172.21.15.28:0/3454408396 shutdown_connections 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 --2- 172.21.15.28:0/3454408396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f92d30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 -- 172.21.15.28:0/3454408396 >> 172.21.15.28:0/3454408396 conn(0x558121f8ed30 msgr2=0x558121f8cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 -- 172.21.15.28:0/3454408396 shutdown_connections 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 -- 172.21.15.28:0/3454408396 wait complete. 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.846+0000 7f89daf44800 1 Processor -- start 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89daf44800 1 -- start start 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89daf44800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f99f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89daf44800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558121f93940 con 0x558121f92930 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89d74f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f99f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89d74f5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f99f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34470/0 (socket says 172.21.15.28:34470) 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 learned_addr learned my addr 172.21.15.28:0/1639755542 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558121f9a4a0 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89d74f5640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f99f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f89d00536e0 tx=0x7f89d00547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f88b8ff9640 1 -- 172.21.15.28:0/1639755542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d001fa90 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f88b8ff9640 1 -- 172.21.15.28:0/1639755542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d001fc30 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f88b8ff9640 1 -- 172.21.15.28:0/1639755542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89d001f500 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558121f9a730 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.847+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558121f9b0b0 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.848+0000 7f88b8ff9640 1 -- 172.21.15.28:0/1639755542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f89d001f6a0 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.848+0000 7f88b8ff9640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88a003d000 0x7f88a003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.848+0000 7f88b8ff9640 1 -- 172.21.15.28:0/1639755542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f89d0034cf0 con 0x558121f92930 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.849+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88a003d000 0x7f88a003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.849+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88a003d000 0x7f88a003f4c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f89c800efd0 tx=0x7f89c804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.849+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.849+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121fa57a0 0x558121fa7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.849+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fa83b0 con 0x558121fa57a0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.849+0000 7f89d64f3640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121fa57a0 0x558121fa7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.849+0000 7f89d64f3640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121fa57a0 0x558121fa7c60 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 realm 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fa87e0 con 0x558121fa57a0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fa90b0 con 0x558121fa57a0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fa9450 con 0x558121fa57a0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121faa1b0 0x558121fac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.850+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121facbb0 con 0x558121faa1b0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89d74f5640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121faa1b0 0x558121fac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89d74f5640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121faa1b0 0x558121fac5a0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89daf44800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:26.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89daf44800 20 searching for the correct realm 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89d74f5640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f89d004d230 0x7f89d0032560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89d004d230 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.851+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f89d004d230 0x7f89d0032560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.852+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f89d004d230 0x7f89d0032560 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.852+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.852+0000 7f89d6cf4640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f89c8017050 0x7f89c80163b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.852+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.852+0000 7f89d64f3640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f89c8017050 0x7f89c80163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.853+0000 7f89d64f3640 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f89c8017050 0x7f89c80163b0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.853+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.853+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.853+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.853+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.853+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.853+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89d004d230 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.854+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89d004d230 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.855+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.856+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.857+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89d004d230 2024-09-19T11:23:26.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.858+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89d004d230 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121faa1b0 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89c8017050 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.859+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x558121fa57a0 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558121fae4e0 con 0x7f89d004d230 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fad5a0 con 0x558121fa57a0 2024-09-19T11:23:26.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.860+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121faddf0 con 0x558121fa57a0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fad1e0 con 0x558121faa1b0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 zone default found 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 4 Realm: () 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:26.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 zonegroup default 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fae980 con 0x7f89c8017050 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fb07b0 con 0x558121faa1b0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.861+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.862+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.862+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fb0c70 con 0x558121faa1b0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.862+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.862+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.862+0000 7f89daf44800 20 started sync module instance, tier type = 2024-09-19T11:23:26.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.862+0000 7f89daf44800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:26.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.862+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fafc50 con 0x7f89d004d230 2024-09-19T11:23:26.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.863+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.863+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94013109239920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121faf8b0 con 0x7f89d004d230 2024-09-19T11:23:26.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.863+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb1a90 con 0x558121fa57a0 2024-09-19T11:23:26.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.864+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94013109239920] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.864+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.864+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94013109244112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb1e80 con 0x558121fa57a0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.864+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb28e0 con 0x558121faa1b0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.865+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94013109244112] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.865+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.865+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94013109248912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb28a0 con 0x558121faa1b0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.865+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb3370 con 0x7f89d004d230 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.866+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94013109248912] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.866+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.866+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94013109251744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb3760 con 0x7f89d004d230 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.866+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb41c0 con 0x558121fa57a0 2024-09-19T11:23:26.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.867+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94013109251744] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.867+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.867+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94013109245168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb45b0 con 0x558121fa57a0 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.867+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb4ff0 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.868+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94013109245168] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.868+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.868+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94013109258944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb4fb0 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.869+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb5a60 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.869+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.869+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94013109261648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb5ed0 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.870+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb6990 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.870+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94013109258944] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94013109265504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb6950 con 0x7f89c8017050 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=0 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=1 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=2 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=3 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=4 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=5 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94013109261648] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=6 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94013109265504] v31'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c005c040 con 0x7f89c8017050 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 20 add_watcher() i=7 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.871+0000 7f89daf44800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.872+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fb5a20 con 0x558121fa57a0 2024-09-19T11:23:26.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.872+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f89c005c040 con 0x558121fa57a0 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.872+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94013109255328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fb5a20 con 0x558121fa57a0 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.873+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94013109255328] v31'126 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x558121fa57a0 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.873+0000 7f888d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.873+0000 7f89daf44800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.873+0000 7f89daf44800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.873+0000 7f89daf44800 5 note: GC not initialized 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.873+0000 7f8807fff640 20 reqs_thread_entry: start 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.873+0000 7f89daf44800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 20 Filter name: none 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f8805ffb640 20 reqs_thread_entry: start 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121fec950 con 0x7f89c8017050 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 246+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x7f89c8017050 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121feaf40 con 0x7f89c8017050 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x7f89c8017050 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-19T11:23:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.874+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~419 in=419b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121feb7c0 con 0x7f89c8017050 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.875+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~419] v31'5 uv5 ondisk = 0) ==== 288+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x7f89c8017050 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89daf44800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89daf44800 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89daf44800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 94013109482256 in=622b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121ff0290 con 0x7f89d004d230 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== watch-notify(notify (1) cookie 94013109251744 notify 133143986194 ret 0) ==== 652+0+0 (crc 0 0 0) 0x7f89c805b040 con 0x7f89d004d230 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f88ba7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986194 cookie 94013109251744 notifier 4537 bl.length()=610 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f88ba7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f88ba7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f88ba7fc640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f889c003ea0 con 0x7f89d004d230 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(65 notify.3 [notify cookie 94013109482256 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== watch-notify(notify_complete (2) cookie 94013109482256 notify 133143986194 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558121f60450 con 0x7f89d004d230 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.876+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121feddc0 con 0x7f89c8017050 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.877+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v31'6 uv6 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x7f89c8017050 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.877+0000 7f89daf44800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.877+0000 7f89daf44800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.877+0000 7f89daf44800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.877+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94013109482208 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558121ff0290 con 0x558121faa1b0 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== watch-notify(notify (1) cookie 94013109248912 notify 133143986180 ret 0) ==== 234+0+0 (crc 0 0 0) 0x558121f9a730 con 0x558121faa1b0 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f88b9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986180 cookie 94013109248912 notifier 4537 bl.length()=192 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f88b9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f88b9ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 notify.2 [notify cookie 94013109482208 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f88b9ffb640 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f880c003400 con 0x558121faa1b0 2024-09-19T11:23:26.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.878+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== watch-notify(notify_complete (2) cookie 94013109482208 notify 133143986180 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558121f9a4a0 con 0x558121faa1b0 2024-09-19T11:23:26.893 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"testx$tenanteduser","display_name":"tenanted-user","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"testx$tenanteduser","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-09-19T11:23:26.875578Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:26.875564Z","tags":[],"group_ids":[]} 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.879+0000 7f888d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.879+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94013109255328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x558121fa57a0 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.880+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94013109255328] v31'127 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x558121fa57a0 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.880+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94013109239920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x7f89d004d230 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.881+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94013109239920] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.881+0000 7f89daf44800 20 remove_watcher() i=0 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.881+0000 7f89daf44800 2 removed watcher, disabling cache 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.881+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94013109244112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x558121fa57a0 2024-09-19T11:23:26.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.882+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94013109244112] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x558121fa57a0 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.882+0000 7f89daf44800 20 remove_watcher() i=1 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.882+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94013109248912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x558121faa1b0 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.883+0000 7f89d74f5640 1 -- 172.21.15.28:0/1639755542 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94013109248912] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89d003a070 con 0x558121faa1b0 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.883+0000 7f89daf44800 20 remove_watcher() i=2 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.883+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94013109251744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x7f89d004d230 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.884+0000 7f89d6cf4640 1 -- 172.21.15.28:0/1639755542 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94013109251744] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c8018950 con 0x7f89d004d230 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.884+0000 7f89daf44800 20 remove_watcher() i=3 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.884+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94013109245168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x558121fa57a0 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.885+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94013109245168] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x558121fa57a0 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.885+0000 7f89daf44800 20 remove_watcher() i=4 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.885+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94013109258944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x7f89c8017050 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.886+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94013109258944] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x7f89c8017050 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.886+0000 7f89daf44800 20 remove_watcher() i=5 2024-09-19T11:23:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.886+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94013109261648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x7f89c8017050 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.887+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94013109261648] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x7f89c8017050 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.887+0000 7f89daf44800 20 remove_watcher() i=6 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.887+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94013109265504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558121fea5f0 con 0x7f89c8017050 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.888+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94013109265504] v31'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89c0080040 con 0x7f89c8017050 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.888+0000 7f89daf44800 20 remove_watcher() i=7 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f89c8017050 msgr2=0x7f89c80163b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f89c8017050 0x7f89c80163b0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121fa57a0 msgr2=0x558121fa7c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121fa57a0 0x558121fa7c60 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f89d004d230 msgr2=0x7f89d0032560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f89d004d230 0x7f89d0032560 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121faa1b0 msgr2=0x558121fac5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121faa1b0 0x558121fac5a0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88a003d000 msgr2=0x7f88a003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88a003d000 0x7f88a003f4c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f89c800efd0 tx=0x7f89c804f000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 msgr2=0x558121f99f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/1639755542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f92930 0x558121f99f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f89d00536e0 tx=0x7f89d00547f0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 reap_dead start 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89d64f3640 1 -- 172.21.15.28:0/1639755542 reap_dead start 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 shutdown_connections 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 >> 172.21.15.28:0/1639755542 conn(0x558121f8ed30 msgr2=0x558121f8cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 shutdown_connections 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/1639755542 wait complete. 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88b003d050 msgr2=0x7f88b003f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88b003d050 0x7f88b003f510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f89c8052c40 tx=0x7f89c800beb0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f85eb0 msgr2=0x558121f862d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f85eb0 0x558121f862d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x558121f60670 tx=0x7f89d00167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 shutdown_connections 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88b003d050 0x7f88b003f510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 --2- 172.21.15.28:0/484299613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121f85eb0 0x558121f862d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 >> 172.21.15.28:0/484299613 conn(0x558121f7a290 msgr2=0x558121f7a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 shutdown_connections 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.889+0000 7f89daf44800 1 -- 172.21.15.28:0/484299613 wait complete. 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121e71af0 msgr2=0x558121f722c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121e71af0 0x558121f722c0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121f74ad0 msgr2=0x558121f76ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121f74ad0 0x558121f76ec0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88bc03d000 msgr2=0x7f88bc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88bc03d000 0x7f88bc03f4c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x558121e6fe20 tx=0x7f89c0050000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 msgr2=0x558121f61580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121f61580 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x558121de43e0 tx=0x7f89c80040e0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 shutdown_connections 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558121f74ad0 0x558121f76ec0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f88bc03d000 0x7f88bc03f4c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558121e71af0 0x558121f722c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 --2- 172.21.15.28:0/194913328 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558121e706e0 0x558121f61580 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 >> 172.21.15.28:0/194913328 conn(0x558121e6bab0 msgr2=0x558121f5a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 shutdown_connections 2024-09-19T11:23:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.890+0000 7f89daf44800 1 -- 172.21.15.28:0/194913328 wait complete. 2024-09-19T11:23:26.904 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-09-19T11:23:26.875578Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:26.875564Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:26.904 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-19T11:23:26.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-19T11:23:26.905 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-09-19T11:23:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fe293d800 1 Processor -- start 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fe293d800 1 -- start start 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fe293d800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d2a65ded60 0x55d2a65df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fe293d800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d2a65e0260 con 0x55d2a65df6a0 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fe293d800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55d2a646b7d0 con 0x55d2a65ded60 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fde62a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d2a65ded60 0x55d2a65df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fdde29640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55d2a65df6a0 0x55d2a65dfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48362/0 (socket says 172.21.15.28:48362) 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.954+0000 7f4fdde29640 1 -- 172.21.15.28:0/1952298490 learned_addr learned my addr 172.21.15.28:0/1952298490 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.955+0000 7f4fde62a640 1 -- 172.21.15.28:0/1952298490 >> v1:172.21.15.22:6789/0 conn(0x55d2a65df6a0 legacy=0x55d2a65dfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.955+0000 7f4fde62a640 1 -- 172.21.15.28:0/1952298490 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a65e0430 con 0x55d2a65ded60 2024-09-19T11:23:26.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.955+0000 7f4fdd628640 1 -- 172.21.15.28:0/1952298490 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838950463 0 0) 0x55d2a65e0260 con 0x55d2a65df6a0 2024-09-19T11:23:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.956+0000 7f4fde62a640 1 --2- 172.21.15.28:0/1952298490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a65df160 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000a470 tx=0x7f4fd0045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9f27c6555cd770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fdd628640 1 -- 172.21.15.28:0/1952298490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0053570 con 0x55d2a65ded60 2024-09-19T11:23:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fdd628640 1 -- 172.21.15.28:0/1952298490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd0058070 con 0x55d2a65ded60 2024-09-19T11:23:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fdd628640 1 -- 172.21.15.28:0/1952298490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0053880 con 0x55d2a65ded60 2024-09-19T11:23:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fe293d800 1 -- 172.21.15.28:0/1952298490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 msgr2=0x55d2a65df160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fe293d800 1 --2- 172.21.15.28:0/1952298490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a65df160 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000a470 tx=0x7f4fd0045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fe293d800 1 -- 172.21.15.28:0/1952298490 shutdown_connections 2024-09-19T11:23:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fe293d800 1 --2- 172.21.15.28:0/1952298490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a65df160 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fe293d800 1 -- 172.21.15.28:0/1952298490 >> 172.21.15.28:0/1952298490 conn(0x55d2a65daa70 msgr2=0x55d2a65dced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fe293d800 1 -- 172.21.15.28:0/1952298490 shutdown_connections 2024-09-19T11:23:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.957+0000 7f4fe293d800 1 -- 172.21.15.28:0/1952298490 wait complete. 2024-09-19T11:23:26.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.972+0000 7f4fe293d800 1 Processor -- start 2024-09-19T11:23:26.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.972+0000 7f4fe293d800 1 -- start start 2024-09-19T11:23:26.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.972+0000 7f4fe293d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.972+0000 7f4fe293d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2a6478dd0 con 0x55d2a65ded60 2024-09-19T11:23:26.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4fde62a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4fde62a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34474/0 (socket says 172.21.15.28:34474) 2024-09-19T11:23:26.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4fde62a640 1 -- 172.21.15.28:0/3483584186 learned_addr learned my addr 172.21.15.28:0/3483584186 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4fde62a640 1 -- 172.21.15.28:0/3483584186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a66d2980 con 0x55d2a65ded60 2024-09-19T11:23:26.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4fde62a640 1 --2- 172.21.15.28:0/3483584186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d2390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005870 tx=0x7f4fd0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf5c1d82b3fa8b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4ecc608640 1 -- 172.21.15.28:0/3483584186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd005c580 con 0x55d2a65ded60 2024-09-19T11:23:26.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4ecc608640 1 -- 172.21.15.28:0/3483584186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd0063030 con 0x55d2a65ded60 2024-09-19T11:23:26.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.973+0000 7f4ecc608640 1 -- 172.21.15.28:0/3483584186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd005c850 con 0x55d2a65ded60 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 -- 172.21.15.28:0/3483584186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 msgr2=0x55d2a66d2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 --2- 172.21.15.28:0/3483584186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d2390 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0005870 tx=0x7f4fd0051000 comp rx=0 tx=0).stop 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 -- 172.21.15.28:0/3483584186 shutdown_connections 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 --2- 172.21.15.28:0/3483584186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d2390 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 -- 172.21.15.28:0/3483584186 >> 172.21.15.28:0/3483584186 conn(0x55d2a65daa70 msgr2=0x55d2a66cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 -- 172.21.15.28:0/3483584186 shutdown_connections 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 -- 172.21.15.28:0/3483584186 wait complete. 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.974+0000 7f4fe293d800 1 Processor -- start 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fe293d800 1 -- start start 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fe293d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fe293d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2a65e0960 con 0x55d2a65ded60 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fde62a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fde62a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34486/0 (socket says 172.21.15.28:34486) 2024-09-19T11:23:26.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fde62a640 1 -- 172.21.15.28:0/2649202992 learned_addr learned my addr 172.21.15.28:0/2649202992 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fde62a640 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a66da310 con 0x55d2a65ded60 2024-09-19T11:23:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.975+0000 7f4fde62a640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d9db0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0045f50 tx=0x7f4fd0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.976+0000 7f4ecae05640 1 -- 172.21.15.28:0/2649202992 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0078070 con 0x55d2a65ded60 2024-09-19T11:23:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.976+0000 7f4ecae05640 1 -- 172.21.15.28:0/2649202992 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd0054a70 con 0x55d2a65ded60 2024-09-19T11:23:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.976+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a66da5a0 con 0x55d2a65ded60 2024-09-19T11:23:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.976+0000 7f4ecae05640 1 -- 172.21.15.28:0/2649202992 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd0054d50 con 0x55d2a65ded60 2024-09-19T11:23:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.976+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a66db1b0 con 0x55d2a65ded60 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.976+0000 7f4ecae05640 1 -- 172.21.15.28:0/2649202992 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4fd007b070 con 0x55d2a65ded60 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4ecae05640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ec403d000 0x7f4ec403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4ecae05640 1 -- 172.21.15.28:0/2649202992 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f4fd0070070 con 0x55d2a65ded60 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4fdde29640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ec403d000 0x7f4ec403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a65df6a0 0x55d2a66e1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a66e1940 con 0x55d2a65df6a0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a65df6a0 0x55d2a66e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4fdde29640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ec403d000 0x7f4ec403f4c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd40013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.977+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a65df6a0 0x55d2a66e1400 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.978+0000 7f4fdee2b640 1 -- 172.21.15.28:0/2649202992 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4fd800a4d0 con 0x55d2a65df6a0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.978+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a66e1ce0 con 0x55d2a65df6a0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.978+0000 7f4fdee2b640 1 -- 172.21.15.28:0/2649202992 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4fd800a4d0 con 0x55d2a65df6a0 2024-09-19T11:23:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.978+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a66e37e0 0x55d2a66e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.978+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a66e63f0 con 0x55d2a66e37e0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.978+0000 7f4fde62a640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a66e37e0 0x55d2a66e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.978+0000 7f4fde62a640 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a66e37e0 0x55d2a66e5c60 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.979+0000 7f4fde62a640 1 -- 172.21.15.28:0/2649202992 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4fd007a030 con 0x55d2a66e37e0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.979+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a66e2530 con 0x55d2a65df6a0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.979+0000 7f4fdee2b640 1 -- 172.21.15.28:0/2649202992 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4fd800a4d0 con 0x55d2a65df6a0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.979+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a66e2980 con 0x55d2a66e37e0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.979+0000 7f4fde62a640 1 -- 172.21.15.28:0/2649202992 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4fd007a030 con 0x55d2a66e37e0 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fe293d800 1 Processor -- start 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fe293d800 1 -- start start 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fe293d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a66f4d80 0x55d2a66f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fe293d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2a64d74a0 con 0x55d2a66f4d80 2024-09-19T11:23:26.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fdee2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a66f4d80 0x55d2a66f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fdee2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a66f4d80 0x55d2a66f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34490/0 (socket says 172.21.15.28:34490) 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fdee2b640 1 -- 172.21.15.28:0/2139164739 learned_addr learned my addr 172.21.15.28:0/2139164739 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.980+0000 7f4fdee2b640 1 -- 172.21.15.28:0/2139164739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a66f57e0 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.981+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a66f4d80 0x55d2a66f51a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55d2a66db3d0 tx=0x7f4fd80167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.981+0000 7f4ec3fff640 1 -- 172.21.15.28:0/2139164739 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd8013640 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.981+0000 7f4ec3fff640 1 -- 172.21.15.28:0/2139164739 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd80137e0 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.981+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a66f6010 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.981+0000 7f4ec3fff640 1 -- 172.21.15.28:0/2139164739 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd8013ae0 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.981+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a66f65b0 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.982+0000 7f4ec3fff640 1 -- 172.21.15.28:0/2139164739 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4fd8011040 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.982+0000 7f4ec3fff640 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4eb803d000 0x7f4eb803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.982+0000 7f4ec3fff640 1 -- 172.21.15.28:0/2139164739 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f4fd800e040 con 0x55d2a66f4d80 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.982+0000 7f4fde62a640 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4eb803d000 0x7f4eb803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.982+0000 7f4fe293d800 1 Processor -- start 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.982+0000 7f4fde62a640 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4eb803d000 0x7f4eb803f4c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0096760 tx=0x7f4fd0054ef0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.982+0000 7f4fe293d800 1 -- start start 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4fe293d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6701c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4fe293d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2a66d2d50 con 0x55d2a6701820 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4fdee2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6701c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4fdee2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6701c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34500/0 (socket says 172.21.15.28:34500) 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4fdee2b640 1 -- 172.21.15.28:0/1949512900 learned_addr learned my addr 172.21.15.28:0/1949512900 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4fdee2b640 1 -- 172.21.15.28:0/1949512900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a67021f0 con 0x55d2a6701820 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/1949512900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6701c00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80547f0 tx=0x7f4fd801fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55acfde7c9f07b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4ec1ffb640 1 -- 172.21.15.28:0/1949512900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd8044030 con 0x55d2a6701820 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4ec1ffb640 1 -- 172.21.15.28:0/1949512900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd8021a90 con 0x55d2a6701820 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.983+0000 7f4ec1ffb640 1 -- 172.21.15.28:0/1949512900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd801f4b0 con 0x55d2a6701820 2024-09-19T11:23:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 -- 172.21.15.28:0/1949512900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 msgr2=0x55d2a6701c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 --2- 172.21.15.28:0/1949512900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6701c00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80547f0 tx=0x7f4fd801fe70 comp rx=0 tx=0).stop 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 -- 172.21.15.28:0/1949512900 shutdown_connections 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 --2- 172.21.15.28:0/1949512900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6701c00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 -- 172.21.15.28:0/1949512900 >> 172.21.15.28:0/1949512900 conn(0x55d2a66fdc20 msgr2=0x55d2a66fbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 -- 172.21.15.28:0/1949512900 shutdown_connections 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 -- 172.21.15.28:0/1949512900 wait complete. 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 Processor -- start 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 -- start start 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6708e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.984+0000 7f4fe293d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d2a6702830 con 0x55d2a6701820 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4fdee2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6708e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4fdee2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6708e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34504/0 (socket says 172.21.15.28:34504) 2024-09-19T11:23:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 learned_addr learned my addr 172.21.15.28:0/798478175 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a6709370 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6708e10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8053670 tx=0x7f4fd8054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4eaffff640 1 -- 172.21.15.28:0/798478175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd8021440 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4eaffff640 1 -- 172.21.15.28:0/798478175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd8021620 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2a6709600 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4eaffff640 1 -- 172.21.15.28:0/798478175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fd8054930 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.985+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a6709f80 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.986+0000 7f4eaffff640 1 -- 172.21.15.28:0/798478175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4fd8054ad0 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.986+0000 7f4eaffff640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ea803d000 0x7f4ea803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.987+0000 7f4eaffff640 1 -- 172.21.15.28:0/798478175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f4fd8034c80 con 0x55d2a6701820 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.987+0000 7f4fde62a640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ea803d000 0x7f4ea803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.987+0000 7f4fde62a640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ea803d000 0x7f4ea803f4c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55d2a66dbea0 tx=0x7f4fd0091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.987+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.987+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a6714740 0x55d2a6716c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.987+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6717350 con 0x55d2a6714740 2024-09-19T11:23:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.987+0000 7f4fdde29640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a6714740 0x55d2a6716c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fdde29640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a6714740 0x55d2a6716c00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4fd400f5f0 con 0x55d2a6714740 2024-09-19T11:23:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 20 realm 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6713a90 con 0x55d2a6714740 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4fd400f5f0 con 0x55d2a6714740 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6717e40 con 0x55d2a6714740 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4fd400f5f0 con 0x55d2a6714740 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.988+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6718710 con 0x55d2a6714740 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4fd400f5f0 con 0x55d2a6714740 2024-09-19T11:23:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fe293d800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a6719130 0x55d2a671b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671bc90 con 0x55d2a6719130 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a6719130 0x55d2a671b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a6719130 0x55d2a671b520 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fe293d800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fe293d800 20 searching for the correct realm 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.989+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fdee2b640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4fd804d2d0 0x7f4fd80359c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fde62a640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4fd804d2d0 0x7f4fd80359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fde62a640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4fd804d2d0 0x7f4fd80359c0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fde62a640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4fd00a6040 0x7f4fd0009410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.990+0000 7f4fdde29640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4fd00a6040 0x7f4fd0009410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.991+0000 7f4fdde29640 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4fd00a6040 0x7f4fd0009410 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.991+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd400f5f0 con 0x7f4fd00a6040 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.991+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.991+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd400f5f0 con 0x55d2a6714740 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.991+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.991+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x55d2a6714740 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.991+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.992+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x55d2a6714740 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x55d2a6714740 2024-09-19T11:23:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.993+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.994+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x55d2a6714740 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4fd402b040 con 0x55d2a6714740 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.995+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd402b040 con 0x55d2a6714740 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.996+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6719130 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.997+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd00a6040 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x55d2a6714740 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671d5c0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671c680 con 0x55d2a6714740 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.998+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671cf20 con 0x55d2a6714740 2024-09-19T11:23:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671da40 con 0x55d2a6719130 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 zone default found 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 4 Realm: () 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 zonegroup default 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671de40 con 0x7f4fd00a6040 2024-09-19T11:23:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:26.999+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671fc70 con 0x55d2a6719130 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fe293d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67200d0 con 0x55d2a6719130 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fe293d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fe293d800 20 started sync module instance, tier type = 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fe293d800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:27.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.000+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671f110 con 0x7f4fd804d2d0 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.001+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.001+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94363223979264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a671ed70 con 0x7f4fd804d2d0 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.001+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6720ef0 con 0x55d2a6714740 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.002+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94363223979264] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.002+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.002+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94363223983408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67212e0 con 0x55d2a6714740 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.002+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6721d40 con 0x55d2a6719130 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.003+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.003+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94363223988208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6721d00 con 0x55d2a6719130 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.003+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6722cf0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.003+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94363223983408] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.004+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94363223988208] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.004+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.004+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94363223990576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67227d0 con 0x7f4fd804d2d0 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.004+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6723700 con 0x55d2a6714740 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.005+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.005+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94363223994960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6723b20 con 0x55d2a6714740 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.005+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6724580 con 0x7f4fd00a6040 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.005+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94363223990576] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.006+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.006+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94363223994960] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.006+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94363223984464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6724540 con 0x7f4fd00a6040 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.006+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67236a0 con 0x7f4fd00a6040 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.007+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94363223984464] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.008+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.008+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94363223998608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67244b0 con 0x7f4fd00a6040 2024-09-19T11:23:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.008+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67254d0 con 0x7f4fd00a6040 2024-09-19T11:23:27.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94363223998608] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94363224001296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6725490 con 0x7f4fd00a6040 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 20 add_watcher() i=0 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 20 add_watcher() i=1 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 20 add_watcher() i=2 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 20 add_watcher() i=3 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 20 add_watcher() i=4 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 20 add_watcher() i=5 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fe293d800 20 add_watcher() i=6 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.009+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94363224001296] v31'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.010+0000 7f4fe293d800 20 add_watcher() i=7 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.010+0000 7f4fe293d800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.010+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67288c0 con 0x55d2a6714740 2024-09-19T11:23:27.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.010+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.010+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94363224002352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6725e10 con 0x55d2a6714740 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.011+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94363224002352] v31'128 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.011+0000 7f4e94fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.011+0000 7f4fe293d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.011+0000 7f4fe293d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.011+0000 7f4fe293d800 5 note: GC not initialized 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.011+0000 7f4e0f7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 20 Filter name: none 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4e0d7fa640 20 reqs_thread_entry: start 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-09-19T11:23:27.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a675b900 con 0x7f4fd00a6040 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 20 rados_obj.operate() r=0 bl.length=419 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.012+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a675b290 con 0x7f4fd804d2d0 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:27.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a675f0a0 con 0x7f4fd00a6040 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(64 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv4 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.013+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6763590 con 0x7f4fd804d2d0 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.014+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(65 foo.buckets [call] v31'24 uv24 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.014+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 8.a 8:532a8d66:root::.bucket.meta.testx%3amyfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [create,call version.inc in=38b,writefull 0~420 in=420b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766ea0 con 0x55d2a6714740 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [create,call,writefull 0~420,setxattr (203)] v31'1 uv1 ondisk = 0) ==== 338+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x7 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify cookie 94363224263888 in=867b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a67668f0 con 0x55d2a6714740 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== watch-notify(notify (1) cookie 94363223994960 notify 133143986187 ret 0) ==== 897+0+0 (crc 0 0 0) 0x55d2a6767240 con 0x55d2a6714740 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4ec1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986187 cookie 94363223994960 notifier 4544 bl.length()=855 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4ec1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x7 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4ec1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4ec1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4ec1ffb640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f4ea4003ea0 con 0x55d2a6714740 2024-09-19T11:23:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(67 notify.4 [notify cookie 94363224263888 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== watch-notify(notify_complete (2) cookie 94363224263888 notify 133143986187 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d2a6763090 con 0x55d2a6714740 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 20 handle_bi_update(): bucket=testx:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_sources= new_sources= 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 20 handle_bi_update(): bucket=testx:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential sources added=[] removed=[] 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 20 handle_bi_update(): bucket=testx:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_dests= new_dests= 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 20 handle_bi_update(): bucket=testx:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential dests added=[] removed=[] 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.016+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766990 con 0x55d2a6714740 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.018+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v31'1 uv1 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.018+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766990 con 0x55d2a6714740 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v31'1 uv1 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fe293d800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-09-19T11:23:27.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fe293d800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fe293d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 94363224270048 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766900 con 0x55d2a6714740 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== watch-notify(notify (1) cookie 94363223994960 notify 133143986188 ret 0) ==== 390+0+0 (crc 0 0 0) 0x55d2a6766d30 con 0x55d2a6714740 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(71 notify.4 [notify cookie 94363224270048 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4ec17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986188 cookie 94363223994960 notifier 4544 bl.length()=348 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4ec17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4ec17fa640 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4ec17fa640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f4e14003400 con 0x55d2a6714740 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.019+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.020+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== watch-notify(notify_complete (2) cookie 94363224270048 notify 133143986188 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d2a6765ce0 con 0x55d2a6714740 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.020+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:73 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766990 con 0x7f4fd804d2d0 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(73 myfoo [call,delete] v31'25 uv25 ondisk = 0) ==== 191+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fe293d800 10 removing default.rgw.meta+root+myfoo from cache 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fe293d800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94363224270048 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766900 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== watch-notify(notify (1) cookie 94363223998608 notify 133143986187 ret 0) ==== 191+0+0 (crc 0 0 0) 0x55d2a6765ce0 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4ec17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986187 cookie 94363223998608 notifier 4544 bl.length()=149 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4ec17fa640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f4e14003400 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(74 notify.6 [notify cookie 94363224270048 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(75 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.021+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== watch-notify(notify_complete (2) cookie 94363224270048 notify 133143986187 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d2a6763090 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.022+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766990 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(76 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call,delete] v31'6 uv6 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4fe293d800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 from cache 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4fe293d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.3 7:c609908c:::notify.5:head [notify cookie 94363224241952 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6766990 con 0x7f4fd00a6040 2024-09-19T11:23:27.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== watch-notify(notify (1) cookie 94363223984464 notify 133143986188 ret 0) ==== 248+0+0 (crc 0 0 0) 0x7f4fd4050040 con 0x7f4fd00a6040 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4ec1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986188 cookie 94363223984464 notifier 4544 bl.length()=206 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4ec1ffb640 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f4ea4003ea0 con 0x7f4fd00a6040 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(77 notify.5 [notify cookie 94363224241952 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.023+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(78 notify.5 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.024+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== watch-notify(notify_complete (2) cookie 94363224241952 notify 133143986188 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d2a6763090 con 0x7f4fd00a6040 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.024+0000 7f4e94fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.024+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94363224002352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x55d2a6714740 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.025+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94363224002352] v31'129 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.025+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94363223979264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x7f4fd804d2d0 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.026+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94363223979264] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.026+0000 7f4fe293d800 20 remove_watcher() i=0 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.026+0000 7f4fe293d800 2 removed watcher, disabling cache 2024-09-19T11:23:27.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.026+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94363223983408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x55d2a6714740 2024-09-19T11:23:27.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.027+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94363223983408] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.027+0000 7f4fe293d800 20 remove_watcher() i=1 2024-09-19T11:23:27.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.027+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94363223988208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x55d2a6719130 2024-09-19T11:23:27.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.028+0000 7f4fdee2b640 1 -- 172.21.15.28:0/798478175 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94363223988208] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd8053020 con 0x55d2a6719130 2024-09-19T11:23:27.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.029+0000 7f4fe293d800 20 remove_watcher() i=2 2024-09-19T11:23:27.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.029+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94363223990576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x7f4fd804d2d0 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.030+0000 7f4fde62a640 1 -- 172.21.15.28:0/798478175 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94363223990576] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd007a030 con 0x7f4fd804d2d0 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.030+0000 7f4fe293d800 20 remove_watcher() i=3 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.030+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94363223994960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x55d2a6714740 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.031+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94363223994960] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x55d2a6714740 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.031+0000 7f4fe293d800 20 remove_watcher() i=4 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.031+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94363223984464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x7f4fd00a6040 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.032+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94363223984464] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.032+0000 7f4fe293d800 20 remove_watcher() i=5 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.032+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94363223998608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x7f4fd00a6040 2024-09-19T11:23:27.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.033+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94363223998608] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.033+0000 7f4fe293d800 20 remove_watcher() i=6 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.033+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94363224001296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55d2a6759630 con 0x7f4fd00a6040 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94363224001296] v31'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4fd403b040 con 0x7f4fd00a6040 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 20 remove_watcher() i=7 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4fd00a6040 msgr2=0x7f4fd0009410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4fd00a6040 0x7f4fd0009410 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a6714740 msgr2=0x55d2a6716c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a6714740 0x55d2a6716c00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4fd804d2d0 msgr2=0x7f4fd80359c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4fd804d2d0 0x7f4fd80359c0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a6719130 msgr2=0x55d2a671b520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a6719130 0x55d2a671b520 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ea803d000 msgr2=0x7f4ea803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ea803d000 0x7f4ea803f4c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55d2a66dbea0 tx=0x7f4fd0091000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 msgr2=0x55d2a6708e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fdde29640 1 -- 172.21.15.28:0/798478175 reap_dead start 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6708e10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8053670 tx=0x7f4fd8054780 comp rx=0 tx=0).stop 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 shutdown_connections 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/798478175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a6701820 0x55d2a6708e10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 >> 172.21.15.28:0/798478175 conn(0x55d2a66fdc20 msgr2=0x55d2a66fbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 shutdown_connections 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/798478175 wait complete. 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4eb803d000 msgr2=0x7f4eb803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4eb803d000 0x7f4eb803f4c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0096760 tx=0x7f4fd0054ef0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a66f4d80 msgr2=0x55d2a66f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a66f4d80 0x55d2a66f51a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55d2a66db3d0 tx=0x7f4fd80167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 shutdown_connections 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.034+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4eb803d000 0x7f4eb803f4c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2139164739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a66f4d80 0x55d2a66f51a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 >> 172.21.15.28:0/2139164739 conn(0x55d2a66e9130 msgr2=0x55d2a66e9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 shutdown_connections 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2139164739 wait complete. 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a65df6a0 msgr2=0x55d2a66e1400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a65df6a0 0x55d2a66e1400 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a66e37e0 msgr2=0x55d2a66e5c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a66e37e0 0x55d2a66e5c60 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ec403d000 msgr2=0x7f4ec403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ec403d000 0x7f4ec403f4c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd40013b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 msgr2=0x55d2a66d9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d9db0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0045f50 tx=0x7f4fd0002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 shutdown_connections 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d2a66e37e0 0x55d2a66e5c60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4ec403d000 0x7f4ec403f4c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d2a65df6a0 0x55d2a66e1400 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 --2- 172.21.15.28:0/2649202992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d2a65ded60 0x55d2a66d9db0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 >> 172.21.15.28:0/2649202992 conn(0x55d2a65daa70 msgr2=0x55d2a66cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 shutdown_connections 2024-09-19T11:23:27.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.035+0000 7f4fe293d800 1 -- 172.21.15.28:0/2649202992 wait complete. 2024-09-19T11:23:27.049 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-09-19T11:23:27.049 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-09-19T11:23:27.049 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-09-19T11:23:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.093+0000 7f5effd39800 1 Processor -- start 2024-09-19T11:23:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.093+0000 7f5effd39800 1 -- start start 2024-09-19T11:23:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.093+0000 7f5effd39800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55887e076aa0 0x55887e07c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5effd39800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55887e075d40 con 0x55887e075690 2024-09-19T11:23:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5effd39800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55887df017d0 con 0x55887e076aa0 2024-09-19T11:23:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5efb1c6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55887e076aa0 0x55887e07c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5efb9c7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55887e075690 0x55887e076390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48370/0 (socket says 172.21.15.28:48370) 2024-09-19T11:23:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5efb9c7640 1 -- 172.21.15.28:0/754922638 learned_addr learned my addr 172.21.15.28:0/754922638 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5efb1c6640 1 -- 172.21.15.28:0/754922638 >> v1:172.21.15.22:6789/0 conn(0x55887e075690 legacy=0x55887e076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5efa9c5640 1 -- 172.21.15.28:0/754922638 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342014812 0 0) 0x55887e075d40 con 0x55887e075690 2024-09-19T11:23:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.094+0000 7f5efb1c6640 1 -- 172.21.15.28:0/754922638 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887e076f50 con 0x55887e076aa0 2024-09-19T11:23:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.096+0000 7f5efb1c6640 1 --2- 172.21.15.28:0/754922638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e076aa0 0x55887e07c640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000a470 tx=0x7f5ef0045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56577f4abb6f8bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.096+0000 7f5efa9c5640 1 -- 172.21.15.28:0/754922638 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0053570 con 0x55887e076aa0 2024-09-19T11:23:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.096+0000 7f5efa9c5640 1 -- 172.21.15.28:0/754922638 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0058070 con 0x55887e076aa0 2024-09-19T11:23:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.096+0000 7f5efa9c5640 1 -- 172.21.15.28:0/754922638 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0053880 con 0x55887e076aa0 2024-09-19T11:23:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.097+0000 7f5effd39800 1 -- 172.21.15.28:0/754922638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e076aa0 msgr2=0x55887e07c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.097+0000 7f5effd39800 1 --2- 172.21.15.28:0/754922638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e076aa0 0x55887e07c640 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000a470 tx=0x7f5ef0045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.097+0000 7f5effd39800 1 -- 172.21.15.28:0/754922638 shutdown_connections 2024-09-19T11:23:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.097+0000 7f5effd39800 1 --2- 172.21.15.28:0/754922638 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e076aa0 0x55887e07c640 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.097+0000 7f5effd39800 1 -- 172.21.15.28:0/754922638 >> 172.21.15.28:0/754922638 conn(0x55887e070a60 msgr2=0x55887e072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.097+0000 7f5effd39800 1 -- 172.21.15.28:0/754922638 shutdown_connections 2024-09-19T11:23:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.097+0000 7f5effd39800 1 -- 172.21.15.28:0/754922638 wait complete. 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.112+0000 7f5effd39800 1 Processor -- start 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.112+0000 7f5effd39800 1 -- start start 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.112+0000 7f5effd39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887dfe97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.112+0000 7f5effd39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55887df0edd0 con 0x55887e075690 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.112+0000 7f5efb9c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887dfe97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.112+0000 7f5efb9c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887dfe97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34518/0 (socket says 172.21.15.28:34518) 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.112+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3927913342 learned_addr learned my addr 172.21.15.28:0/3927913342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.113+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3927913342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887dfe9d00 con 0x55887e075690 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.113+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/3927913342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887dfe97c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001ea0 tx=0x7f5eec00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc1ec0c8bdb0d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.113+0000 7f5de9e0b640 1 -- 172.21.15.28:0/3927913342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eec008070 con 0x55887e075690 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.113+0000 7f5de9e0b640 1 -- 172.21.15.28:0/3927913342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec00d500 con 0x55887e075690 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.113+0000 7f5de9e0b640 1 -- 172.21.15.28:0/3927913342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eec00d880 con 0x55887e075690 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 -- 172.21.15.28:0/3927913342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 msgr2=0x55887dfe97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 --2- 172.21.15.28:0/3927913342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887dfe97c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001ea0 tx=0x7f5eec00b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 -- 172.21.15.28:0/3927913342 shutdown_connections 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 --2- 172.21.15.28:0/3927913342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887dfe97c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 -- 172.21.15.28:0/3927913342 >> 172.21.15.28:0/3927913342 conn(0x55887e070a60 msgr2=0x55887e160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 -- 172.21.15.28:0/3927913342 shutdown_connections 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 -- 172.21.15.28:0/3927913342 wait complete. 2024-09-19T11:23:27.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 Processor -- start 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 -- start start 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887e16ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5effd39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55887e07cd90 con 0x55887e075690 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5efb9c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887e16ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5efb9c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887e16ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34528/0 (socket says 172.21.15.28:34528) 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.114+0000 7f5efb9c7640 1 -- 172.21.15.28:0/2199927775 learned_addr learned my addr 172.21.15.28:0/2199927775 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.115+0000 7f5efb9c7640 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887e1704c0 con 0x55887e075690 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.115+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887e16ff60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0012f0 tx=0x7f5eec000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.115+0000 7f5de3fff640 1 -- 172.21.15.28:0/2199927775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eec00f9d0 con 0x55887e075690 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.115+0000 7f5de3fff640 1 -- 172.21.15.28:0/2199927775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eec00fb70 con 0x55887e075690 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.115+0000 7f5de3fff640 1 -- 172.21.15.28:0/2199927775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eec00fe50 con 0x55887e075690 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.115+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55887e170750 con 0x55887e075690 2024-09-19T11:23:27.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.115+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887e171100 con 0x55887e075690 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.116+0000 7f5de3fff640 1 -- 172.21.15.28:0/2199927775 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5eec017040 con 0x55887e075690 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.116+0000 7f5de3fff640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5de403d1a0 0x7f5de403f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.116+0000 7f5de3fff640 1 -- 172.21.15.28:0/2199927775 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5eec015020 con 0x55887e075690 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.116+0000 7f5efb1c6640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5de403d1a0 0x7f5de403f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.116+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e076aa0 0x55887e177560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.116+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e177aa0 con 0x55887e076aa0 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.117+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e076aa0 0x55887e177560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.117+0000 7f5efb1c6640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5de403d1a0 0x7f5de403f660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0057000 tx=0x7f5ef0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.117+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e076aa0 0x55887e177560 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.117+0000 7f5efc1c8640 1 -- 172.21.15.28:0/2199927775 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e076aa0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.117+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e177e40 con 0x55887e076aa0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.117+0000 7f5efc1c8640 1 -- 172.21.15.28:0/2199927775 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e076aa0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.118+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e179940 0x55887e17bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.118+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e17c550 con 0x55887e179940 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.118+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e179940 0x55887e17bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.118+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e179940 0x55887e17bdc0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.118+0000 7f5efb9c7640 1 -- 172.21.15.28:0/2199927775 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5eec02d070 con 0x55887e179940 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.118+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e178690 con 0x55887e076aa0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.119+0000 7f5efc1c8640 1 -- 172.21.15.28:0/2199927775 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e076aa0 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.119+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e178ae0 con 0x55887e179940 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.119+0000 7f5efb9c7640 1 -- 172.21.15.28:0/2199927775 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5eec02d070 con 0x55887e179940 2024-09-19T11:23:27.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.119+0000 7f5effd39800 1 Processor -- start 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.119+0000 7f5effd39800 1 -- start start 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.119+0000 7f5effd39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e18aec0 0x55887e18b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5effd39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55887df6d4a0 con 0x55887e18aec0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5efc1c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e18aec0 0x55887e18b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5efc1c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e18aec0 0x55887e18b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34538/0 (socket says 172.21.15.28:34538) 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5efc1c8640 1 -- 172.21.15.28:0/714275791 learned_addr learned my addr 172.21.15.28:0/714275791 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5efc1c8640 1 -- 172.21.15.28:0/714275791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887e18b920 con 0x55887e18aec0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e18aec0 0x55887e18b2e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55887e170f10 tx=0x7f5ef40167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5de17fa640 1 -- 172.21.15.28:0/714275791 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4013640 con 0x55887e18aec0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5de17fa640 1 -- 172.21.15.28:0/714275791 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef40137e0 con 0x55887e18aec0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55887e18c150 con 0x55887e18aec0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.120+0000 7f5de17fa640 1 -- 172.21.15.28:0/714275791 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4013ae0 con 0x55887e18aec0 2024-09-19T11:23:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.121+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887e18c6f0 con 0x55887e18aec0 2024-09-19T11:23:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.121+0000 7f5de17fa640 1 -- 172.21.15.28:0/714275791 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5ef4011040 con 0x55887e18aec0 2024-09-19T11:23:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.121+0000 7f5de17fa640 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dd403d000 0x7f5dd403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5de17fa640 1 -- 172.21.15.28:0/714275791 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5ef400e040 con 0x55887e18aec0 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dd403d000 0x7f5dd403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5effd39800 1 Processor -- start 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dd403d000 0x7f5dd403f4c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0531e0 tx=0x7f5eec0011b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5effd39800 1 -- start start 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5effd39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e197d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5effd39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55887e16b2f0 con 0x55887e197920 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5efc1c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5efc1c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34542/0 (socket says 172.21.15.28:34542) 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5efc1c8640 1 -- 172.21.15.28:0/1371413113 learned_addr learned my addr 172.21.15.28:0/1371413113 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.122+0000 7f5efc1c8640 1 -- 172.21.15.28:0/1371413113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887e198310 con 0x55887e197920 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/1371413113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e197d20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4054810 tx=0x7f5ef40143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a571717ae005ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5ddaffd640 1 -- 172.21.15.28:0/1371413113 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4051070 con 0x55887e197920 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5ddaffd640 1 -- 172.21.15.28:0/1371413113 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef4053440 con 0x55887e197920 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5ddaffd640 1 -- 172.21.15.28:0/1371413113 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4021a40 con 0x55887e197920 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5effd39800 1 -- 172.21.15.28:0/1371413113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 msgr2=0x55887e197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5effd39800 1 --2- 172.21.15.28:0/1371413113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e197d20 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4054810 tx=0x7f5ef40143c0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5effd39800 1 -- 172.21.15.28:0/1371413113 shutdown_connections 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5effd39800 1 --2- 172.21.15.28:0/1371413113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e197d20 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5effd39800 1 -- 172.21.15.28:0/1371413113 >> 172.21.15.28:0/1371413113 conn(0x55887e193d20 msgr2=0x55887e191f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5effd39800 1 -- 172.21.15.28:0/1371413113 shutdown_connections 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.123+0000 7f5effd39800 1 -- 172.21.15.28:0/1371413113 wait complete. 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5effd39800 1 Processor -- start 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5effd39800 1 -- start start 2024-09-19T11:23:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5effd39800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5effd39800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55887e198930 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5efc1c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5efc1c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e19ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34548/0 (socket says 172.21.15.28:34548) 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 learned_addr learned my addr 172.21.15.28:0/3911901865 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.124+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887e19f480 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.125+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e19ef20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4022ff0 tx=0x7f5ef4054770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.125+0000 7f5dd97fa640 1 -- 172.21.15.28:0/3911901865 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4021a40 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.125+0000 7f5dd97fa640 1 -- 172.21.15.28:0/3911901865 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef4013c80 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.125+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55887e19f710 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.125+0000 7f5dd97fa640 1 -- 172.21.15.28:0/3911901865 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ef4045130 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.125+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887e1a0090 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.125+0000 7f5dd97fa640 1 -- 172.21.15.28:0/3911901865 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5ef40452d0 con 0x55887e197920 2024-09-19T11:23:27.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5dd97fa640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dc403d000 0x7f5dc403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5dd97fa640 1 -- 172.21.15.28:0/3911901865 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5ef403d8a0 con 0x55887e197920 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dc403d000 0x7f5dc403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e1aa7e0 0x55887e1acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ad3f0 con 0x55887e1aa7e0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5efb1c6640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e1aa7e0 0x55887e1acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.126+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dc403d000 0x7f5dc403f4c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00efd0 tx=0x7f5eec041000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5efb1c6640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e1aa7e0 0x55887e1acca0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 20 realm 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ad820 con 0x55887e1aa7e0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ae0f0 con 0x55887e1aa7e0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.127+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ae490 con 0x55887e1aa7e0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5effd39800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e1af1f0 0x55887e1b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b1b20 con 0x55887e1af1f0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e1af1f0 0x55887e1b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e1af1f0 0x55887e1b15e0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5effd39800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5effd39800 20 searching for the correct realm 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.128+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efc1c8640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5ef4056050 0x7f5ef401a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5ef4056050 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5ef4056050 0x7f5ef401a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5ef4056050 0x7f5ef401a0b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efb9c7640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5eec053530 0x7f5eec053970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.129+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.130+0000 7f5efb1c6640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5eec053530 0x7f5eec053970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.130+0000 7f5efb1c6640 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5eec053530 0x7f5eec053970 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.130+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.130+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.130+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.130+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5ef4056050 2024-09-19T11:23:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.131+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.132+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5ef4056050 2024-09-19T11:23:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.133+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.134+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5ef4056050 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.135+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5ef4056050 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.136+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1af1f0 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5eec053530 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x55887e1aa7e0 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3490 con 0x7f5ef4056050 2024-09-19T11:23:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.137+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b2590 con 0x55887e1aa7e0 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b2e30 con 0x55887e1aa7e0 2024-09-19T11:23:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b21d0 con 0x55887e1af1f0 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 zone default found 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 4 Realm: () 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 zonegroup default 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.138+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b3e10 con 0x7f5eec053530 2024-09-19T11:23:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b5cd0 con 0x55887e1af1f0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b6140 con 0x55887e1af1f0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 20 started sync module instance, tier type = 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.139+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b5040 con 0x7f5ef4056050 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.140+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.140+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94044719633456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b4ca0 con 0x7f5ef4056050 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.140+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b7050 con 0x55887e1aa7e0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.141+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.141+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94044719633456] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.141+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94044719639456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b6fc0 con 0x55887e1aa7e0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.141+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b7f10 con 0x55887e1af1f0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.142+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.142+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94044719643232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b7e80 con 0x55887e1af1f0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.142+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b8f00 con 0x7f5ef4056050 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.142+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94044719639456] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.143+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94044719643232] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.143+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.143+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94044719645440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b7440 con 0x7f5ef4056050 2024-09-19T11:23:27.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.143+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b97e0 con 0x55887e1aa7e0 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.144+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.144+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94044719649520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ba080 con 0x55887e1aa7e0 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.144+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1bab60 con 0x7f5eec053530 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.145+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94044719645440] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.145+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94044719649520] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.146+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.146+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94044719654512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b9b00 con 0x7f5eec053530 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.146+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1bb400 con 0x7f5eec053530 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.147+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94044719654512] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.147+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.147+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94044719647184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1b9b00 con 0x7f5eec053530 2024-09-19T11:23:27.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.147+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1bba20 con 0x7f5eec053530 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94044719647184] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94044719656704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1bba20 con 0x7f5eec053530 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 20 add_watcher() i=0 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 20 add_watcher() i=1 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 20 add_watcher() i=2 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 20 add_watcher() i=3 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 20 add_watcher() i=4 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 20 add_watcher() i=5 2024-09-19T11:23:27.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.148+0000 7f5effd39800 20 add_watcher() i=6 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.149+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94044719656704] v31'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.149+0000 7f5effd39800 20 add_watcher() i=7 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.149+0000 7f5effd39800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.149+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1bed60 con 0x55887e1aa7e0 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.149+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.149+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94044719658720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1bbfb0 con 0x55887e1aa7e0 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.150+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94044719658720] v31'130 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.150+0000 7f5db27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.150+0000 7f5effd39800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.150+0000 7f5effd39800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.150+0000 7f5effd39800 5 note: GC not initialized 2024-09-19T11:23:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.150+0000 7f5db17e2640 20 reqs_thread_entry: start 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.151+0000 7f5effd39800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.151+0000 7f5effd39800 20 Filter name: none 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.151+0000 7f5daa7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.151+0000 7f5effd39800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.151+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55887e1f1700 con 0x55887e1aa7e0 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.151+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 281+0+332 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.151+0000 7f5effd39800 20 rados_obj.operate() r=0 bl.length=193 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.152+0000 7f5db27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.152+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94044719658720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1bd930 con 0x55887e1aa7e0 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.153+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94044719658720] v31'131 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.153+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94044719633456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x7f5ef4056050 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.154+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94044719633456] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.154+0000 7f5effd39800 20 remove_watcher() i=0 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.154+0000 7f5effd39800 2 removed watcher, disabling cache 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.154+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94044719639456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x55887e1aa7e0 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.155+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94044719639456] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.155+0000 7f5effd39800 20 remove_watcher() i=1 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.155+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94044719643232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x55887e1af1f0 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.156+0000 7f5efc1c8640 1 -- 172.21.15.28:0/3911901865 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94044719643232] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef400a4d0 con 0x55887e1af1f0 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.156+0000 7f5effd39800 20 remove_watcher() i=2 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.156+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94044719645440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x7f5ef4056050 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.157+0000 7f5efb9c7640 1 -- 172.21.15.28:0/3911901865 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94044719645440] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5eec039070 con 0x7f5ef4056050 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.157+0000 7f5effd39800 20 remove_watcher() i=3 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.157+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94044719649520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x55887e1aa7e0 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.158+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94044719649520] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x55887e1aa7e0 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.158+0000 7f5effd39800 20 remove_watcher() i=4 2024-09-19T11:23:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.158+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94044719654512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x7f5eec053530 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.159+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94044719654512] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.159+0000 7f5effd39800 20 remove_watcher() i=5 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.159+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94044719647184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x7f5eec053530 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.160+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94044719647184] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.160+0000 7f5effd39800 20 remove_watcher() i=6 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.160+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94044719656704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55887e1ef6b0 con 0x7f5eec053530 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94044719656704] v31'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5ef0051070 con 0x7f5eec053530 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 20 remove_watcher() i=7 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5eec053530 msgr2=0x7f5eec053970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5eec053530 0x7f5eec053970 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e1aa7e0 msgr2=0x55887e1acca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e1aa7e0 0x55887e1acca0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5ef4056050 msgr2=0x7f5ef401a0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5ef4056050 0x7f5ef401a0b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e1af1f0 msgr2=0x55887e1b15e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e1af1f0 0x55887e1b15e0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dc403d000 msgr2=0x7f5dc403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.161+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dc403d000 0x7f5dc403f4c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00efd0 tx=0x7f5eec041000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 msgr2=0x55887e19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e19ef20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4022ff0 tx=0x7f5ef4054770 comp rx=0 tx=0).stop 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5efb1c6640 1 -- 172.21.15.28:0/3911901865 reap_dead start 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 shutdown_connections 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/3911901865 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e197920 0x55887e19ef20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 >> 172.21.15.28:0/3911901865 conn(0x55887e193d20 msgr2=0x55887e191f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 shutdown_connections 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/3911901865 wait complete. 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dd403d000 msgr2=0x7f5dd403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dd403d000 0x7f5dd403f4c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0531e0 tx=0x7f5eec0011b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e18aec0 msgr2=0x55887e18b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e18aec0 0x55887e18b2e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55887e170f10 tx=0x7f5ef40167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 shutdown_connections 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5dd403d000 0x7f5dd403f4c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/714275791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e18aec0 0x55887e18b2e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 >> 172.21.15.28:0/714275791 conn(0x55887e17f270 msgr2=0x55887e17f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 shutdown_connections 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/714275791 wait complete. 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e076aa0 msgr2=0x55887e177560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e076aa0 0x55887e177560 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e179940 msgr2=0x55887e17bdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.162+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e179940 0x55887e17bdc0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5de403d1a0 msgr2=0x7f5de403f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5de403d1a0 0x7f5de403f660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0057000 tx=0x7f5ef0052000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 msgr2=0x55887e16ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887e16ff60 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0012f0 tx=0x7f5eec000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 shutdown_connections 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55887e179940 0x55887e17bdc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5de403d1a0 0x7f5de403f660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55887e076aa0 0x55887e177560 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 --2- 172.21.15.28:0/2199927775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55887e075690 0x55887e16ff60 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 >> 172.21.15.28:0/2199927775 conn(0x55887e070a60 msgr2=0x55887e15f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 shutdown_connections 2024-09-19T11:23:27.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.163+0000 7f5effd39800 1 -- 172.21.15.28:0/2199927775 wait complete. 2024-09-19T11:23:27.173 INFO:teuthology.orchestra.run.smithi028.stdout:{"key":"bucket:testx/myfoo","ver":{"tag":"kId/ksCy/dljAkMGvEjETiY5","ver":2},"mtime":"2024-09-19T11:23:27.018982Z","data":{"bucket":{"name":"myfoo","marker":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","bucket_id":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","tenant":"testx","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"testx$tenanteduser","creation_time":"2024-09-19T11:23:24.614937Z","linked":true,"has_bucket_info":false}} 2024-09-19T11:23:27.173 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'kId/ksCy/dljAkMGvEjETiY5', 'ver': 2}, 'mtime': '2024-09-19T11:23:27.018982Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'bucket_id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-09-19T11:23:24.614937Z', 'linked': True, 'has_bucket_info': False}} 2024-09-19T11:23:27.173 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-09-19T11:23:27.174 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-09-19T11:23:27.174 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-09-19T11:23:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:27.222 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:27.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.221+0000 7f7f45a0f800 1 Processor -- start 2024-09-19T11:23:27.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.221+0000 7f7f45a0f800 1 -- start start 2024-09-19T11:23:27.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.221+0000 7f7f45a0f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563cf1cc1860 0x563cf1cca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.221+0000 7f7f45a0f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563cf1cca8c0 con 0x563cf1cbfd40 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.221+0000 7f7f45a0f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x563cf1b4c7d0 con 0x563cf1cc1860 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.222+0000 7f7f40921640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563cf1cc1860 0x563cf1cca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.222+0000 7f7f41122640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x563cf1cbfd40 0x563cf1cc0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48380/0 (socket says 172.21.15.28:48380) 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.222+0000 7f7f41122640 1 -- 172.21.15.28:0/3790955420 learned_addr learned my addr 172.21.15.28:0/3790955420 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.222+0000 7f7f33fff640 1 -- 172.21.15.28:0/3790955420 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850112093 0 0) 0x563cf1cca8c0 con 0x563cf1cbfd40 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.222+0000 7f7f33fff640 1 -- 172.21.15.28:0/3790955420 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f2c001e10 con 0x563cf1cbfd40 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.222+0000 7f7f40921640 1 -- 172.21.15.28:0/3790955420 >> v1:172.21.15.22:6789/0 conn(0x563cf1cbfd40 legacy=0x563cf1cc0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.222+0000 7f7f40921640 1 -- 172.21.15.28:0/3790955420 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf1ccbaa0 con 0x563cf1cc1860 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f40921640 1 --2- 172.21.15.28:0/3790955420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cc1860 0x563cf1cca380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800a470 tx=0x7f7f38045920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e614c13721055ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f33fff640 1 -- 172.21.15.28:0/3790955420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38059070 con 0x563cf1cc1860 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f33fff640 1 -- 172.21.15.28:0/3790955420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f38054570 con 0x563cf1cc1860 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f33fff640 1 -- 172.21.15.28:0/3790955420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38054870 con 0x563cf1cc1860 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3790955420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cc1860 msgr2=0x563cf1cca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/3790955420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cc1860 0x563cf1cca380 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800a470 tx=0x7f7f38045920 comp rx=0 tx=0).stop 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3790955420 shutdown_connections 2024-09-19T11:23:27.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.224+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/3790955420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cc1860 0x563cf1cca380 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.225+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3790955420 >> 172.21.15.28:0/3790955420 conn(0x563cf1cbba70 msgr2=0x563cf1cbded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.225+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3790955420 shutdown_connections 2024-09-19T11:23:27.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.225+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3790955420 wait complete. 2024-09-19T11:23:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.241+0000 7f7f45a0f800 1 Processor -- start 2024-09-19T11:23:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.241+0000 7f7f45a0f800 1 -- start start 2024-09-19T11:23:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.241+0000 7f7f45a0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1db33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.241+0000 7f7f45a0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563cf1b59dd0 con 0x563cf1cbfd40 2024-09-19T11:23:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.241+0000 7f7f41122640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1db33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.241+0000 7f7f41122640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1db33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34568/0 (socket says 172.21.15.28:34568) 2024-09-19T11:23:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.241+0000 7f7f41122640 1 -- 172.21.15.28:0/2624284454 learned_addr learned my addr 172.21.15.28:0/2624284454 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.242+0000 7f7f41122640 1 -- 172.21.15.28:0/2624284454 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf1db39b0 con 0x563cf1cbfd40 2024-09-19T11:23:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.242+0000 7f7f41122640 1 --2- 172.21.15.28:0/2624284454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1db33c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7f34001e30 tx=0x7f7f3400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba4e1011cd0a332 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.242+0000 7f7e2f606640 1 -- 172.21.15.28:0/2624284454 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f34012020 con 0x563cf1cbfd40 2024-09-19T11:23:27.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.242+0000 7f7e2f606640 1 -- 172.21.15.28:0/2624284454 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3400e470 con 0x563cf1cbfd40 2024-09-19T11:23:27.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.242+0000 7f7e2f606640 1 -- 172.21.15.28:0/2624284454 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3400e790 con 0x563cf1cbfd40 2024-09-19T11:23:27.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 -- 172.21.15.28:0/2624284454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 msgr2=0x563cf1db33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/2624284454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1db33c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7f34001e30 tx=0x7f7f3400b780 comp rx=0 tx=0).stop 2024-09-19T11:23:27.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 -- 172.21.15.28:0/2624284454 shutdown_connections 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/2624284454 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1db33c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 -- 172.21.15.28:0/2624284454 >> 172.21.15.28:0/2624284454 conn(0x563cf1cbba70 msgr2=0x563cf1daed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 -- 172.21.15.28:0/2624284454 shutdown_connections 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 -- 172.21.15.28:0/2624284454 wait complete. 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.243+0000 7f7f45a0f800 1 Processor -- start 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f45a0f800 1 -- start start 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f45a0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1dbaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f45a0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563cf1ccad60 con 0x563cf1cbfd40 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f41122640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1dbaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f41122640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1dbaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34582/0 (socket says 172.21.15.28:34582) 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f41122640 1 -- 172.21.15.28:0/1451158643 learned_addr learned my addr 172.21.15.28:0/1451158643 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f41122640 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf1dbb430 con 0x563cf1cbfd40 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.244+0000 7f7f41122640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1dbaed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x563cf1db4350 tx=0x7f7f340040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.245+0000 7f7e2de03640 1 -- 172.21.15.28:0/1451158643 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f34012030 con 0x563cf1cbfd40 2024-09-19T11:23:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.245+0000 7f7e2de03640 1 -- 172.21.15.28:0/1451158643 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3401a7a0 con 0x563cf1cbfd40 2024-09-19T11:23:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.245+0000 7f7e2de03640 1 -- 172.21.15.28:0/1451158643 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3401aaa0 con 0x563cf1cbfd40 2024-09-19T11:23:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.245+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563cf1dbb6c0 con 0x563cf1cbfd40 2024-09-19T11:23:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.245+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf1dbc0c0 con 0x563cf1cbfd40 2024-09-19T11:23:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.246+0000 7f7e2de03640 1 -- 172.21.15.28:0/1451158643 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7f34015020 con 0x563cf1cbfd40 2024-09-19T11:23:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.246+0000 7f7e2de03640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e2803d000 0x7f7e2803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.246+0000 7f7e2de03640 1 -- 172.21.15.28:0/1451158643 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f7f34020070 con 0x563cf1cbfd40 2024-09-19T11:23:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.246+0000 7f7f40921640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e2803d000 0x7f7e2803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.246+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1cc1860 0x563cf1dc2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.246+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dbccb0 con 0x563cf1cc1860 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.247+0000 7f7f41923640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1cc1860 0x563cf1dc2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.247+0000 7f7f40921640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e2803d000 0x7f7e2803f4c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x563cf1c33f60 tx=0x7f7f38050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.247+0000 7f7f41923640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1cc1860 0x563cf1dc2520 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.247+0000 7f7f41923640 1 -- 172.21.15.28:0/1451158643 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1cc1860 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.247+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dbc5a0 con 0x563cf1cc1860 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.248+0000 7f7f41923640 1 -- 172.21.15.28:0/1451158643 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1cc1860 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.248+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dc4a10 0x563cf1dc6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.248+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dc74d0 con 0x563cf1dc4a10 2024-09-19T11:23:27.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.248+0000 7f7f41122640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dc4a10 0x563cf1dc6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.248+0000 7f7f41122640 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dc4a10 0x563cf1dc6e00 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.248+0000 7f7f41122640 1 -- 172.21.15.28:0/1451158643 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7f34018950 con 0x563cf1dc4a10 2024-09-19T11:23:27.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.248+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dc38b0 con 0x563cf1cc1860 2024-09-19T11:23:27.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.249+0000 7f7f41923640 1 -- 172.21.15.28:0/1451158643 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1cc1860 2024-09-19T11:23:27.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.249+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dc3d10 con 0x563cf1dc4a10 2024-09-19T11:23:27.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.249+0000 7f7f41122640 1 -- 172.21.15.28:0/1451158643 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7f34018950 con 0x563cf1dc4a10 2024-09-19T11:23:27.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.249+0000 7f7f45a0f800 1 Processor -- start 2024-09-19T11:23:27.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.249+0000 7f7f45a0f800 1 -- start start 2024-09-19T11:23:27.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f45a0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1dd5e20 0x563cf1dd6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f45a0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563cf1bb84a0 con 0x563cf1dd5e20 2024-09-19T11:23:27.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f41923640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1dd5e20 0x563cf1dd6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f41923640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1dd5e20 0x563cf1dd6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34592/0 (socket says 172.21.15.28:34592) 2024-09-19T11:23:27.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f41923640 1 -- 172.21.15.28:0/3197413223 learned_addr learned my addr 172.21.15.28:0/3197413223 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f41923640 1 -- 172.21.15.28:0/3197413223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf1dd6880 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f41923640 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1dd5e20 0x563cf1dd6240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x563cf1dbc290 tx=0x7f7f3c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7e26ffd640 1 -- 172.21.15.28:0/3197413223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c013640 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7e26ffd640 1 -- 172.21.15.28:0/3197413223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c0137e0 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7e26ffd640 1 -- 172.21.15.28:0/3197413223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c013ac0 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563cf1dd70b0 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.250+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf1dd7650 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.251+0000 7f7e26ffd640 1 -- 172.21.15.28:0/3197413223 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7f3c011040 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.251+0000 7f7e26ffd640 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e1c03d000 0x7f7e1c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.251+0000 7f7e26ffd640 1 -- 172.21.15.28:0/3197413223 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f7f3c00e040 con 0x563cf1dd5e20 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.251+0000 7f7f41122640 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e1c03d000 0x7f7e1c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f45a0f800 1 Processor -- start 2024-09-19T11:23:27.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f41122640 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e1c03d000 0x7f7e1c03f4c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f34052c40 tx=0x7f7f3400beb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f45a0f800 1 -- start start 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f45a0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f45a0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563cf1db3de0 con 0x563cf1de28c0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f41923640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f41923640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34600/0 (socket says 172.21.15.28:34600) 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f41923640 1 -- 172.21.15.28:0/571955149 learned_addr learned my addr 172.21.15.28:0/571955149 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.252+0000 7f7f41923640 1 -- 172.21.15.28:0/571955149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf1de32b0 con 0x563cf1de28c0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f41923640 1 --2- 172.21.15.28:0/571955149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de2cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c020000 tx=0x7f7f3c0204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae442c81428030c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7e24ff9640 1 -- 172.21.15.28:0/571955149 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c021440 con 0x563cf1de28c0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7e24ff9640 1 -- 172.21.15.28:0/571955149 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c052070 con 0x563cf1de28c0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7e24ff9640 1 -- 172.21.15.28:0/571955149 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c021600 con 0x563cf1de28c0 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f45a0f800 1 -- 172.21.15.28:0/571955149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 msgr2=0x563cf1de2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/571955149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de2cc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c020000 tx=0x7f7f3c0204d0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f45a0f800 1 -- 172.21.15.28:0/571955149 shutdown_connections 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/571955149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de2cc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f45a0f800 1 -- 172.21.15.28:0/571955149 >> 172.21.15.28:0/571955149 conn(0x563cf1ddece0 msgr2=0x563cf1ddcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f45a0f800 1 -- 172.21.15.28:0/571955149 shutdown_connections 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.253+0000 7f7f45a0f800 1 -- 172.21.15.28:0/571955149 wait complete. 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f45a0f800 1 Processor -- start 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f45a0f800 1 -- start start 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f45a0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f45a0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563cf1de38d0 con 0x563cf1de28c0 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f41923640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f41923640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34610/0 (socket says 172.21.15.28:34610) 2024-09-19T11:23:27.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 learned_addr learned my addr 172.21.15.28:0/279264350 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cf1dea440 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f41923640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de9ee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c013fd0 tx=0x7f7f3c054840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7e12ffd640 1 -- 172.21.15.28:0/279264350 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c020de0 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7e12ffd640 1 -- 172.21.15.28:0/279264350 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3c01fa90 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563cf1dea6d0 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.254+0000 7f7e12ffd640 1 -- 172.21.15.28:0/279264350 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c01fd70 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.255+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cf1deb260 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.255+0000 7f7e12ffd640 1 -- 172.21.15.28:0/279264350 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7f3c01f500 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.256+0000 7f7e12ffd640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e0c03cfb0 0x7f7e0c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.256+0000 7f7e12ffd640 1 -- 172.21.15.28:0/279264350 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f7f3c034c30 con 0x563cf1de28c0 2024-09-19T11:23:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.256+0000 7f7f41122640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e0c03cfb0 0x7f7e0c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.256+0000 7f7f41122640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e0c03cfb0 0x7f7e0c03f470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7f34018c60 tx=0x7f7f3404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.256+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.256+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1df57d0 0x563cf1df7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.257+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1df83e0 con 0x563cf1df57d0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.257+0000 7f7f40921640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1df57d0 0x563cf1df7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f40921640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1df57d0 0x563cf1df7c90 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f45a0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f45a0f800 20 realm 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1df8810 con 0x563cf1df57d0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f45a0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.258+0000 7f7f45a0f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1df90e0 con 0x563cf1df57d0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f45a0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1df9480 con 0x563cf1df57d0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.259+0000 7f7f45a0f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.260+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.260+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dfa1e0 0x563cf1dfc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.260+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfcbe0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.260+0000 7f7f41923640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dfa1e0 0x563cf1dfc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.260+0000 7f7f41923640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dfa1e0 0x563cf1dfc5d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.261+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.261+0000 7f7f45a0f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.261+0000 7f7f45a0f800 20 searching for the correct realm 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.261+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.261+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.261+0000 7f7f41923640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7f3c045620 0x7f7f3c012730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.261+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f3c045620 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.262+0000 7f7f41122640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7f3c045620 0x7f7f3c012730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.262+0000 7f7f41122640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7f3c045620 0x7f7f3c012730 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.262+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.262+0000 7f7f41122640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7f34017050 0x7f7f340163b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.262+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.263+0000 7f7f40921640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7f34017050 0x7f7f340163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.263+0000 7f7f40921640 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7f34017050 0x7f7f340163b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.263+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.263+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.264+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f3c045620 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.265+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.265+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.265+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.265+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.265+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.265+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f3c045620 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.266+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.267+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.267+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.267+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.267+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.267+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.267+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.267+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.268+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.268+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.268+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.268+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.268+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.268+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.268+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f3c045620 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.269+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.270+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.270+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.270+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.270+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f3c045620 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.270+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.270+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1dfa1e0 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f34017050 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x563cf1df57d0 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe510 con 0x7f7f3c045620 2024-09-19T11:23:27.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.271+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfd5a0 con 0x563cf1df57d0 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfddf0 con 0x563cf1df57d0 2024-09-19T11:23:27.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfd1e0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 zone default found 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 4 Realm: () 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 zonegroup default 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.272+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dfe990 con 0x7f7f34017050 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e00770 con 0x563cf1dfa1e0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e00c10 con 0x563cf1dfa1e0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 20 started sync module instance, tier type = 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.273+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dffc10 con 0x7f7f3c045620 2024-09-19T11:23:27.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.274+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.274+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94819756015616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1dff870 con 0x7f7f3c045620 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.274+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e01a30 con 0x563cf1df57d0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.275+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94819756015616] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.275+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.275+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94819756019792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e01e20 con 0x563cf1df57d0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.275+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e02880 con 0x563cf1dfa1e0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.276+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94819756019792] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.277+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.277+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94819756024624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e02840 con 0x563cf1dfa1e0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.277+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e03330 con 0x7f7f3c045620 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.278+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94819756024624] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.278+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.278+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94819756027456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e03720 con 0x7f7f3c045620 2024-09-19T11:23:27.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.278+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e04140 con 0x563cf1df57d0 2024-09-19T11:23:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.279+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94819756027456] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.279+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.279+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94819756031024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e04560 con 0x563cf1df57d0 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.279+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e04fa0 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.280+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94819756031024] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.280+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.280+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94819756034672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e04f60 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.280+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e05a50 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.281+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94819756034672] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.281+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.281+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94819756020848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e05a50 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.281+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e06410 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94819756040032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e06830 con 0x7f7f34017050 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 20 add_watcher() i=0 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 20 add_watcher() i=1 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 20 add_watcher() i=2 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 20 add_watcher() i=3 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 20 add_watcher() i=4 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 20 add_watcher() i=5 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94819756020848] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.282+0000 7f7f45a0f800 20 add_watcher() i=6 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.283+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94819756040032] v31'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3805c040 con 0x7f7f34017050 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.283+0000 7f7f45a0f800 20 add_watcher() i=7 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.283+0000 7f7f45a0f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.283+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e06830 con 0x563cf1df57d0 2024-09-19T11:23:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.284+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7f3805c040 con 0x563cf1df57d0 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.284+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94819756041088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e067a0 con 0x563cf1df57d0 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94819756041088] v31'132 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7df7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 5 note: GC not initialized 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7df6fe5640 20 reqs_thread_entry: start 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 20 Filter name: none 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7df4fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.285+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3c740 con 0x563cf1df57d0 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3c1a0 con 0x563cf1df57d0 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) ==== 197+0+241 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 20 rados_obj.operate() r=0 bl.length=193 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.286+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 8.a 8:532a8d66:root::.bucket.meta.testx%3amyfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3fe80 con 0x563cf1df57d0 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~420 out=420b,getxattrs out=298b] v0'0 uv1 ondisk = 0) ==== 338+0+782 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 20 rados_obj.operate() r=0 bl.length=420 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.287+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e44540 con 0x563cf1df57d0 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.288+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v31'2 uv2 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.288+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [create,call version.inc in=38b,writefull 0~387 in=387b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e47e50 con 0x7f7f34017050 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(66 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [create,call,writefull 0~387,setxattr (123)] v31'7 uv7 ondisk = 0) ==== 332+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f45a0f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x7 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f45a0f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f45a0f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f45a0f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94819756301296 in=748b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e47910 con 0x7f7f34017050 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== watch-notify(notify (1) cookie 94819756034672 notify 133143986189 ret 0) ==== 778+0+0 (crc 0 0 0) 0x563cf1e44250 con 0x7f7f34017050 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7e24ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986189 cookie 94819756034672 notifier 4558 bl.length()=736 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7e24ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x7 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7e24ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7e24ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(67 notify.5 [notify cookie 94819756301296 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.289+0000 7f7e24ff9640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f7e04003ea0 con 0x7f7f34017050 2024-09-19T11:23:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.290+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.290+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== watch-notify(notify_complete (2) cookie 94819756301296 notify 133143986189 ret 0) ==== 42+0+48 (crc 0 0 0) 0x563cf1e40270 con 0x7f7f34017050 2024-09-19T11:23:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.290+0000 7f7f45a0f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_sources= new_sources= 2024-09-19T11:23:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.290+0000 7f7f45a0f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential sources added=[] removed=[] 2024-09-19T11:23:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.290+0000 7f7f45a0f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_dests= new_dests= 2024-09-19T11:23:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.290+0000 7f7f45a0f800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential dests added=[] removed=[] 2024-09-19T11:23:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.290+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e47a30 con 0x7f7f3c045620 2024-09-19T11:23:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.291+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(69 foo.buckets [call] v31'26 uv26 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.291+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e47a30 con 0x7f7f3c045620 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v31'27 uv27 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7f45a0f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7f45a0f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7f45a0f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94819756307840 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e48310 con 0x7f7f34017050 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== watch-notify(notify (1) cookie 94819756020848 notify 133143986190 ret 0) ==== 356+0+0 (crc 0 0 0) 0x563cf1e3e3a0 con 0x7f7f34017050 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7e24ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986190 cookie 94819756020848 notifier 4558 bl.length()=314 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7e24ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7e24ff9640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7e24ff9640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f7e04003ea0 con 0x7f7f34017050 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.292+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(71 notify.6 [notify cookie 94819756307840 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.293+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.293+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== watch-notify(notify_complete (2) cookie 94819756307840 notify 133143986190 ret 0) ==== 42+0+48 (crc 0 0 0) 0x563cf1ccbaa0 con 0x7f7f34017050 2024-09-19T11:23:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.293+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e47a30 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(73 testx/myfoo [call,delete] v31'2 uv2 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7f45a0f800 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7f45a0f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify cookie 94819756307840 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e44be0 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify (1) cookie 94819756031024 notify 133143986189 ret 0) ==== 197+0+0 (crc 0 0 0) 0x563cf1ccbaa0 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7e13fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986189 cookie 94819756031024 notifier 4558 bl.length()=155 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(74 notify.4 [notify cookie 94819756307840 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.294+0000 7f7e13fff640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f7d680032c0 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.295+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== watch-notify(notify_complete (2) cookie 94819756307840 notify 133143986189 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f7f38082650 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.295+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(75 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.295+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 8.a 8:532a8d66:root::.bucket.meta.testx%3amyfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e479b0 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(76 .bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call,delete] v31'2 uv2 ondisk = 0) ==== 254+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f45a0f800 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 from cache 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f45a0f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [notify cookie 94819756279104 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e479b0 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== watch-notify(notify (1) cookie 94819756031024 notify 133143986190 ret 0) ==== 254+0+0 (crc 0 0 0) 0x7f7f38082650 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7e13fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986190 cookie 94819756031024 notifier 4558 bl.length()=212 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7e13fff640 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f7d68003330 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(77 notify.4 [notify cookie 94819756279104 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== watch-notify(notify_complete (2) cookie 94819756279104 notify 133143986190 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f7f38082910 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.296+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(78 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.298+0000 7f7df7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.298+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94819756041088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.298+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94819756041088] v31'133 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.299+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94819756015616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x7f7f3c045620 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.299+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94819756015616] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.299+0000 7f7f45a0f800 20 remove_watcher() i=0 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.299+0000 7f7f45a0f800 2 removed watcher, disabling cache 2024-09-19T11:23:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.300+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94819756019792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x563cf1df57d0 2024-09-19T11:23:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.301+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94819756019792] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.301+0000 7f7f45a0f800 20 remove_watcher() i=1 2024-09-19T11:23:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.301+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94819756024624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.302+0000 7f7f41923640 1 -- 172.21.15.28:0/279264350 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94819756024624] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f3c00a4d0 con 0x563cf1dfa1e0 2024-09-19T11:23:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.302+0000 7f7f45a0f800 20 remove_watcher() i=2 2024-09-19T11:23:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.302+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94819756027456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x7f7f3c045620 2024-09-19T11:23:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.303+0000 7f7f41122640 1 -- 172.21.15.28:0/279264350 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94819756027456] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f34018950 con 0x7f7f3c045620 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.303+0000 7f7f45a0f800 20 remove_watcher() i=3 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.303+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94819756031024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x563cf1df57d0 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.304+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94819756031024] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x563cf1df57d0 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.304+0000 7f7f45a0f800 20 remove_watcher() i=4 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.304+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94819756034672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x7f7f34017050 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.305+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94819756034672] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.305+0000 7f7f45a0f800 20 remove_watcher() i=5 2024-09-19T11:23:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.305+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94819756020848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x7f7f34017050 2024-09-19T11:23:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.306+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94819756020848] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.306+0000 7f7f45a0f800 20 remove_watcher() i=6 2024-09-19T11:23:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.306+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94819756040032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x563cf1e3a5c0 con 0x7f7f34017050 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94819756040032] v31'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f38080040 con 0x7f7f34017050 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 20 remove_watcher() i=7 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7f34017050 msgr2=0x7f7f340163b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7f34017050 0x7f7f340163b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1df57d0 msgr2=0x563cf1df7c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1df57d0 0x563cf1df7c90 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7f3c045620 msgr2=0x7f7f3c012730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f7f3c045620 0x7f7f3c012730 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dfa1e0 msgr2=0x563cf1dfc5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dfa1e0 0x563cf1dfc5d0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e0c03cfb0 msgr2=0x7f7e0c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e0c03cfb0 0x7f7e0c03f470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7f34018c60 tx=0x7f7f3404f000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 msgr2=0x563cf1de9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/279264350 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1de28c0 0x563cf1de9ee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c013fd0 tx=0x7f7f3c054840 comp rx=0 tx=0).stop 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 reap_dead start 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f40921640 1 -- 172.21.15.28:0/279264350 reap_dead start 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 shutdown_connections 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 >> 172.21.15.28:0/279264350 conn(0x563cf1ddece0 msgr2=0x563cf1ddcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 shutdown_connections 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/279264350 wait complete. 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e1c03d000 msgr2=0x7f7e1c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e1c03d000 0x7f7e1c03f4c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f34052c40 tx=0x7f7f3400beb0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1dd5e20 msgr2=0x563cf1dd6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1dd5e20 0x563cf1dd6240 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x563cf1dbc290 tx=0x7f7f3c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 shutdown_connections 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e1c03d000 0x7f7e1c03f4c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/3197413223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1dd5e20 0x563cf1dd6240 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.307+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 >> 172.21.15.28:0/3197413223 conn(0x563cf1dca1d0 msgr2=0x563cf1dca5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 shutdown_connections 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/3197413223 wait complete. 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1cc1860 msgr2=0x563cf1dc2520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1cc1860 0x563cf1dc2520 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dc4a10 msgr2=0x563cf1dc6e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dc4a10 0x563cf1dc6e00 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e2803d000 msgr2=0x7f7e2803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e2803d000 0x7f7e2803f4c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x563cf1c33f60 tx=0x7f7f38050000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 msgr2=0x563cf1dbaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1dbaed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x563cf1db4350 tx=0x7f7f340040e0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 shutdown_connections 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563cf1dc4a10 0x563cf1dc6e00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7e2803d000 0x7f7e2803f4c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563cf1cc1860 0x563cf1dc2520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 --2- 172.21.15.28:0/1451158643 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563cf1cbfd40 0x563cf1dbaed0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 >> 172.21.15.28:0/1451158643 conn(0x563cf1cbba70 msgr2=0x563cf1db48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 shutdown_connections 2024-09-19T11:23:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.308+0000 7f7f45a0f800 1 -- 172.21.15.28:0/1451158643 wait complete. 2024-09-19T11:23:27.319 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-19T11:23:27.319 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-19T11:23:27.319 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-09-19T11:23:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.366+0000 7f0266b46800 1 Processor -- start 2024-09-19T11:23:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.366+0000 7f0266b46800 1 -- start start 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0266b46800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55b051a47ab0 0x55b051a49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0266b46800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b051a4a5f0 con 0x55b051a43f40 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0266b46800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55b0518ce7d0 con 0x55b051a47ab0 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0263e36640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55b051a47ab0 0x55b051a49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0264637640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55b051a43f40 0x55b051a46330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48396/0 (socket says 172.21.15.28:48396) 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0263e36640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55b051a47ab0 0x55b051a49f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34626/0 (socket says 172.21.15.28:34626) 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0264637640 1 -- 172.21.15.28:0/4008900571 learned_addr learned my addr 172.21.15.28:0/4008900571 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0263e36640 1 -- 172.21.15.28:0/4008900571 >> v1:172.21.15.22:6789/0 conn(0x55b051a43f40 legacy=0x55b051a46330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0263e36640 1 -- 172.21.15.28:0/4008900571 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b051a4b820 con 0x55b051a47ab0 2024-09-19T11:23:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.367+0000 7f0263635640 1 -- 172.21.15.28:0/4008900571 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3725318346 0 0) 0x55b051a4a5f0 con 0x55b051a43f40 2024-09-19T11:23:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.369+0000 7f0263e36640 1 --2- 172.21.15.28:0/4008900571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a47ab0 0x55b051a49f70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f025800a470 tx=0x7f0258045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7c0d3c40a4988b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.369+0000 7f0263635640 1 -- 172.21.15.28:0/4008900571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0258053570 con 0x55b051a47ab0 2024-09-19T11:23:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.369+0000 7f0263635640 1 -- 172.21.15.28:0/4008900571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0258058070 con 0x55b051a47ab0 2024-09-19T11:23:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.369+0000 7f0263635640 1 -- 172.21.15.28:0/4008900571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0258053880 con 0x55b051a47ab0 2024-09-19T11:23:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.370+0000 7f0266b46800 1 -- 172.21.15.28:0/4008900571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a47ab0 msgr2=0x55b051a49f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.370+0000 7f0266b46800 1 --2- 172.21.15.28:0/4008900571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a47ab0 0x55b051a49f70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f025800a470 tx=0x7f0258045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.370+0000 7f0266b46800 1 -- 172.21.15.28:0/4008900571 shutdown_connections 2024-09-19T11:23:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.370+0000 7f0266b46800 1 --2- 172.21.15.28:0/4008900571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a47ab0 0x55b051a49f70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.370+0000 7f0266b46800 1 -- 172.21.15.28:0/4008900571 >> 172.21.15.28:0/4008900571 conn(0x55b051a3daa0 msgr2=0x55b051a3fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.370+0000 7f0266b46800 1 -- 172.21.15.28:0/4008900571 shutdown_connections 2024-09-19T11:23:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.370+0000 7f0266b46800 1 -- 172.21.15.28:0/4008900571 wait complete. 2024-09-19T11:23:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.385+0000 7f0266b46800 1 Processor -- start 2024-09-19T11:23:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.385+0000 7f0266b46800 1 -- start start 2024-09-19T11:23:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.385+0000 7f0266b46800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b33120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.385+0000 7f0266b46800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b0518dbdd0 con 0x55b051a43f40 2024-09-19T11:23:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.385+0000 7f0264637640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b33120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.386+0000 7f0264637640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b33120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34630/0 (socket says 172.21.15.28:34630) 2024-09-19T11:23:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.386+0000 7f0264637640 1 -- 172.21.15.28:0/663723692 learned_addr learned my addr 172.21.15.28:0/663723692 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.386+0000 7f0264637640 1 -- 172.21.15.28:0/663723692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b051b33710 con 0x55b051a43f40 2024-09-19T11:23:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.386+0000 7f0264637640 1 --2- 172.21.15.28:0/663723692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b33120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0254001ea0 tx=0x7f025400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d94af6362c3948 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.386+0000 7f015260c640 1 -- 172.21.15.28:0/663723692 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0254008070 con 0x55b051a43f40 2024-09-19T11:23:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.386+0000 7f015260c640 1 -- 172.21.15.28:0/663723692 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025400d500 con 0x55b051a43f40 2024-09-19T11:23:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.386+0000 7f015260c640 1 -- 172.21.15.28:0/663723692 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025400d880 con 0x55b051a43f40 2024-09-19T11:23:27.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 -- 172.21.15.28:0/663723692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 msgr2=0x55b051b33120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 --2- 172.21.15.28:0/663723692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b33120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0254001ea0 tx=0x7f025400b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 -- 172.21.15.28:0/663723692 shutdown_connections 2024-09-19T11:23:27.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 --2- 172.21.15.28:0/663723692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b33120 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 -- 172.21.15.28:0/663723692 >> 172.21.15.28:0/663723692 conn(0x55b051a3daa0 msgr2=0x55b051b2eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 -- 172.21.15.28:0/663723692 shutdown_connections 2024-09-19T11:23:27.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 -- 172.21.15.28:0/663723692 wait complete. 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 Processor -- start 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 -- start start 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.387+0000 7f0266b46800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b051a4aac0 con 0x55b051a43f40 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0264637640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0264637640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b3ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34636/0 (socket says 172.21.15.28:34636) 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0264637640 1 -- 172.21.15.28:0/3357939628 learned_addr learned my addr 172.21.15.28:0/3357939628 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0264637640 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b051b3b190 con 0x55b051a43f40 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0264637640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b3ac30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f02540012f0 tx=0x7f0254000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0150e09640 1 -- 172.21.15.28:0/3357939628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025400f9d0 con 0x55b051a43f40 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0150e09640 1 -- 172.21.15.28:0/3357939628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025400fb70 con 0x55b051a43f40 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b051b3b420 con 0x55b051a43f40 2024-09-19T11:23:27.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0150e09640 1 -- 172.21.15.28:0/3357939628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025400fe50 con 0x55b051a43f40 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.388+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b051b3be20 con 0x55b051a43f40 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.389+0000 7f0150e09640 1 -- 172.21.15.28:0/3357939628 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0254017040 con 0x55b051a43f40 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.389+0000 7f0150e09640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f014c03d1a0 0x7f014c03f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.389+0000 7f0150e09640 1 -- 172.21.15.28:0/3357939628 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f0254015020 con 0x55b051a43f40 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.389+0000 7f0263e36640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f014c03d1a0 0x7f014c03f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.390+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051a47ab0 0x55b051b42280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.390+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b427c0 con 0x55b051a47ab0 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.390+0000 7f0264e38640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051a47ab0 0x55b051b42280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.390+0000 7f0263e36640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f014c03d1a0 0x7f014c03f660 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0258057000 tx=0x7f0258052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.390+0000 7f0264e38640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051a47ab0 0x55b051b42280 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.390+0000 7f0264e38640 1 -- 172.21.15.28:0/3357939628 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051a47ab0 2024-09-19T11:23:27.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.390+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b42b60 con 0x55b051a47ab0 2024-09-19T11:23:27.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.391+0000 7f0264e38640 1 -- 172.21.15.28:0/3357939628 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051a47ab0 2024-09-19T11:23:27.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.391+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b44660 0x55b051b46ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.391+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b47270 con 0x55b051b44660 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.391+0000 7f0264637640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b44660 0x55b051b46ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.391+0000 7f0264637640 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b44660 0x55b051b46ae0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.391+0000 7f0264637640 1 -- 172.21.15.28:0/3357939628 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f025402d070 con 0x55b051b44660 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.391+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b433b0 con 0x55b051a47ab0 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.392+0000 7f0264e38640 1 -- 172.21.15.28:0/3357939628 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051a47ab0 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.392+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b43800 con 0x55b051b44660 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.392+0000 7f0264637640 1 -- 172.21.15.28:0/3357939628 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f025402d070 con 0x55b051b44660 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.392+0000 7f0266b46800 1 Processor -- start 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.392+0000 7f0266b46800 1 -- start start 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.392+0000 7f0266b46800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a436c0 0x55b051a41f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.392+0000 7f0266b46800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b05193a4a0 con 0x55b051a436c0 2024-09-19T11:23:27.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0264e38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a436c0 0x55b051a41f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0264e38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a436c0 0x55b051a41f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34648/0 (socket says 172.21.15.28:34648) 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0264e38640 1 -- 172.21.15.28:0/3181641862 learned_addr learned my addr 172.21.15.28:0/3181641862 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0264e38640 1 -- 172.21.15.28:0/3181641862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b051a43be0 con 0x55b051a436c0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0264e38640 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a436c0 0x55b051a41f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55b051b3bc30 tx=0x7f025c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0149ffb640 1 -- 172.21.15.28:0/3181641862 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c013640 con 0x55b051a436c0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0149ffb640 1 -- 172.21.15.28:0/3181641862 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c0137e0 con 0x55b051a436c0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0149ffb640 1 -- 172.21.15.28:0/3181641862 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c013ae0 con 0x55b051a436c0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b051a42470 con 0x55b051a436c0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.393+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b051a42a10 con 0x55b051a436c0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.394+0000 7f0149ffb640 1 -- 172.21.15.28:0/3181641862 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f025c011040 con 0x55b051a436c0 2024-09-19T11:23:27.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.394+0000 7f0149ffb640 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f013c03cfb0 0x7f013c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.394+0000 7f0149ffb640 1 -- 172.21.15.28:0/3181641862 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f025c00e040 con 0x55b051a436c0 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.394+0000 7f0264637640 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f013c03cfb0 0x7f013c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0266b46800 1 Processor -- start 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0266b46800 1 -- start start 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0264637640 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f013c03cfb0 0x7f013c03f470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f02540531e0 tx=0x7f02540011b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0266b46800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b64cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0266b46800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b051b33b40 con 0x55b051b648b0 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0264e38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b64cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0264e38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b64cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34658/0 (socket says 172.21.15.28:34658) 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.395+0000 7f0264e38640 1 -- 172.21.15.28:0/2005871932 learned_addr learned my addr 172.21.15.28:0/2005871932 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0264e38640 1 -- 172.21.15.28:0/2005871932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b051b652a0 con 0x55b051b648b0 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0264e38640 1 --2- 172.21.15.28:0/2005871932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b64cb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f025c054840 tx=0x7f025c014480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67b067a53e3a958 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f013b7fe640 1 -- 172.21.15.28:0/2005871932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c021a40 con 0x55b051b648b0 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f013b7fe640 1 -- 172.21.15.28:0/2005871932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c021440 con 0x55b051b648b0 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f013b7fe640 1 -- 172.21.15.28:0/2005871932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c01f340 con 0x55b051b648b0 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0266b46800 1 -- 172.21.15.28:0/2005871932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 msgr2=0x55b051b64cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0266b46800 1 --2- 172.21.15.28:0/2005871932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b64cb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f025c054840 tx=0x7f025c014480 comp rx=0 tx=0).stop 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0266b46800 1 -- 172.21.15.28:0/2005871932 shutdown_connections 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0266b46800 1 --2- 172.21.15.28:0/2005871932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b64cb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0266b46800 1 -- 172.21.15.28:0/2005871932 >> 172.21.15.28:0/2005871932 conn(0x55b051b60cb0 msgr2=0x55b051b5eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0266b46800 1 -- 172.21.15.28:0/2005871932 shutdown_connections 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.396+0000 7f0266b46800 1 -- 172.21.15.28:0/2005871932 wait complete. 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0266b46800 1 Processor -- start 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0266b46800 1 -- start start 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0266b46800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b6be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0266b46800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b051b658c0 con 0x55b051b648b0 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0264e38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b6be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0264e38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b6be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34674/0 (socket says 172.21.15.28:34674) 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 learned_addr learned my addr 172.21.15.28:0/1894852112 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.397+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b051b6c3e0 con 0x55b051b648b0 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.398+0000 7f0264e38640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b6be80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f025c020790 tx=0x7f025c0547a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.398+0000 7f0139ffb640 1 -- 172.21.15.28:0/1894852112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c013c80 con 0x55b051b648b0 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.398+0000 7f0139ffb640 1 -- 172.21.15.28:0/1894852112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c013e20 con 0x55b051b648b0 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.398+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b051b6c670 con 0x55b051b648b0 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.398+0000 7f0139ffb640 1 -- 172.21.15.28:0/1894852112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f025c054950 con 0x55b051b648b0 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.398+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b051b6cff0 con 0x55b051b648b0 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.398+0000 7f0139ffb640 1 -- 172.21.15.28:0/1894852112 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f025c054af0 con 0x55b051b648b0 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.399+0000 7f0139ffb640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f012c03d000 0x7f012c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.399+0000 7f0139ffb640 1 -- 172.21.15.28:0/1894852112 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f025c004ad0 con 0x55b051b648b0 2024-09-19T11:23:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.399+0000 7f0264637640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f012c03d000 0x7f012c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.399+0000 7f0264637640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f012c03d000 0x7f012c03f4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f025400efd0 tx=0x7f0254041000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.399+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.399+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051b77730 0x55b051b79bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.399+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b7a340 con 0x55b051b77730 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0263e36640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051b77730 0x55b051b79bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0263e36640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051b77730 0x55b051b79bf0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0266b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0266b46800 20 realm 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b7a770 con 0x55b051b77730 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.400+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b7b040 con 0x55b051b77730 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b7b3e0 con 0x55b051b77730 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b7c140 0x55b051b7e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b7eb40 con 0x55b051b7c140 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.401+0000 7f0264e38640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b7c140 0x55b051b7e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0264e38640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b7c140 0x55b051b7e530 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0266b46800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0266b46800 20 searching for the correct realm 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0264e38640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f025c033440 0x7f025c0451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f025c033440 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.402+0000 7f0264637640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f025c033440 0x7f025c0451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.403+0000 7f0264637640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f025c033440 0x7f025c0451d0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.403+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.403+0000 7f0264637640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02540534f0 0x7f0254053910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.403+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.403+0000 7f0263e36640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02540534f0 0x7f0254053910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.403+0000 7f0263e36640 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02540534f0 0x7f0254053910 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.404+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f025c033440 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.405+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f025c033440 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.406+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.407+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.408+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f025c033440 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.409+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.409+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.409+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.409+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.409+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.409+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.409+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.410+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.410+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.410+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f025c033440 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.410+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.410+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b7c140 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.410+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.410+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f02540534f0 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x55b051b77730 2024-09-19T11:23:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80470 con 0x7f025c033440 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b7f500 con 0x55b051b77730 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.411+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b7fd50 con 0x55b051b77730 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b808f0 con 0x55b051b7c140 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 zone default found 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 4 Realm: () 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 zonegroup default 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b80cf0 con 0x7f02540534f0 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.412+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b82b20 con 0x55b051b7c140 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0266b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b83000 con 0x55b051b7c140 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0266b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0266b46800 20 started sync module instance, tier type = 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0266b46800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.413+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b81fc0 con 0x7f025c033440 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.414+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.414+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94215773627312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b81c20 con 0x7f025c033440 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.414+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b83e20 con 0x55b051b77730 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.415+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94215773627312] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.415+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.415+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94215773631584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b84210 con 0x55b051b77730 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.415+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b84c70 con 0x55b051b7c140 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.416+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94215773631584] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.416+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.416+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94215773636384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b84c30 con 0x55b051b7c140 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.416+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b85700 con 0x7f025c033440 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.417+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94215773636384] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.417+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.417+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94215773639120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b85af0 con 0x7f025c033440 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.417+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b86530 con 0x55b051b77730 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.418+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.418+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94215773642752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b86dd0 con 0x55b051b77730 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.418+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b878d0 con 0x7f02540534f0 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.418+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94215773639120] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.419+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94215773642752] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.419+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.419+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94215773647808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b868d0 con 0x7f02540534f0 2024-09-19T11:23:27.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.419+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b88170 con 0x7f02540534f0 2024-09-19T11:23:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.420+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94215773647808] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.420+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.420+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94215773632640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b868d0 con 0x7f02540534f0 2024-09-19T11:23:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.420+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b88790 con 0x7f02540534f0 2024-09-19T11:23:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94215773632640] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94215773649968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b88790 con 0x7f02540534f0 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=0 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=1 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=2 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=3 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=4 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=5 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=6 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94215773649968] v31'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0258051070 con 0x7f02540534f0 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 20 add_watcher() i=7 2024-09-19T11:23:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.422+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051b8bae0 con 0x55b051b77730 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.423+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.423+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94215773664496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b88900 con 0x55b051b77730 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.424+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94215773664496] v31'134 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0258051070 con 0x55b051b77730 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.424+0000 7f011a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.424+0000 7f0266b46800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.424+0000 7f0266b46800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.424+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b94310 con 0x55b051b77730 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.425+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.425+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b949e0 con 0x7f02540534f0 2024-09-19T11:23:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.426+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.426+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b94e40 con 0x55b051b7c140 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.427+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.427+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b94d30 con 0x55b051b77730 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.428+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'135 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.428+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b952e0 con 0x7f025c033440 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.429+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'38 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.429+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95850 con 0x55b051b7c140 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.430+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'25 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.430+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95850 con 0x55b051b77730 2024-09-19T11:23:27.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.431+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.431+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95850 con 0x7f02540534f0 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.432+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'25 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.432+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b958e0 con 0x7f025c033440 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.433+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.433+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95eb0 con 0x55b051b7c140 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.434+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'22 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.434+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x7f025c033440 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.435+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'39 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.435+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x7f02540534f0 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.436+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.436+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x7f025c033440 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.437+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.437+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x7f02540534f0 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.438+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'32 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.438+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b77730 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.439+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'136 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.439+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b7c140 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.440+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'32 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.440+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x7f025c033440 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.441+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'40 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.441+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b7c140 2024-09-19T11:23:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.442+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.442+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b77730 2024-09-19T11:23:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.443+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.443+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b77730 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.444+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'137 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.444+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b7c140 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.445+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'33 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.445+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x7f025c033440 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.446+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.446+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b7c140 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.447+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.447+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b7c140 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.448+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.448+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b7c140 2024-09-19T11:23:27.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.449+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.449+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95e60 con 0x55b051b7c140 2024-09-19T11:23:27.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.450+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'26 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.450+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95ec0 con 0x7f02540534f0 2024-09-19T11:23:27.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.451+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.451+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95ef0 con 0x55b051b7c140 2024-09-19T11:23:27.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.452+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.452+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95f30 con 0x7f02540534f0 2024-09-19T11:23:27.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.453+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'26 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.453+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95ef0 con 0x55b051b7c140 2024-09-19T11:23:27.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.454+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'34 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.454+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95ef0 con 0x7f02540534f0 2024-09-19T11:23:27.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.455+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'33 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.455+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b95f50 con 0x7f025c033440 2024-09-19T11:23:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.456+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.456+0000 7f01197e2640 20 reqs_thread_entry: start 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.456+0000 7f0266b46800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.456+0000 7f0266b46800 20 Filter name: none 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.456+0000 7f008affd640 20 reqs_thread_entry: start 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.456+0000 7f0266b46800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.456+0000 7f0266b46800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:94 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc0e70 con 0x7f02540534f0 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(94 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 20 rados_obj.operate() r=0 bl.length=419 2024-09-19T11:23:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-09-19T11:23:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-19T11:23:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc3220 con 0x55b051b77730 2024-09-19T11:23:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(95 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 170+0+15 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-09-19T11:23:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.457+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc10f0 con 0x7f02540534f0 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.458+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v31'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.458+0000 7f0266b46800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.458+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 94215773883568 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc39b0 con 0x55b051b7c140 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== watch-notify(notify (1) cookie 94215773636384 notify 133143986181 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55b051b93d60 con 0x55b051b7c140 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f013b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986181 cookie 94215773636384 notifier 4565 bl.length()=170 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(97 notify.2 [notify cookie 94215773883568 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f013b7fe640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f0128003d60 con 0x55b051b7c140 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== watch-notify(notify_complete (2) cookie 94215773883568 notify 133143986181 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55b051b6c3e0 con 0x55b051b7c140 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f0266b46800 10 removing email index: 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f0266b46800 10 removing user buckets index 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.459+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:99 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc0c80 con 0x55b051b77730 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.460+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(99 testx$tenanteduser.buckets [delete] v31'3 uv3 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.460+0000 7f0266b46800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.460+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [notify cookie 94215773883456 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc39b0 con 0x55b051b77730 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.460+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== watch-notify(notify (1) cookie 94215773642752 notify 133143986191 ret 0) ==== 217+0+0 (crc 0 0 0) 0x55b051b949e0 con 0x55b051b77730 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.460+0000 7f013affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986191 cookie 94215773642752 notifier 4565 bl.length()=175 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.460+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(100 notify.4 [notify cookie 94215773883456 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.460+0000 7f013affd640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f00980032c0 con 0x55b051b77730 2024-09-19T11:23:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.461+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(101 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.461+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== watch-notify(notify_complete (2) cookie 94215773883456 notify 133143986191 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55b051b6c670 con 0x55b051b77730 2024-09-19T11:23:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.461+0000 7f0266b46800 10 removing user index: testx$tenanteduser 2024-09-19T11:23:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.461+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc5870 con 0x7f02540534f0 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(102 testx$tenanteduser [call,call,delete] v31'8 uv8 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f0266b46800 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f0266b46800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:103 7.e 7:7759931f:::notify.3:head [notify cookie 94215773885312 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55b051bc2c60 con 0x7f025c033440 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== watch-notify(notify (1) cookie 94215773639120 notify 133143986195 ret 0) ==== 209+0+0 (crc 0 0 0) 0x55b051b952a0 con 0x7f025c033440 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f013b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986195 cookie 94215773639120 notifier 4565 bl.length()=167 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f013b7fe640 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f0128003d60 con 0x7f025c033440 2024-09-19T11:23:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.462+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(103 notify.3 [notify cookie 94215773885312 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.463+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== watch-notify(notify_complete (2) cookie 94215773885312 notify 133143986195 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55b051b94d30 con 0x7f025c033440 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.463+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(104 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.464+0000 7f011a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.464+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94215773664496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b91b80 con 0x55b051b77730 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.465+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94215773664496] v31'138 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.465+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94215773627312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b91b80 con 0x7f025c033440 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.466+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94215773627312] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.466+0000 7f0266b46800 20 remove_watcher() i=0 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.466+0000 7f0266b46800 2 removed watcher, disabling cache 2024-09-19T11:23:27.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.466+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94215773631584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b823b0 con 0x55b051b77730 2024-09-19T11:23:27.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.467+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94215773631584] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.467+0000 7f0266b46800 20 remove_watcher() i=1 2024-09-19T11:23:27.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.467+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94215773636384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b823b0 con 0x55b051b7c140 2024-09-19T11:23:27.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.468+0000 7f0264e38640 1 -- 172.21.15.28:0/1894852112 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94215773636384] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025c00a4d0 con 0x55b051b7c140 2024-09-19T11:23:27.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.468+0000 7f0266b46800 20 remove_watcher() i=2 2024-09-19T11:23:27.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.468+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94215773639120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b84720 con 0x7f025c033440 2024-09-19T11:23:27.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.469+0000 7f0264637640 1 -- 172.21.15.28:0/1894852112 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94215773639120] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0254039070 con 0x7f025c033440 2024-09-19T11:23:27.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.469+0000 7f0266b46800 20 remove_watcher() i=3 2024-09-19T11:23:27.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.469+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94215773642752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b823b0 con 0x55b051b77730 2024-09-19T11:23:27.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.470+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94215773642752] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025807f040 con 0x55b051b77730 2024-09-19T11:23:27.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.470+0000 7f0266b46800 20 remove_watcher() i=4 2024-09-19T11:23:27.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.470+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94215773647808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b84720 con 0x7f02540534f0 2024-09-19T11:23:27.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.471+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94215773647808] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.471+0000 7f0266b46800 20 remove_watcher() i=5 2024-09-19T11:23:27.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.471+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94215773632640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b823b0 con 0x7f02540534f0 2024-09-19T11:23:27.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.472+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94215773632640] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.472+0000 7f0266b46800 20 remove_watcher() i=6 2024-09-19T11:23:27.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.472+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94215773649968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55b051b823b0 con 0x7f02540534f0 2024-09-19T11:23:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94215773649968] v31'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f025807f040 con 0x7f02540534f0 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 20 remove_watcher() i=7 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02540534f0 msgr2=0x7f0254053910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f02540534f0 0x7f0254053910 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051b77730 msgr2=0x55b051b79bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051b77730 0x55b051b79bf0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f025c033440 msgr2=0x7f025c0451d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f025c033440 0x7f025c0451d0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b7c140 msgr2=0x55b051b7e530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b7c140 0x55b051b7e530 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f012c03d000 msgr2=0x7f012c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f012c03d000 0x7f012c03f4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f025400efd0 tx=0x7f0254041000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 msgr2=0x55b051b6be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 --2- 172.21.15.28:0/1894852112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051b648b0 0x55b051b6be80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f025c020790 tx=0x7f025c0547a0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 reap_dead start 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0263e36640 1 -- 172.21.15.28:0/1894852112 reap_dead start 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 shutdown_connections 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 >> 172.21.15.28:0/1894852112 conn(0x55b051b60cb0 msgr2=0x55b051b5eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 shutdown_connections 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.473+0000 7f0266b46800 1 -- 172.21.15.28:0/1894852112 wait complete. 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f013c03cfb0 msgr2=0x7f013c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f013c03cfb0 0x7f013c03f470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f02540531e0 tx=0x7f02540011b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a436c0 msgr2=0x55b051a41f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a436c0 0x55b051a41f30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55b051b3bc30 tx=0x7f025c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 shutdown_connections 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f013c03cfb0 0x7f013c03f470 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3181641862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a436c0 0x55b051a41f30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 >> 172.21.15.28:0/3181641862 conn(0x55b051b49f90 msgr2=0x55b051b4a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 shutdown_connections 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3181641862 wait complete. 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051a47ab0 msgr2=0x55b051b42280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051a47ab0 0x55b051b42280 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b44660 msgr2=0x55b051b46ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b44660 0x55b051b46ae0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f014c03d1a0 msgr2=0x7f014c03f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f014c03d1a0 0x7f014c03f660 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0258057000 tx=0x7f0258052000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 msgr2=0x55b051b3ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b3ac30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f02540012f0 tx=0x7f0254000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 shutdown_connections 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b051b44660 0x55b051b46ae0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f014c03d1a0 0x7f014c03f660 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b051a47ab0 0x55b051b42280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 --2- 172.21.15.28:0/3357939628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b051a43f40 0x55b051b3ac30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 >> 172.21.15.28:0/3357939628 conn(0x55b051a3daa0 msgr2=0x55b051b34610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 shutdown_connections 2024-09-19T11:23:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.474+0000 7f0266b46800 1 -- 172.21.15.28:0/3357939628 wait complete. 2024-09-19T11:23:27.491 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-09-19T11:23:27.494 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-09-19T11:23:27.537 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:27.537 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-09-19T11:23:27.538 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-09-19T11:23:27.538 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-09-19T11:23:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.585+0000 7f96746bb800 1 Processor -- start 2024-09-19T11:23:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.585+0000 7f96746bb800 1 -- start start 2024-09-19T11:23:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.585+0000 7f96746bb800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5591770ebae0 0x5591770edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.585+0000 7f96746bb800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5591770ee5f0 con 0x5591770e7fd0 2024-09-19T11:23:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.585+0000 7f96746bb800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x559176f727d0 con 0x5591770ebae0 2024-09-19T11:23:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.586+0000 7f96719a4640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5591770ebae0 0x5591770edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.586+0000 7f96721a5640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5591770e7fd0 0x5591770ea3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48402/0 (socket says 172.21.15.28:48402) 2024-09-19T11:23:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.586+0000 7f96721a5640 1 -- 172.21.15.28:0/3229944570 learned_addr learned my addr 172.21.15.28:0/3229944570 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.586+0000 7f96719a4640 1 -- 172.21.15.28:0/3229944570 >> v1:172.21.15.22:6789/0 conn(0x5591770e7fd0 legacy=0x5591770ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.586+0000 7f96719a4640 1 -- 172.21.15.28:0/3229944570 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591770ef820 con 0x5591770ebae0 2024-09-19T11:23:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96719a4640 1 --2- 172.21.15.28:0/3229944570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770ebae0 0x5591770edf70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f966800a470 tx=0x7f9668045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922d21c759573ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96711a3640 1 -- 172.21.15.28:0/3229944570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9668053570 con 0x5591770ebae0 2024-09-19T11:23:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96711a3640 1 -- 172.21.15.28:0/3229944570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9668058070 con 0x5591770ebae0 2024-09-19T11:23:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96711a3640 1 -- 172.21.15.28:0/3229944570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9668053880 con 0x5591770ebae0 2024-09-19T11:23:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96746bb800 1 -- 172.21.15.28:0/3229944570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770ebae0 msgr2=0x5591770edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96746bb800 1 --2- 172.21.15.28:0/3229944570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770ebae0 0x5591770edf70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f966800a470 tx=0x7f9668045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96746bb800 1 -- 172.21.15.28:0/3229944570 shutdown_connections 2024-09-19T11:23:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96746bb800 1 --2- 172.21.15.28:0/3229944570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770ebae0 0x5591770edf70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96746bb800 1 -- 172.21.15.28:0/3229944570 >> 172.21.15.28:0/3229944570 conn(0x5591770e1a70 msgr2=0x5591770e3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96746bb800 1 -- 172.21.15.28:0/3229944570 shutdown_connections 2024-09-19T11:23:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.588+0000 7f96746bb800 1 -- 172.21.15.28:0/3229944570 wait complete. 2024-09-19T11:23:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96746bb800 1 Processor -- start 2024-09-19T11:23:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96746bb800 1 -- start start 2024-09-19T11:23:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96746bb800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96746bb800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559176f7fdd0 con 0x5591770e7fd0 2024-09-19T11:23:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96721a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96721a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771d7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34704/0 (socket says 172.21.15.28:34704) 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96721a5640 1 -- 172.21.15.28:0/4098415727 learned_addr learned my addr 172.21.15.28:0/4098415727 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.604+0000 7f96721a5640 1 -- 172.21.15.28:0/4098415727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591771d7930 con 0x5591770e7fd0 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f96721a5640 1 --2- 172.21.15.28:0/4098415727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771d7340 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9664001800 tx=0x7f9664001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734c2d284f2285b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f9560608640 1 -- 172.21.15.28:0/4098415727 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9664004520 con 0x5591770e7fd0 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f9560608640 1 -- 172.21.15.28:0/4098415727 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96640046c0 con 0x5591770e7fd0 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f9560608640 1 -- 172.21.15.28:0/4098415727 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96640049e0 con 0x5591770e7fd0 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f96746bb800 1 -- 172.21.15.28:0/4098415727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 msgr2=0x5591771d7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f96746bb800 1 --2- 172.21.15.28:0/4098415727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771d7340 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9664001800 tx=0x7f9664001cc0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f96746bb800 1 -- 172.21.15.28:0/4098415727 shutdown_connections 2024-09-19T11:23:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f96746bb800 1 --2- 172.21.15.28:0/4098415727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771d7340 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.605+0000 7f96746bb800 1 -- 172.21.15.28:0/4098415727 >> 172.21.15.28:0/4098415727 conn(0x5591770e1a70 msgr2=0x5591771d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96746bb800 1 -- 172.21.15.28:0/4098415727 shutdown_connections 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96746bb800 1 -- 172.21.15.28:0/4098415727 wait complete. 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96746bb800 1 Processor -- start 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96746bb800 1 -- start start 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96746bb800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771dee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96746bb800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591770eeac0 con 0x5591770e7fd0 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96721a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771dee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96721a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771dee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34710/0 (socket says 172.21.15.28:34710) 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96721a5640 1 -- 172.21.15.28:0/4008503833 learned_addr learned my addr 172.21.15.28:0/4008503833 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.606+0000 7f96721a5640 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591771df3f0 con 0x5591770e7fd0 2024-09-19T11:23:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.607+0000 7f96721a5640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771dee90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9664000f80 tx=0x7f9664001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.607+0000 7f955ee05640 1 -- 172.21.15.28:0/4008503833 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9664022020 con 0x5591770e7fd0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.607+0000 7f955ee05640 1 -- 172.21.15.28:0/4008503833 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9664014890 con 0x5591770e7fd0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.607+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5591771df680 con 0x5591770e7fd0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.607+0000 7f955ee05640 1 -- 172.21.15.28:0/4008503833 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9664014bb0 con 0x5591770e7fd0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.607+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591771e0080 con 0x5591770e7fd0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.608+0000 7f955ee05640 1 -- 172.21.15.28:0/4008503833 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9664025020 con 0x5591770e7fd0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.608+0000 7f955ee05640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f955803d000 0x7f955803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.608+0000 7f955ee05640 1 -- 172.21.15.28:0/4008503833 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f9664018030 con 0x5591770e7fd0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.608+0000 7f96719a4640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f955803d000 0x7f955803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.608+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5591770ebae0 0x5591771e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.608+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591771e6a20 con 0x5591770ebae0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.608+0000 7f96729a6640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5591770ebae0 0x5591771e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96719a4640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f955803d000 0x7f955803f4c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9668057000 tx=0x7f9668052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96729a6640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5591770ebae0 0x5591771e64e0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96729a6640 1 -- 172.21.15.28:0/4008503833 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f966c0083e0 con 0x5591770ebae0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591771e6dc0 con 0x5591770ebae0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96729a6640 1 -- 172.21.15.28:0/4008503833 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f966c0083e0 con 0x5591770ebae0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5591771e88c0 0x5591771ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591771eb4d0 con 0x5591771e88c0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.609+0000 7f96721a5640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5591771e88c0 0x5591771ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.610+0000 7f96721a5640 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5591771e88c0 0x5591771ead40 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.610+0000 7f96721a5640 1 -- 172.21.15.28:0/4008503833 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f966401b070 con 0x5591771e88c0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.610+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591771e7610 con 0x5591770ebae0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.610+0000 7f96729a6640 1 -- 172.21.15.28:0/4008503833 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f966c0083e0 con 0x5591770ebae0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.610+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591771e7a00 con 0x5591771e88c0 2024-09-19T11:23:27.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.611+0000 7f96721a5640 1 -- 172.21.15.28:0/4008503833 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f966401b070 con 0x5591771e88c0 2024-09-19T11:23:27.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.611+0000 7f96746bb800 1 Processor -- start 2024-09-19T11:23:27.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.611+0000 7f96746bb800 1 -- start start 2024-09-19T11:23:27.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.611+0000 7f96746bb800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591771f9e90 0x5591771fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.611+0000 7f96746bb800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559176fde4a0 con 0x5591771f9e90 2024-09-19T11:23:27.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f96729a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591771f9e90 0x5591771fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f96729a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591771f9e90 0x5591771fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34724/0 (socket says 172.21.15.28:34724) 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f96729a6640 1 -- 172.21.15.28:0/2023642834 learned_addr learned my addr 172.21.15.28:0/2023642834 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f96729a6640 1 -- 172.21.15.28:0/2023642834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591771fa8f0 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f96729a6640 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591771f9e90 0x5591771fa2b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5591771dfe90 tx=0x7f966c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f9557fff640 1 -- 172.21.15.28:0/2023642834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f966c01c020 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f9557fff640 1 -- 172.21.15.28:0/2023642834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966c007c90 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f9557fff640 1 -- 172.21.15.28:0/2023642834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f966c018440 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.612+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5591771fb120 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.613+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591771fb6c0 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.613+0000 7f9557fff640 1 -- 172.21.15.28:0/2023642834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f966c01f020 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f9557fff640 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f954c03d000 0x7f954c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f9557fff640 1 -- 172.21.15.28:0/2023642834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f966c055a00 con 0x5591771f9e90 2024-09-19T11:23:27.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96721a5640 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f954c03d000 0x7f954c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96746bb800 1 Processor -- start 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96746bb800 1 -- start start 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96721a5640 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f954c03d000 0x7f954c03f4c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5591771eb8a0 tx=0x7f9664006820 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96746bb800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x559177206d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96746bb800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591771d7d60 con 0x559177206950 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96729a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x559177206d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96729a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x559177206d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34736/0 (socket says 172.21.15.28:34736) 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.614+0000 7f96729a6640 1 -- 172.21.15.28:0/554395046 learned_addr learned my addr 172.21.15.28:0/554395046 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96729a6640 1 -- 172.21.15.28:0/554395046 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559177207340 con 0x559177206950 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96729a6640 1 --2- 172.21.15.28:0/554395046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x559177206d50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f966c053d80 tx=0x7f966c0153e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36f2a12c0aca7bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f9555ffb640 1 -- 172.21.15.28:0/554395046 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f966c04a070 con 0x559177206950 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f9555ffb640 1 -- 172.21.15.28:0/554395046 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966c0174d0 con 0x559177206950 2024-09-19T11:23:27.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f9555ffb640 1 -- 172.21.15.28:0/554395046 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f966c052070 con 0x559177206950 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96746bb800 1 -- 172.21.15.28:0/554395046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 msgr2=0x559177206d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96746bb800 1 --2- 172.21.15.28:0/554395046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x559177206d50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f966c053d80 tx=0x7f966c0153e0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96746bb800 1 -- 172.21.15.28:0/554395046 shutdown_connections 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96746bb800 1 --2- 172.21.15.28:0/554395046 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x559177206d50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96746bb800 1 -- 172.21.15.28:0/554395046 >> 172.21.15.28:0/554395046 conn(0x559177202d50 msgr2=0x559177200f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96746bb800 1 -- 172.21.15.28:0/554395046 shutdown_connections 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.615+0000 7f96746bb800 1 -- 172.21.15.28:0/554395046 wait complete. 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96746bb800 1 Processor -- start 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96746bb800 1 -- start start 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96746bb800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x55917720df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96746bb800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559177207960 con 0x559177206950 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96729a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x55917720df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96729a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x55917720df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34748/0 (socket says 172.21.15.28:34748) 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 learned_addr learned my addr 172.21.15.28:0/143716082 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.616+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917720e490 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.617+0000 7f96729a6640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x55917720df30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f966c0540f0 tx=0x7f966c053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.617+0000 7f9543fff640 1 -- 172.21.15.28:0/143716082 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f966c02e030 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.617+0000 7f9543fff640 1 -- 172.21.15.28:0/143716082 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f966c013320 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.617+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55917720e720 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.617+0000 7f9543fff640 1 -- 172.21.15.28:0/143716082 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f966c04a040 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.617+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917720f0a0 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.618+0000 7f9543fff640 1 -- 172.21.15.28:0/143716082 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f966c031030 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.618+0000 7f9543fff640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f953c03cfb0 0x7f953c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.618+0000 7f9543fff640 1 -- 172.21.15.28:0/143716082 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f966c02f070 con 0x559177206950 2024-09-19T11:23:27.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.618+0000 7f96721a5640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f953c03cfb0 0x7f953c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.618+0000 7f96721a5640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f953c03cfb0 0x7f953c03f470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9664014fd0 tx=0x7f9664052d00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559177219850 0x55917721bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55917721c460 con 0x559177219850 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96719a4640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559177219850 0x55917721bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96719a4640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559177219850 0x55917721bd10 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96746bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96746bb800 20 realm 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.619+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55917721c890 con 0x559177219850 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55917721d160 con 0x559177219850 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55917721d500 con 0x559177219850 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55917721e260 0x559177220650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559177220c60 con 0x55917721e260 2024-09-19T11:23:27.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96729a6640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55917721e260 0x559177220650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.620+0000 7f96729a6640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55917721e260 0x559177220650 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96746bb800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96746bb800 20 searching for the correct realm 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96729a6640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f966c021020 0x7f966c016550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f966c021020 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96721a5640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f966c021020 0x7f966c016550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.621+0000 7f96721a5640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f966c021020 0x7f966c016550 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.622+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.622+0000 7f96721a5640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9664027020 0x7f966401e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.622+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.622+0000 7f96719a4640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9664027020 0x7f966401e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.622+0000 7f96719a4640 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9664027020 0x7f966401e3f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.622+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.622+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.623+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f966c021020 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.624+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f966c021020 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.625+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.626+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.627+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.627+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.627+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.627+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.627+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f966c021020 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.627+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.627+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.628+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.628+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.628+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.628+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.628+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.628+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.628+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.629+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f966c021020 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.629+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.629+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x55917721e260 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.629+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.629+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f9664027020 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.629+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.629+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x559177219850 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x559177222590 con 0x7f966c021020 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559177221620 con 0x559177219850 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559177221e70 con 0x559177219850 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.630+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559177222a10 con 0x55917721e260 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 zone default found 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 4 Realm: () 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 zonegroup default 2024-09-19T11:23:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559177222e10 con 0x7f9664027020 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591772250f0 con 0x55917721e260 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.631+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559177225490 con 0x55917721e260 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.632+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.632+0000 7f96746bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.632+0000 7f96746bb800 20 started sync module instance, tier type = 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.632+0000 7f96746bb800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.632+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177224530 con 0x7f966c021020 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.633+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.633+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94083257352320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772258c0 con 0x7f966c021020 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.633+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177226110 con 0x559177219850 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.634+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94083257352320] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.634+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.634+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94083257359456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177226530 con 0x559177219850 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.634+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177226d80 con 0x55917721e260 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.635+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.635+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94083257362224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177226d50 con 0x55917721e260 2024-09-19T11:23:27.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.635+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177227d70 con 0x7f966c021020 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.635+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94083257359456] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.636+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94083257362224] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.636+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.636+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94083257364848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177226080 con 0x7f966c021020 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.636+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177228780 con 0x559177219850 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.637+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.637+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94083257369296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177228ba0 con 0x559177219850 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.637+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177229650 con 0x7f9664027020 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.637+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94083257364848] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.638+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94083257369296] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.638+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.638+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94083257366592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772286f0 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.638+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177229ef0 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.639+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94083257366592] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.639+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.639+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94083257372976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772286f0 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.639+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917722a4d0 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94083257376672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917722a940 con 0x7f9664027020 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96746bb800 20 add_watcher() i=0 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96746bb800 20 add_watcher() i=1 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96746bb800 20 add_watcher() i=2 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96746bb800 20 add_watcher() i=3 2024-09-19T11:23:27.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96746bb800 20 add_watcher() i=4 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96746bb800 20 add_watcher() i=5 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.640+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94083257372976] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.641+0000 7f96746bb800 20 add_watcher() i=6 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.641+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94083257376672] v31'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.641+0000 7f96746bb800 20 add_watcher() i=7 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.641+0000 7f96746bb800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.641+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55917722a940 con 0x559177219850 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.642+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.642+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94083257377936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917722a8b0 con 0x559177219850 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.643+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94083257377936] v31'139 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.643+0000 7f9528fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.643+0000 7f96746bb800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:27.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.643+0000 7f96746bb800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:27.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.643+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177236250 con 0x559177219850 2024-09-19T11:23:27.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.644+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v31'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.644+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177236250 con 0x7f9664027020 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.645+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v31'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.645+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177236c70 con 0x55917721e260 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.646+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.646+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177234860 con 0x559177219850 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.647+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v31'140 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.647+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177234cc0 con 0x7f966c021020 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.648+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v31'41 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.648+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772377d0 con 0x55917721e260 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.649+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v31'27 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.649+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772377d0 con 0x559177219850 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.650+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v31'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.650+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772377d0 con 0x7f9664027020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.651+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v31'27 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.651+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772377d0 con 0x7f966c021020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.652+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v31'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.652+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237c80 con 0x55917721e260 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.653+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v31'23 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.653+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237c80 con 0x7f966c021020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.654+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v31'42 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.654+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237c80 con 0x7f9664027020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.655+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v31'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.655+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237d10 con 0x7f966c021020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.656+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v31'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.656+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237d10 con 0x7f9664027020 2024-09-19T11:23:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.657+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v31'34 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.657+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237e00 con 0x559177219850 2024-09-19T11:23:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.658+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v31'141 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.658+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237e00 con 0x55917721e260 2024-09-19T11:23:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.659+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v31'35 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.659+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237da0 con 0x7f966c021020 2024-09-19T11:23:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.660+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v31'43 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.660+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237da0 con 0x55917721e260 2024-09-19T11:23:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.661+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v31'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.661+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237e10 con 0x559177219850 2024-09-19T11:23:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.662+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v31'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.663+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237e10 con 0x559177219850 2024-09-19T11:23:27.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.664+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v31'142 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.664+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237e10 con 0x55917721e260 2024-09-19T11:23:27.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.665+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v31'36 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.665+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237e10 con 0x7f966c021020 2024-09-19T11:23:27.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.666+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v31'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.666+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177237e10 con 0x55917721e260 2024-09-19T11:23:27.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.667+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.667+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x55917721e260 2024-09-19T11:23:27.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.668+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v31'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.668+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x55917721e260 2024-09-19T11:23:27.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.669+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v31'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.669+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x55917721e260 2024-09-19T11:23:27.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.670+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v31'28 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.670+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x7f9664027020 2024-09-19T11:23:27.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.671+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v31'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.671+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x55917721e260 2024-09-19T11:23:27.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.672+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v31'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.672+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x7f9664027020 2024-09-19T11:23:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.673+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v31'28 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.673+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x55917721e260 2024-09-19T11:23:27.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.674+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v31'37 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.674+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x7f9664027020 2024-09-19T11:23:27.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.675+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v31'35 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.675+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772383d0 con 0x7f966c021020 2024-09-19T11:23:27.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.676+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v31'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.676+0000 7f94a37fe640 20 reqs_thread_entry: start 2024-09-19T11:23:27.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.676+0000 7f96746bb800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:27.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.676+0000 7f96746bb800 20 Filter name: none 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f94a17fa640 20 reqs_thread_entry: start 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591772630e0 con 0x7f966c021020 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:95 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x559177264800 con 0x7f9664027020 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(95 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:27.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.677+0000 7f96746bb800 20 get_obj_state: octx=0x7f9664010400 obj=myfoo:four state=0x5591772655b8 s->prefetch_data=0 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 3.1fs0 3:fa1685db:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1_four:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5591772677f0 con 0x559177219850 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(96 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1_four [call out=24b,getxattrs out=849b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 276+0+889 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96746bb800 10 manifest: total_size = 4 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96746bb800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.3978359192935344992 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96746bb800 20 get_obj_state: octx=0x7f9664010400 obj=myfoo:four state=0x5591772655b8 s->prefetch_data=0 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96746bb800 20 prepare_atomic_modification: state is not atomic. state=0x5591772655b8 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96746bb800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.678+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 4.d 4:b3b80c54:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917726b380 con 0x55917721e260 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.679+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(97 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 [stat,call,call] v31'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.679+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:98 3.1fs0 3:fa1685db:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917726b380 con 0x559177219850 2024-09-19T11:23:27.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.680+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(98 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1_four [call] v31'2 uv2 ondisk = 0) ==== 192+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.680+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:99 4.d 4:b3b80c54:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917726be10 con 0x55917721e260 2024-09-19T11:23:27.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.681+0000 7f9528fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.682+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94083257377936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917726be10 con 0x559177219850 2024-09-19T11:23:27.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.682+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(99 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 [stat,call,call] v31'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.682+0000 7f9555ffb640 20 handle_completion(): completion ok for obj=four 2024-09-19T11:23:27.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.682+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(100 data_loggenerations_metadata [watch unwatch cookie 94083257377936] v31'143 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.682+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94083257352320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55917726be10 con 0x7f966c021020 2024-09-19T11:23:27.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.683+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(101 notify.0 [watch unwatch cookie 94083257352320] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.683+0000 7f96746bb800 20 remove_watcher() i=0 2024-09-19T11:23:27.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.683+0000 7f96746bb800 2 removed watcher, disabling cache 2024-09-19T11:23:27.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.683+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:102 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94083257359456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177224080 con 0x559177219850 2024-09-19T11:23:27.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.685+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(102 notify.1 [watch unwatch cookie 94083257359456] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.685+0000 7f96746bb800 20 remove_watcher() i=1 2024-09-19T11:23:27.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.685+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:103 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94083257362224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177225c60 con 0x55917721e260 2024-09-19T11:23:27.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.686+0000 7f96729a6640 1 -- 172.21.15.28:0/143716082 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(103 notify.2 [watch unwatch cookie 94083257362224] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966c03a070 con 0x55917721e260 2024-09-19T11:23:27.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.686+0000 7f96746bb800 20 remove_watcher() i=2 2024-09-19T11:23:27.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.686+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94083257364848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177224080 con 0x7f966c021020 2024-09-19T11:23:27.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.687+0000 7f96721a5640 1 -- 172.21.15.28:0/143716082 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(104 notify.3 [watch unwatch cookie 94083257364848] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f966401b070 con 0x7f966c021020 2024-09-19T11:23:27.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.687+0000 7f96746bb800 20 remove_watcher() i=3 2024-09-19T11:23:27.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.687+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94083257369296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177225c60 con 0x559177219850 2024-09-19T11:23:27.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.687+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(105 notify.4 [watch unwatch cookie 94083257369296] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x559177219850 2024-09-19T11:23:27.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.688+0000 7f96746bb800 20 remove_watcher() i=4 2024-09-19T11:23:27.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.688+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94083257366592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177224080 con 0x7f9664027020 2024-09-19T11:23:27.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.689+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(106 notify.5 [watch unwatch cookie 94083257366592] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.689+0000 7f96746bb800 20 remove_watcher() i=5 2024-09-19T11:23:27.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.689+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:107 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94083257372976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5591772282d0 con 0x7f9664027020 2024-09-19T11:23:27.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.690+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(107 notify.6 [watch unwatch cookie 94083257372976] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.690+0000 7f96746bb800 20 remove_watcher() i=6 2024-09-19T11:23:27.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.690+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:108 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94083257376672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x559177225c60 con 0x7f9664027020 2024-09-19T11:23:27.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(108 notify.7 [watch unwatch cookie 94083257376672] v31'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9668051070 con 0x7f9664027020 2024-09-19T11:23:27.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 20 remove_watcher() i=7 2024-09-19T11:23:27.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9664027020 msgr2=0x7f966401e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9664027020 0x7f966401e3f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559177219850 msgr2=0x55917721bd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559177219850 0x55917721bd10 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f966c021020 msgr2=0x7f966c016550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f966c021020 0x7f966c016550 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55917721e260 msgr2=0x559177220650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55917721e260 0x559177220650 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f953c03cfb0 msgr2=0x7f953c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f953c03cfb0 0x7f953c03f470 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9664014fd0 tx=0x7f9664052d00 comp rx=0 tx=0).stop 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 msgr2=0x55917720df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96719a4640 1 -- 172.21.15.28:0/143716082 reap_dead start 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x55917720df30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f966c0540f0 tx=0x7f966c053ce0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 shutdown_connections 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 --2- 172.21.15.28:0/143716082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559177206950 0x55917720df30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 >> 172.21.15.28:0/143716082 conn(0x559177202d50 msgr2=0x559177200f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 shutdown_connections 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.691+0000 7f96746bb800 1 -- 172.21.15.28:0/143716082 wait complete. 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f954c03d000 msgr2=0x7f954c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f954c03d000 0x7f954c03f4c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5591771eb8a0 tx=0x7f9664006820 comp rx=0 tx=0).stop 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591771f9e90 msgr2=0x5591771fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591771f9e90 0x5591771fa2b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5591771dfe90 tx=0x7f966c00bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 shutdown_connections 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f954c03d000 0x7f954c03f4c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/2023642834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591771f9e90 0x5591771fa2b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 >> 172.21.15.28:0/2023642834 conn(0x5591771ee240 msgr2=0x5591771ee620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 shutdown_connections 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/2023642834 wait complete. 2024-09-19T11:23:27.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5591770ebae0 msgr2=0x5591771e64e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5591770ebae0 0x5591771e64e0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5591771e88c0 msgr2=0x5591771ead40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5591771e88c0 0x5591771ead40 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f955803d000 msgr2=0x7f955803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f955803d000 0x7f955803f4c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9668057000 tx=0x7f9668052000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 msgr2=0x5591771dee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771dee90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9664000f80 tx=0x7f9664001250 comp rx=0 tx=0).stop 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 shutdown_connections 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5591771e88c0 0x5591771ead40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f955803d000 0x7f955803f4c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5591770ebae0 0x5591771e64e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 --2- 172.21.15.28:0/4008503833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591770e7fd0 0x5591771dee90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 >> 172.21.15.28:0/4008503833 conn(0x5591770e1a70 msgr2=0x5591771d8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 shutdown_connections 2024-09-19T11:23:27.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.692+0000 7f96746bb800 1 -- 172.21.15.28:0/4008503833 wait complete. 2024-09-19T11:23:27.702 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-09-19T11:23:27.702 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-09-19T11:23:27.703 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-09-19T11:23:27.748 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:27.748 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:27.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.747+0000 7f5c0952c800 1 Processor -- start 2024-09-19T11:23:27.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.747+0000 7f5c0952c800 1 -- start start 2024-09-19T11:23:27.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.747+0000 7f5c0952c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c8eb833280 0x55c8eb8c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.747+0000 7f5c0952c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c8eb8c7910 con 0x55c8eb834610 2024-09-19T11:23:27.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.747+0000 7f5c0952c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55c8eb7497d0 con 0x55c8eb833280 2024-09-19T11:23:27.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.748+0000 7f5c04a4e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c8eb833280 0x55c8eb8c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.748+0000 7f5c0524f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55c8eb834610 0x55c8eb832a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48410/0 (socket says 172.21.15.28:48410) 2024-09-19T11:23:27.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.748+0000 7f5c0524f640 1 -- 172.21.15.28:0/3202495693 learned_addr learned my addr 172.21.15.28:0/3202495693 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:27.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.748+0000 7f5c04a4e640 1 -- 172.21.15.28:0/3202495693 >> v1:172.21.15.22:6789/0 conn(0x55c8eb834610 legacy=0x55c8eb832a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.748+0000 7f5c04a4e640 1 -- 172.21.15.28:0/3202495693 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8eb8c8af0 con 0x55c8eb833280 2024-09-19T11:23:27.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c04a4e640 1 --2- 172.21.15.28:0/3202495693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb8c73d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000a470 tx=0x7f5bf0045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db89e7268728f64f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5bfffff640 1 -- 172.21.15.28:0/3202495693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0053570 con 0x55c8eb833280 2024-09-19T11:23:27.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5bfffff640 1 -- 172.21.15.28:0/3202495693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0058070 con 0x55c8eb833280 2024-09-19T11:23:27.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5bfffff640 1 -- 172.21.15.28:0/3202495693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0053880 con 0x55c8eb833280 2024-09-19T11:23:27.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c0952c800 1 -- 172.21.15.28:0/3202495693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 msgr2=0x55c8eb8c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3202495693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb8c73d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000a470 tx=0x7f5bf0045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:27.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c0952c800 1 -- 172.21.15.28:0/3202495693 shutdown_connections 2024-09-19T11:23:27.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3202495693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb8c73d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c0952c800 1 -- 172.21.15.28:0/3202495693 >> 172.21.15.28:0/3202495693 conn(0x55c8eb8b9c60 msgr2=0x55c8eb8bc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c0952c800 1 -- 172.21.15.28:0/3202495693 shutdown_connections 2024-09-19T11:23:27.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.750+0000 7f5c0952c800 1 -- 172.21.15.28:0/3202495693 wait complete. 2024-09-19T11:23:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0952c800 1 Processor -- start 2024-09-19T11:23:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0952c800 1 -- start start 2024-09-19T11:23:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0952c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0952c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c8eb756dd0 con 0x55c8eb833280 2024-09-19T11:23:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0524f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0524f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34766/0 (socket says 172.21.15.28:34766) 2024-09-19T11:23:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0524f640 1 -- 172.21.15.28:0/3443256790 learned_addr learned my addr 172.21.15.28:0/3443256790 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.766+0000 7f5c0524f640 1 -- 172.21.15.28:0/3443256790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8eb9b0ac0 con 0x55c8eb833280 2024-09-19T11:23:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5c0524f640 1 --2- 172.21.15.28:0/3443256790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b04d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5c00001800 tx=0x7f5c00001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a275d069bfb6eed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5af3606640 1 -- 172.21.15.28:0/3443256790 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c00004520 con 0x55c8eb833280 2024-09-19T11:23:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5af3606640 1 -- 172.21.15.28:0/3443256790 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c000046c0 con 0x55c8eb833280 2024-09-19T11:23:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5af3606640 1 -- 172.21.15.28:0/3443256790 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c000049e0 con 0x55c8eb833280 2024-09-19T11:23:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5c0952c800 1 -- 172.21.15.28:0/3443256790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 msgr2=0x55c8eb9b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3443256790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b04d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5c00001800 tx=0x7f5c00001cc0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5c0952c800 1 -- 172.21.15.28:0/3443256790 shutdown_connections 2024-09-19T11:23:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3443256790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b04d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.767+0000 7f5c0952c800 1 -- 172.21.15.28:0/3443256790 >> 172.21.15.28:0/3443256790 conn(0x55c8eb8b9c60 msgr2=0x55c8eb9abe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0952c800 1 -- 172.21.15.28:0/3443256790 shutdown_connections 2024-09-19T11:23:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0952c800 1 -- 172.21.15.28:0/3443256790 wait complete. 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0952c800 1 Processor -- start 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0952c800 1 -- start start 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0952c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0952c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c8eb8c7de0 con 0x55c8eb833280 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0524f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0524f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34780/0 (socket says 172.21.15.28:34780) 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0524f640 1 -- 172.21.15.28:0/4024612057 learned_addr learned my addr 172.21.15.28:0/4024612057 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.768+0000 7f5c0524f640 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8eb9b85a0 con 0x55c8eb833280 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.769+0000 7f5c0524f640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b8040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5c00000f80 tx=0x7f5c00001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.769+0000 7f5af1e03640 1 -- 172.21.15.28:0/4024612057 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c00022020 con 0x55c8eb833280 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.769+0000 7f5af1e03640 1 -- 172.21.15.28:0/4024612057 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c00014890 con 0x55c8eb833280 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.769+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8eb9b8830 con 0x55c8eb833280 2024-09-19T11:23:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.769+0000 7f5af1e03640 1 -- 172.21.15.28:0/4024612057 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c00014bb0 con 0x55c8eb833280 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.769+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8eb9b9230 con 0x55c8eb833280 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.770+0000 7f5af1e03640 1 -- 172.21.15.28:0/4024612057 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5c00025020 con 0x55c8eb833280 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.770+0000 7f5af1e03640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5aec03d000 0x7f5aec03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.770+0000 7f5af1e03640 1 -- 172.21.15.28:0/4024612057 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5c00018030 con 0x55c8eb833280 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.770+0000 7f5c04a4e640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5aec03d000 0x7f5aec03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.770+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb834610 0x55c8eb9bf690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.770+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9bfbd0 con 0x55c8eb834610 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.770+0000 7f5c05a50640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb834610 0x55c8eb9bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c04a4e640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5aec03d000 0x7f5aec03f4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0057000 tx=0x7f5bf0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c05a50640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb834610 0x55c8eb9bf690 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c05a50640 1 -- 172.21.15.28:0/4024612057 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb834610 2024-09-19T11:23:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9bff70 con 0x55c8eb834610 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c05a50640 1 -- 172.21.15.28:0/4024612057 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb834610 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9c1a70 0x55c8eb9c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9c4680 con 0x55c8eb9c1a70 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.771+0000 7f5c0524f640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9c1a70 0x55c8eb9c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.772+0000 7f5c0524f640 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9c1a70 0x55c8eb9c3ef0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.772+0000 7f5c0524f640 1 -- 172.21.15.28:0/4024612057 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5c0001b070 con 0x55c8eb9c1a70 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.772+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9c07c0 con 0x55c8eb834610 2024-09-19T11:23:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.772+0000 7f5c05a50640 1 -- 172.21.15.28:0/4024612057 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb834610 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.772+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9c0bb0 con 0x55c8eb9c1a70 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c0524f640 1 -- 172.21.15.28:0/4024612057 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5c0001b070 con 0x55c8eb9c1a70 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c0952c800 1 Processor -- start 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c0952c800 1 -- start start 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c0952c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9d3040 0x55c8eb9d3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c0952c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c8eb7b54a0 con 0x55c8eb9d3040 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c05a50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9d3040 0x55c8eb9d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c05a50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9d3040 0x55c8eb9d3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34792/0 (socket says 172.21.15.28:34792) 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.773+0000 7f5c05a50640 1 -- 172.21.15.28:0/3806810915 learned_addr learned my addr 172.21.15.28:0/3806810915 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.774+0000 7f5c05a50640 1 -- 172.21.15.28:0/3806810915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8eb9d3aa0 con 0x55c8eb9d3040 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.774+0000 7f5c05a50640 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9d3040 0x55c8eb9d3460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55c8eb9b9040 tx=0x7f5bf800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.774+0000 7f5aeaffd640 1 -- 172.21.15.28:0/3806810915 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf801c020 con 0x55c8eb9d3040 2024-09-19T11:23:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.774+0000 7f5aeaffd640 1 -- 172.21.15.28:0/3806810915 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf8007c90 con 0x55c8eb9d3040 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.774+0000 7f5aeaffd640 1 -- 172.21.15.28:0/3806810915 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf8018440 con 0x55c8eb9d3040 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.774+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8eb9d42d0 con 0x55c8eb9d3040 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.774+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8eb9d4870 con 0x55c8eb9d3040 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.775+0000 7f5aeaffd640 1 -- 172.21.15.28:0/3806810915 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5bf801f020 con 0x55c8eb9d3040 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.775+0000 7f5aeaffd640 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5adc03d000 0x7f5adc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.775+0000 7f5aeaffd640 1 -- 172.21.15.28:0/3806810915 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5bf8055a70 con 0x55c8eb9d3040 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.775+0000 7f5c0524f640 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5adc03d000 0x7f5adc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.775+0000 7f5c0952c800 1 Processor -- start 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c0524f640 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5adc03d000 0x7f5adc03f4c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55c8eb9c4a50 tx=0x7f5c00006820 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c0952c800 1 -- start start 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c0952c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c0952c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c8eb9b0ef0 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c05a50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c05a50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9dfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34806/0 (socket says 172.21.15.28:34806) 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c05a50640 1 -- 172.21.15.28:0/3500536690 learned_addr learned my addr 172.21.15.28:0/3500536690 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c05a50640 1 -- 172.21.15.28:0/3500536690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8eb9e0490 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5c05a50640 1 --2- 172.21.15.28:0/3500536690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9dfea0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8053df0 tx=0x7f5bf8015820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e55cf5bce08350 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5ae8ff9640 1 -- 172.21.15.28:0/3500536690 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf80174d0 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5ae8ff9640 1 -- 172.21.15.28:0/3500536690 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf8052070 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.776+0000 7f5ae8ff9640 1 -- 172.21.15.28:0/3500536690 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf8016d80 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 -- 172.21.15.28:0/3500536690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 msgr2=0x55c8eb9dfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3500536690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9dfea0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8053df0 tx=0x7f5bf8015820 comp rx=0 tx=0).stop 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 -- 172.21.15.28:0/3500536690 shutdown_connections 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3500536690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9dfea0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 -- 172.21.15.28:0/3500536690 >> 172.21.15.28:0/3500536690 conn(0x55c8eb9dbea0 msgr2=0x55c8eb9da0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 -- 172.21.15.28:0/3500536690 shutdown_connections 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 -- 172.21.15.28:0/3500536690 wait complete. 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 Processor -- start 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 -- start start 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9e7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.777+0000 7f5c0952c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c8eb9e0ab0 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5c05a50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5c05a50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9e7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34820/0 (socket says 172.21.15.28:34820) 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 learned_addr learned my addr 172.21.15.28:0/2510205059 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8eb9e7570 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5c05a50640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9e7010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80206f0 tx=0x7f5bf80088f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5adaffd640 1 -- 172.21.15.28:0/2510205059 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf8013e80 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5adaffd640 1 -- 172.21.15.28:0/2510205059 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf80189d0 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8eb9e7800 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.778+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8eb9e7800 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.779+0000 7f5adaffd640 1 -- 172.21.15.28:0/2510205059 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf8018cb0 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.779+0000 7f5adaffd640 1 -- 172.21.15.28:0/2510205059 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5bf8031060 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5adaffd640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ad003cfb0 0x7f5ad003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5adaffd640 1 -- 172.21.15.28:0/2510205059 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f5bf803ba40 con 0x55c8eb9dfaa0 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5c0524f640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ad003cfb0 0x7f5ad003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb9f2770 0x55c8eb9f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9f5380 con 0x55c8eb9f2770 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5c04a4e640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb9f2770 0x55c8eb9f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5c0524f640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ad003cfb0 0x7f5ad003f470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c00014fd0 tx=0x7f5c00052d00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.780+0000 7f5c04a4e640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb9f2770 0x55c8eb9f4c30 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 realm 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9f57b0 con 0x55c8eb9f2770 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9f6080 con 0x55c8eb9f2770 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9f6420 con 0x55c8eb9f2770 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.781+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9f7180 0x55c8eb9f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9f9ab0 con 0x55c8eb9f7180 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c05a50640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9f7180 0x55c8eb9f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c05a50640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9f7180 0x55c8eb9f9570 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c0952c800 20 searching for the correct realm 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c05a50640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5bf8021020 0x7f5bf8052580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.782+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5bf8021020 2024-09-19T11:23:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.783+0000 7f5c0524f640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5bf8021020 0x7f5bf8052580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.783+0000 7f5c0524f640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5bf8021020 0x7f5bf8052580 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.783+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.783+0000 7f5c0524f640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c00027020 0x7f5c0001e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.783+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.783+0000 7f5c04a4e640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c00027020 0x7f5c0001e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.784+0000 7f5c04a4e640 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c00027020 0x7f5c0001e3f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.784+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.784+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.784+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.784+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5bf8021020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.785+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.786+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5bf8021020 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.787+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.788+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5bf8021020 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.789+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5bf8021020 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.790+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f7180 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5c00027020 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x55c8eb9f2770 2024-09-19T11:23:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fb420 con 0x7f5bf8021020 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.791+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fa520 con 0x55c8eb9f2770 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fadc0 con 0x55c8eb9f2770 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fa160 con 0x55c8eb9f7180 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 zone default found 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 4 Realm: () 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:27.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 zonegroup default 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.792+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fbda0 con 0x7f5c00027020 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fdc60 con 0x55c8eb9f7180 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fe0d0 con 0x55c8eb9f7180 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 20 started sync module instance, tier type = 2024-09-19T11:23:27.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.793+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fcfd0 con 0x7f5bf8021020 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.794+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.794+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94321434938144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fe510 con 0x7f5bf8021020 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.794+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9fed60 con 0x55c8eb9f2770 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.795+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94321434938144] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.795+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.795+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94321434945712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9ff180 con 0x55c8eb9f2770 2024-09-19T11:23:27.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.795+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9ffa10 con 0x55c8eb9f7180 2024-09-19T11:23:27.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.796+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94321434945712] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.797+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.797+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94321434946768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9ff5a0 con 0x55c8eb9f7180 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.797+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba00480 con 0x7f5bf8021020 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.798+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94321434946768] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.798+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.798+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94321434947824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eb9ff510 con 0x7f5bf8021020 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.798+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba01340 con 0x55c8eb9f2770 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.799+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.799+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94321434955408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba01760 con 0x55c8eb9f2770 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.799+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94321434947824] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.799+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba02280 con 0x7f5c00027020 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.800+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94321434955408] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.800+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.800+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94321434959184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba012b0 con 0x7f5c00027020 2024-09-19T11:23:27.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.800+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba02b20 con 0x7f5c00027020 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.801+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.801+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94321434961520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba012b0 con 0x7f5c00027020 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.801+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba03680 con 0x7f5c00027020 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.801+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94321434959184] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94321434953840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba03640 con 0x7f5c00027020 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 20 add_watcher() i=0 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 20 add_watcher() i=1 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 20 add_watcher() i=2 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 20 add_watcher() i=3 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 20 add_watcher() i=4 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 20 add_watcher() i=5 2024-09-19T11:23:27.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94321434961520] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.802+0000 7f5c0952c800 20 add_watcher() i=6 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.803+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94321434953840] v31'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.803+0000 7f5c0952c800 20 add_watcher() i=7 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.803+0000 7f5c0952c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.803+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba03640 con 0x55c8eb9f2770 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.803+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.803+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94321434964368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba035b0 con 0x55c8eb9f2770 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.804+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94321434964368] v31'144 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.804+0000 7f5abbfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.804+0000 7f5c0952c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.804+0000 7f5c0952c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.804+0000 7f5c0952c800 5 note: GC not initialized 2024-09-19T11:23:27.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5abafe5640 20 reqs_thread_entry: start 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0952c800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0952c800 20 Filter name: none 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5ab8fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba39500 con 0x7f5bf8021020 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.805+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba39d90 con 0x7f5c00027020 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(63 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:27.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba387f0 con 0x7f5bf8021020 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba3b3f0 con 0x7f5c00027020 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(65 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.806+0000 7f5c0952c800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 4.33 4:cfe5f5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba3ae90 con 0x7f5c00027020 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 4.8 4:13d14a70:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba3fd40 con 0x55c8eb9f7180 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 4.2b 4:d7493e99:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba3aed0 con 0x55c8eb9f2770 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 4.27 4:e79b5c58:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba44f00 con 0x55c8eb9f2770 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 4.20 4:06d6ec91:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba47d40 con 0x55c8eb9f2770 2024-09-19T11:23:27.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 4.f 4:f2b3320c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba4a670 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:72 4.d 4:b3b80c54:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba4d0b0 con 0x55c8eb9f7180 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(66 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 4.32 4:4d09bac8:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba4a670 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(68 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 4.3c 4:3c98ea8c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba40590 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(69 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(70 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 4.c 4:31f23ca3:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba41b60 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 4.31 4:8fabc4b1:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba421c0 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(71 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.5 [call out=128b] v0'0 uv9 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(72 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.6 [call out=128b] v0'0 uv7 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.807+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(75 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.9 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.808+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(73 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.7 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.808+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(74 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.8 [call out=128b] v0'0 uv5 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.808+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(76 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.1.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.808+0000 7f5abbfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.809+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94321434964368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.810+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94321434964368] v31'145 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.810+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94321434938144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x7f5bf8021020 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.810+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94321434938144] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.810+0000 7f5c0952c800 20 remove_watcher() i=0 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.810+0000 7f5c0952c800 2 removed watcher, disabling cache 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.810+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94321434945712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.811+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94321434945712] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.811+0000 7f5c0952c800 20 remove_watcher() i=1 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.811+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94321434946768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x55c8eb9f7180 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.812+0000 7f5c05a50640 1 -- 172.21.15.28:0/2510205059 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94321434946768] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf80083e0 con 0x55c8eb9f7180 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.812+0000 7f5c0952c800 20 remove_watcher() i=2 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.812+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94321434947824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x7f5bf8021020 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.813+0000 7f5c0524f640 1 -- 172.21.15.28:0/2510205059 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94321434947824] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5c0001b070 con 0x7f5bf8021020 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.813+0000 7f5c0952c800 20 remove_watcher() i=3 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.813+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94321434955408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x55c8eb9f2770 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.814+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94321434955408] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x55c8eb9f2770 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.814+0000 7f5c0952c800 20 remove_watcher() i=4 2024-09-19T11:23:27.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.814+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94321434959184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x7f5c00027020 2024-09-19T11:23:27.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.816+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94321434959184] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.816+0000 7f5c0952c800 20 remove_watcher() i=5 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.816+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94321434961520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x7f5c00027020 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.816+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94321434961520] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.816+0000 7f5c0952c800 20 remove_watcher() i=6 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.816+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94321434953840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55c8eba36ea0 con 0x7f5c00027020 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.817+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94321434953840] v31'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0051070 con 0x7f5c00027020 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 20 remove_watcher() i=7 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c00027020 msgr2=0x7f5c0001e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5c00027020 0x7f5c0001e3f0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb9f2770 msgr2=0x55c8eb9f4c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb9f2770 0x55c8eb9f4c30 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5bf8021020 msgr2=0x7f5bf8052580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5bf8021020 0x7f5bf8052580 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9f7180 msgr2=0x55c8eb9f9570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9f7180 0x55c8eb9f9570 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ad003cfb0 msgr2=0x7f5ad003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5ad003cfb0 0x7f5ad003f470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5c00014fd0 tx=0x7f5c00052d00 comp rx=0 tx=0).stop 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 msgr2=0x55c8eb9e7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c04a4e640 1 -- 172.21.15.28:0/2510205059 reap_dead start 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9e7010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80206f0 tx=0x7f5bf80088f0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 shutdown_connections 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/2510205059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9dfaa0 0x55c8eb9e7010 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 >> 172.21.15.28:0/2510205059 conn(0x55c8eb9dbea0 msgr2=0x55c8eb9da0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 shutdown_connections 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/2510205059 wait complete. 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5adc03d000 msgr2=0x7f5adc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5adc03d000 0x7f5adc03f4c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55c8eb9c4a50 tx=0x7f5c00006820 comp rx=0 tx=0).stop 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9d3040 msgr2=0x55c8eb9d3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9d3040 0x55c8eb9d3460 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55c8eb9b9040 tx=0x7f5bf800bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 shutdown_connections 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5adc03d000 0x7f5adc03f4c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 --2- 172.21.15.28:0/3806810915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb9d3040 0x55c8eb9d3460 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 >> 172.21.15.28:0/3806810915 conn(0x55c8eb9c73f0 msgr2=0x55c8eb9c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.818+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 shutdown_connections 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/3806810915 wait complete. 2024-09-19T11:23:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb834610 msgr2=0x55c8eb9bf690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb834610 0x55c8eb9bf690 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9c1a70 msgr2=0x55c8eb9c3ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9c1a70 0x55c8eb9c3ef0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5aec03d000 msgr2=0x7f5aec03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5aec03d000 0x7f5aec03f4c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0057000 tx=0x7f5bf0052000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 msgr2=0x55c8eb9b8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b8040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5c00000f80 tx=0x7f5c00001250 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 shutdown_connections 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c8eb9c1a70 0x55c8eb9c3ef0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f5aec03d000 0x7f5aec03f4c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c8eb834610 0x55c8eb9bf690 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 --2- 172.21.15.28:0/4024612057 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c8eb833280 0x55c8eb9b8040 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 >> 172.21.15.28:0/4024612057 conn(0x55c8eb8b9c60 msgr2=0x55c8eb9b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 shutdown_connections 2024-09-19T11:23:27.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.819+0000 7f5c0952c800 1 -- 172.21.15.28:0/4024612057 wait complete. 2024-09-19T11:23:27.832 INFO:teuthology.orchestra.run.smithi028.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"d3056e04-d444-48b9-8e61-96c58b445cdd","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","marker":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1","index_type":"Normal","index_generation":0,"num_shards":11,"reshard_status":"None","judge_reshard_lock_time":"0.000000","object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-09-19T11:23:27.289017Z","creation_time":"2024-09-19T11:23:24.614937Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":0,"size_actual":0,"size_utilized":0,"size_kb":0,"size_kb_actual":0,"size_kb_utilized":0,"num_objects":0}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"read_tracker":2} 2024-09-19T11:23:27.833 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:23:27.289017Z', 'creation_time': '2024-09-19T11:23:24.614937Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 2} 2024-09-19T11:23:27.833 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-09-19T11:23:27.833 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-09-19T11:23:27.833 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-09-19T11:23:27.876 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:27.877 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:27.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.876+0000 7fd317d0c800 1 Processor -- start 2024-09-19T11:23:27.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.876+0000 7fd317d0c800 1 -- start start 2024-09-19T11:23:27.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.876+0000 7fd317d0c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55886b5d5df0 0x55886b5db740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.876+0000 7fd317d0c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55886b5d62a0 con 0x55886b5d2cd0 2024-09-19T11:23:27.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.876+0000 7fd317d0c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55886b45f7d0 con 0x55886b5d5df0 2024-09-19T11:23:27.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.877+0000 7fd30ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55886b5d5df0 0x55886b5db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.877+0000 7fd3157f6640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55886b5d2cd0 0x55886b5d56e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48418/0 (socket says 172.21.15.28:48418) 2024-09-19T11:23:27.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.877+0000 7fd3157f6640 1 -- 172.21.15.28:0/288007556 learned_addr learned my addr 172.21.15.28:0/288007556 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:27.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.877+0000 7fd30ffff640 1 -- 172.21.15.28:0/288007556 >> v1:172.21.15.22:6789/0 conn(0x55886b5d2cd0 legacy=0x55886b5d56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.877+0000 7fd314ff5640 1 -- 172.21.15.28:0/288007556 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1135132549 0 0) 0x55886b5d62a0 con 0x55886b5d2cd0 2024-09-19T11:23:27.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.877+0000 7fd30ffff640 1 -- 172.21.15.28:0/288007556 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886b5d6470 con 0x55886b5d5df0 2024-09-19T11:23:27.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd30ffff640 1 --2- 172.21.15.28:0/288007556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d5df0 0x55886b5db740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd30000a470 tx=0x7fd300045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4562b86d96c7bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd314ff5640 1 -- 172.21.15.28:0/288007556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd300053570 con 0x55886b5d5df0 2024-09-19T11:23:27.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd314ff5640 1 -- 172.21.15.28:0/288007556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd300058070 con 0x55886b5d5df0 2024-09-19T11:23:27.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd314ff5640 1 -- 172.21.15.28:0/288007556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd300053880 con 0x55886b5d5df0 2024-09-19T11:23:27.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd317d0c800 1 -- 172.21.15.28:0/288007556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d5df0 msgr2=0x55886b5db740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd317d0c800 1 --2- 172.21.15.28:0/288007556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d5df0 0x55886b5db740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd30000a470 tx=0x7fd300045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:27.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd317d0c800 1 -- 172.21.15.28:0/288007556 shutdown_connections 2024-09-19T11:23:27.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd317d0c800 1 --2- 172.21.15.28:0/288007556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d5df0 0x55886b5db740 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd317d0c800 1 -- 172.21.15.28:0/288007556 >> 172.21.15.28:0/288007556 conn(0x55886b5ce1b0 msgr2=0x55886b5d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd317d0c800 1 -- 172.21.15.28:0/288007556 shutdown_connections 2024-09-19T11:23:27.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.879+0000 7fd317d0c800 1 -- 172.21.15.28:0/288007556 wait complete. 2024-09-19T11:23:27.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd317d0c800 1 Processor -- start 2024-09-19T11:23:27.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd317d0c800 1 -- start start 2024-09-19T11:23:27.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd317d0c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd317d0c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55886b46cdd0 con 0x55886b5d2cd0 2024-09-19T11:23:27.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd3157f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd3157f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34842/0 (socket says 172.21.15.28:34842) 2024-09-19T11:23:27.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd3157f6640 1 -- 172.21.15.28:0/2841367590 learned_addr learned my addr 172.21.15.28:0/2841367590 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.895+0000 7fd3157f6640 1 -- 172.21.15.28:0/2841367590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886b6c4990 con 0x55886b5d2cd0 2024-09-19T11:23:27.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd3157f6640 1 --2- 172.21.15.28:0/2841367590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6c43a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd308001ea0 tx=0x7fd30800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9861ad840835793 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd203606640 1 -- 172.21.15.28:0/2841367590 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd308008070 con 0x55886b5d2cd0 2024-09-19T11:23:27.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd203606640 1 -- 172.21.15.28:0/2841367590 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30800d500 con 0x55886b5d2cd0 2024-09-19T11:23:27.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd203606640 1 -- 172.21.15.28:0/2841367590 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30800d880 con 0x55886b5d2cd0 2024-09-19T11:23:27.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd317d0c800 1 -- 172.21.15.28:0/2841367590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 msgr2=0x55886b6c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2841367590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6c43a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd308001ea0 tx=0x7fd30800b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd317d0c800 1 -- 172.21.15.28:0/2841367590 shutdown_connections 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2841367590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6c43a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.896+0000 7fd317d0c800 1 -- 172.21.15.28:0/2841367590 >> 172.21.15.28:0/2841367590 conn(0x55886b5ce1b0 msgr2=0x55886b6bfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd317d0c800 1 -- 172.21.15.28:0/2841367590 shutdown_connections 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd317d0c800 1 -- 172.21.15.28:0/2841367590 wait complete. 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd317d0c800 1 Processor -- start 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd317d0c800 1 -- start start 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd317d0c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd317d0c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55886b5dbf80 con 0x55886b5d2cd0 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd3157f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd3157f6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34852/0 (socket says 172.21.15.28:34852) 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd3157f6640 1 -- 172.21.15.28:0/583326351 learned_addr learned my addr 172.21.15.28:0/583326351 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.897+0000 7fd3157f6640 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886b6cc3e0 con 0x55886b5d2cd0 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.898+0000 7fd3157f6640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6cbe80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd3080012f0 tx=0x7fd308000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.898+0000 7fd201e03640 1 -- 172.21.15.28:0/583326351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30800f9d0 con 0x55886b5d2cd0 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.898+0000 7fd201e03640 1 -- 172.21.15.28:0/583326351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd30800fb70 con 0x55886b5d2cd0 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.898+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55886b6cc670 con 0x55886b5d2cd0 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.898+0000 7fd201e03640 1 -- 172.21.15.28:0/583326351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30800fe50 con 0x55886b5d2cd0 2024-09-19T11:23:27.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.898+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886b6cd070 con 0x55886b5d2cd0 2024-09-19T11:23:27.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.899+0000 7fd201e03640 1 -- 172.21.15.28:0/583326351 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd308017040 con 0x55886b5d2cd0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.899+0000 7fd201e03640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1fc03d150 0x7fd1fc03f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.899+0000 7fd201e03640 1 -- 172.21.15.28:0/583326351 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd308015020 con 0x55886b5d2cd0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.899+0000 7fd30ffff640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1fc03d150 0x7fd1fc03f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.899+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b5d5df0 0x55886b6d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.899+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b6d3a10 con 0x55886b5d5df0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.899+0000 7fd315ff7640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b5d5df0 0x55886b6d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd30ffff640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1fc03d150 0x7fd1fc03f610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd300057000 tx=0x7fd300052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd315ff7640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b5d5df0 0x55886b6d34d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd315ff7640 1 -- 172.21.15.28:0/583326351 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b5d5df0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b6d3db0 con 0x55886b5d5df0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd315ff7640 1 -- 172.21.15.28:0/583326351 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b5d5df0 2024-09-19T11:23:27.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b6d58b0 0x55886b6d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b6d84c0 con 0x55886b6d58b0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.900+0000 7fd3157f6640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b6d58b0 0x55886b6d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.901+0000 7fd3157f6640 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b6d58b0 0x55886b6d7d30 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.901+0000 7fd3157f6640 1 -- 172.21.15.28:0/583326351 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd30802d070 con 0x55886b6d58b0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.901+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b6d4600 con 0x55886b5d5df0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.901+0000 7fd315ff7640 1 -- 172.21.15.28:0/583326351 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b5d5df0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.901+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b6d4a20 con 0x55886b6d58b0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd3157f6640 1 -- 172.21.15.28:0/583326351 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd30802d070 con 0x55886b6d58b0 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd317d0c800 1 Processor -- start 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd317d0c800 1 -- start start 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd317d0c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b6e6e50 0x55886b6e7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd317d0c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55886b4cb4a0 con 0x55886b6e6e50 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd315ff7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b6e6e50 0x55886b6e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd315ff7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b6e6e50 0x55886b6e7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34864/0 (socket says 172.21.15.28:34864) 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.902+0000 7fd315ff7640 1 -- 172.21.15.28:0/1747479460 learned_addr learned my addr 172.21.15.28:0/1747479460 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.903+0000 7fd315ff7640 1 -- 172.21.15.28:0/1747479460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886b6e78b0 con 0x55886b6e6e50 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.903+0000 7fd315ff7640 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b6e6e50 0x55886b6e7270 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55886b6cce80 tx=0x7fd3100167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.903+0000 7fd1faffd640 1 -- 172.21.15.28:0/1747479460 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd310013640 con 0x55886b6e6e50 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.903+0000 7fd1faffd640 1 -- 172.21.15.28:0/1747479460 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3100137e0 con 0x55886b6e6e50 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.903+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55886b6e80e0 con 0x55886b6e6e50 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.903+0000 7fd1faffd640 1 -- 172.21.15.28:0/1747479460 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd310013ae0 con 0x55886b6e6e50 2024-09-19T11:23:27.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.903+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886b6e8680 con 0x55886b6e6e50 2024-09-19T11:23:27.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.904+0000 7fd1faffd640 1 -- 172.21.15.28:0/1747479460 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd310011040 con 0x55886b6e6e50 2024-09-19T11:23:27.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.904+0000 7fd1faffd640 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1ec03d020 0x7fd1ec03f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.904+0000 7fd1faffd640 1 -- 172.21.15.28:0/1747479460 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd31000e040 con 0x55886b6e6e50 2024-09-19T11:23:27.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.904+0000 7fd3157f6640 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1ec03d020 0x7fd1ec03f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd317d0c800 1 Processor -- start 2024-09-19T11:23:27.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd3157f6640 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1ec03d020 0x7fd1ec03f4e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55886b6d8890 tx=0x7fd308053170 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd317d0c800 1 -- start start 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd317d0c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b5d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd317d0c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55886b6c4dc0 con 0x55886b5d36e0 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd315ff7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b5d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd315ff7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b5d3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34880/0 (socket says 172.21.15.28:34880) 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd315ff7640 1 -- 172.21.15.28:0/2390719298 learned_addr learned my addr 172.21.15.28:0/2390719298 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd315ff7640 1 -- 172.21.15.28:0/2390719298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886b5d40d0 con 0x55886b5d36e0 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd315ff7640 1 --2- 172.21.15.28:0/2390719298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b5d3ae0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd310054840 tx=0x7fd3100204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9a6015cfcd82e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.905+0000 7fd1f8ff9640 1 -- 172.21.15.28:0/2390719298 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd310051070 con 0x55886b5d36e0 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd1f8ff9640 1 -- 172.21.15.28:0/2390719298 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd310053440 con 0x55886b5d36e0 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd1f8ff9640 1 -- 172.21.15.28:0/2390719298 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd310021a80 con 0x55886b5d36e0 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 -- 172.21.15.28:0/2390719298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 msgr2=0x55886b5d3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2390719298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b5d3ae0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd310054840 tx=0x7fd3100204d0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 -- 172.21.15.28:0/2390719298 shutdown_connections 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2390719298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b5d3ae0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 -- 172.21.15.28:0/2390719298 >> 172.21.15.28:0/2390719298 conn(0x55886b6efd10 msgr2=0x55886b6edf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 -- 172.21.15.28:0/2390719298 shutdown_connections 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 -- 172.21.15.28:0/2390719298 wait complete. 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 Processor -- start 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 -- start start 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b6fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.906+0000 7fd317d0c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55886b5d4740 con 0x55886b5d36e0 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd315ff7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b6fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd315ff7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b6fd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34890/0 (socket says 172.21.15.28:34890) 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 learned_addr learned my addr 172.21.15.28:0/2349454215 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:27.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886b6fd640 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd315ff7640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b6fd0e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55886b6eecc0 tx=0x7fd310045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd1eaffd640 1 -- 172.21.15.28:0/2349454215 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3100459e0 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd1eaffd640 1 -- 172.21.15.28:0/2349454215 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd310045bc0 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55886b6fd8d0 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd1eaffd640 1 -- 172.21.15.28:0/2349454215 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3100451d0 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.907+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886b6fd8d0 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.908+0000 7fd1eaffd640 1 -- 172.21.15.28:0/2349454215 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd310045370 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.908+0000 7fd1eaffd640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1e003cfb0 0x7fd1e003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.908+0000 7fd1eaffd640 1 -- 172.21.15.28:0/2349454215 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fd310034c80 con 0x55886b5d36e0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.909+0000 7fd3157f6640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1e003cfb0 0x7fd1e003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.909+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.909+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b7087c0 0x55886b70ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.909+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b70b3d0 con 0x55886b7087c0 2024-09-19T11:23:27.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.909+0000 7fd30ffff640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b7087c0 0x55886b70ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.909+0000 7fd3157f6640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1e003cfb0 0x7fd1e003f470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd308019fd0 tx=0x7fd308048000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.909+0000 7fd30ffff640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b7087c0 0x55886b70ac80 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 realm 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b70b800 con 0x55886b7087c0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b70c0d0 con 0x55886b7087c0 2024-09-19T11:23:27.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b70c470 con 0x55886b7087c0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b70d1d0 0x55886b70f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b70fb00 con 0x55886b70d1d0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.910+0000 7fd315ff7640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b70d1d0 0x55886b70f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd315ff7640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b70d1d0 0x55886b70f5c0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd317d0c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:27.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd317d0c800 20 searching for the correct realm 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd315ff7640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd310054c20 0x7fd3100323a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd310054c20 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.911+0000 7fd3157f6640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd310054c20 0x7fd3100323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.912+0000 7fd3157f6640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd310054c20 0x7fd3100323a0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.912+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd30802d070 con 0x7fd310054c20 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.912+0000 7fd3157f6640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3080535b0 0x7fd3080539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.912+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.912+0000 7fd30ffff640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3080535b0 0x7fd3080539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.912+0000 7fd30ffff640 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3080535b0 0x7fd3080539d0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.913+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd310054c20 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd30802d070 con 0x7fd310054c20 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.914+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd310054c20 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd30802d070 con 0x7fd310054c20 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.915+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.916+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.917+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.917+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.917+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.917+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.917+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.917+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.918+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.918+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd310054c20 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.918+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd30802d070 con 0x7fd310054c20 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.918+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.918+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.918+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.919+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.919+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.919+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.919+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.919+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.919+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd310054c20 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.919+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd30802d070 con 0x7fd310054c20 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.920+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b70d1d0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.920+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.920+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd3080535b0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.920+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.920+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x55886b7087c0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.920+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.920+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b711470 con 0x7fd310054c20 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:27.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b710540 con 0x55886b7087c0 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b710de0 con 0x55886b7087c0 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b710180 con 0x55886b70d1d0 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 zone default found 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 4 Realm: () 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 zonegroup default 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.921+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b710a20 con 0x7fd3080535b0 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd317d0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b713d10 con 0x55886b70d1d0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd317d0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd317d0c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.922+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b7140e0 con 0x55886b70d1d0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.923+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.923+0000 7fd317d0c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.923+0000 7fd317d0c800 20 started sync module instance, tier type = 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.923+0000 7fd317d0c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.923+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b7131a0 con 0x7fd310054c20 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.924+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.924+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94044406494608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b7149d0 con 0x7fd310054c20 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.924+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b715220 con 0x55886b7087c0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.925+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94044406494608] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.925+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.925+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94044406498592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b7152a0 con 0x55886b7087c0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.925+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b715dc0 con 0x55886b70d1d0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.926+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94044406498592] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.926+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.926+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94044406503568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b715d80 con 0x55886b70d1d0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.926+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b716840 con 0x7fd310054c20 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.927+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94044406503568] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.927+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.927+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94044406506320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd310054c20 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.927+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b717650 con 0x55886b7087c0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.928+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94044406506320] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.928+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.928+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94044406509888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b717a70 con 0x55886b7087c0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.928+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b718490 con 0x7fd3080535b0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.929+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94044406509888] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.929+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.929+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94044406513536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b718450 con 0x7fd3080535b0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.929+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b718ea0 con 0x7fd3080535b0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.930+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(56 notify.6 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.930+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94044406516208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b7192c0 con 0x7fd3080535b0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.930+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b719de0 con 0x7fd3080535b0 2024-09-19T11:23:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.930+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94044406513536] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94044406516208] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94044406519984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b719da0 con 0x7fd3080535b0 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 20 add_watcher() i=0 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 20 add_watcher() i=1 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 20 add_watcher() i=2 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 20 add_watcher() i=3 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 20 add_watcher() i=4 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 20 add_watcher() i=5 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.931+0000 7fd317d0c800 20 add_watcher() i=6 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.932+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94044406519984] v31'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.932+0000 7fd317d0c800 20 add_watcher() i=7 2024-09-19T11:23:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.932+0000 7fd317d0c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.932+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55886b718e70 con 0x55886b7087c0 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.933+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.933+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94044406500720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b718e70 con 0x55886b7087c0 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94044406500720] v31'146 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd1cbfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd317d0c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd317d0c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd317d0c800 5 note: GC not initialized 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd1cafe5640 20 reqs_thread_entry: start 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd317d0c800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd317d0c800 20 Filter name: none 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.934+0000 7fd1c8fe1640 20 reqs_thread_entry: start 2024-09-19T11:23:27.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.935+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 6.0 6:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b74f290 con 0x7fd3080535b0 2024-09-19T11:23:27.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.935+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(62 [pgnls start_epoch 31 out=254b] v31'35 uv18 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.935+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.10 6:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b710a20 con 0x7fd3080535b0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.935+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(63 [pgnls start_epoch 31 out=131b] v31'4 uv4 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.935+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.8 6:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b710a20 con 0x55886b70d1d0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 [pgnls start_epoch 31 out=295b] v31'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 6.18 6:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b7140e0 con 0x7fd3080535b0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(65 [pgnls start_epoch 31 out=172b] v31'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b7140e0 con 0x55886b7087c0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(66 [pgnls start_epoch 31 out=335b] v31'146 uv23 ondisk = 1) ==== 144+0+335 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 6.14 6:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b7140e0 con 0x7fd310054c20 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(67 [pgnls start_epoch 31 out=213b] v31'10 uv10 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.936+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 6.c 6:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.937+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 [pgnls start_epoch 31 out=254b] v31'10 uv10 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.937+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 6.1c 6:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.937+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(69 [pgnls start_epoch 31 out=295b] v31'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.937+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:70 6.2 6:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.937+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(70 [pgnls start_epoch 31 out=213b] v31'11 uv10 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.937+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 6.12 6:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd310054c20 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(71 [pgnls start_epoch 31 out=172b] v31'18 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.a 6:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd310054c20 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(72 [pgnls start_epoch 31 out=336b] v31'27 uv19 ondisk = 1) ==== 144+0+336 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 6.1a 6:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(73 [pgnls start_epoch 31 out=131b] v31'17 uv9 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:74 6.6 6:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(74 [pgnls start_epoch 31 out=254b] v31'12 uv12 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.938+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 6.16 6:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.939+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(75 [pgnls start_epoch 31 out=90b] v31'17 uv9 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.939+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.e 6:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b7087c0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.939+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(76 [pgnls start_epoch 31 out=213b] v31'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.939+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.1e 6:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.939+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(77 [pgnls start_epoch 31 out=254b] v31'10 uv10 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.939+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:78 6.1 6:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b7087c0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(78 [pgnls start_epoch 31 out=254b] v31'14 uv14 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.11 6:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(79 [pgnls start_epoch 31 out=213b] v31'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:80 6.9 6:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(80 [pgnls start_epoch 31 out=172b] v31'28 uv12 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.19 6:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b7087c0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(81 [pgnls start_epoch 31 out=295b] v31'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.940+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:82 6.5 6:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd310054c20 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.941+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(82 [pgnls start_epoch 31 out=276b] v31'15 uv15 ondisk = 1) ==== 144+0+276 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.941+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.15 6:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd310054c20 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.941+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(83 [pgnls start_epoch 31 out=172b] v31'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.941+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:84 6.d 6:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b7087c0 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.941+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(84 [pgnls start_epoch 31 out=295b] v31'25 uv17 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.941+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:85 6.1d 6:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd3080535b0 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.942+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(85 [pgnls start_epoch 31 out=172b] v31'28 uv12 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.942+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:86 6.3 6:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd310054c20 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.942+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(86 [pgnls start_epoch 31 out=213b] v31'43 uv19 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.942+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:87 6.13 6:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd3080535b0 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.942+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(87 [pgnls start_epoch 31 out=172b] v31'19 uv11 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd300051070 con 0x7fd3080535b0 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.942+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:88 6.b 6:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.943+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(88 [pgnls start_epoch 31 out=172b] v31'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.943+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1b 6:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.943+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(89 [pgnls start_epoch 31 out=90b] v31'13 uv5 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.943+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:90 6.7 6:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b70d1d0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.943+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(90 [pgnls start_epoch 31 out=90b] v31'37 uv13 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.943+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:91 6.17 6:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x55886b7087c0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.944+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(91 [pgnls start_epoch 31 out=131b] v31'6 uv6 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7fd300051070 con 0x55886b7087c0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.944+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:92 6.f 6:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd310054c20 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.944+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(92 [pgnls start_epoch 31 out=295b] v31'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.944+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:93 6.1f 6:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x55886b716c30 con 0x7fd3080535b0 2024-09-19T11:23:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.944+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(93 [pgnls start_epoch 31 out=213b] v31'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7fd3000990e0 con 0x7fd3080535b0 2024-09-19T11:23:27.956 INFO:teuthology.orchestra.run.smithi028.stdout:["obj_delete_at_hint.0000000078","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","data_loggenerations_metadata","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","obj_delete_at_hint.0000000027","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","data_log.0","obj_delete_at_hint.0000000012","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2024-09-19T11:23:27.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.946+0000 7fd1cbfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:27.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.946+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94044406500720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b710a20 con 0x55886b7087c0 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.947+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94044406500720] v31'147 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd3000990e0 con 0x55886b7087c0 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.947+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94044406494608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b7140e0 con 0x7fd310054c20 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.948+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94044406494608] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.948+0000 7fd317d0c800 20 remove_watcher() i=0 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.948+0000 7fd317d0c800 2 removed watcher, disabling cache 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.948+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94044406498592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b74ece0 con 0x55886b7087c0 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.949+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94044406498592] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3000990e0 con 0x55886b7087c0 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.949+0000 7fd317d0c800 20 remove_watcher() i=1 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.949+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94044406503568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b710a20 con 0x55886b70d1d0 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.950+0000 7fd315ff7640 1 -- 172.21.15.28:0/2349454215 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94044406503568] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd31000a4d0 con 0x55886b70d1d0 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.950+0000 7fd317d0c800 20 remove_watcher() i=2 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.950+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94044406506320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b74ece0 con 0x7fd310054c20 2024-09-19T11:23:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.951+0000 7fd3157f6640 1 -- 172.21.15.28:0/2349454215 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94044406506320] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd308061040 con 0x7fd310054c20 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.951+0000 7fd317d0c800 20 remove_watcher() i=3 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.951+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94044406509888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b74ece0 con 0x55886b7087c0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.952+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94044406509888] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3000990e0 con 0x55886b7087c0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.952+0000 7fd317d0c800 20 remove_watcher() i=4 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.952+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94044406513536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b74ece0 con 0x7fd3080535b0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.953+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94044406513536] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3000990e0 con 0x7fd3080535b0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.953+0000 7fd317d0c800 20 remove_watcher() i=5 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.954+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94044406516208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b74ece0 con 0x7fd3080535b0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.955+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94044406516208] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3000990e0 con 0x7fd3080535b0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.955+0000 7fd317d0c800 20 remove_watcher() i=6 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.955+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94044406519984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55886b74ece0 con 0x7fd3080535b0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94044406519984] v31'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd3000990e0 con 0x7fd3080535b0 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 20 remove_watcher() i=7 2024-09-19T11:23:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3080535b0 msgr2=0x7fd3080539d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd3080535b0 0x7fd3080539d0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b7087c0 msgr2=0x55886b70ac80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b7087c0 0x55886b70ac80 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd310054c20 msgr2=0x7fd3100323a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd310054c20 0x7fd3100323a0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b70d1d0 msgr2=0x55886b70f5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b70d1d0 0x55886b70f5c0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1e003cfb0 msgr2=0x7fd1e003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1e003cfb0 0x7fd1e003f470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd308019fd0 tx=0x7fd308048000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 msgr2=0x55886b6fd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b6fd0e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55886b6eecc0 tx=0x7fd310045000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd30ffff640 1 -- 172.21.15.28:0/2349454215 reap_dead start 2024-09-19T11:23:27.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 shutdown_connections 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/2349454215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d36e0 0x55886b6fd0e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 >> 172.21.15.28:0/2349454215 conn(0x55886b6efd10 msgr2=0x55886b6edf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 shutdown_connections 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/2349454215 wait complete. 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1ec03d020 msgr2=0x7fd1ec03f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1ec03d020 0x7fd1ec03f4e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55886b6d8890 tx=0x7fd308053170 comp rx=0 tx=0).stop 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b6e6e50 msgr2=0x55886b6e7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.956+0000 7fd317d0c800 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b6e6e50 0x55886b6e7270 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x55886b6cce80 tx=0x7fd3100167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 shutdown_connections 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1ec03d020 0x7fd1ec03f4e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/1747479460 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b6e6e50 0x55886b6e7270 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 >> 172.21.15.28:0/1747479460 conn(0x55886b6db200 msgr2=0x55886b6db5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 shutdown_connections 2024-09-19T11:23:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/1747479460 wait complete. 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b5d5df0 msgr2=0x55886b6d34d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b5d5df0 0x55886b6d34d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b6d58b0 msgr2=0x55886b6d7d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b6d58b0 0x55886b6d7d30 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1fc03d150 msgr2=0x7fd1fc03f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1fc03d150 0x7fd1fc03f610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd300057000 tx=0x7fd300052000 comp rx=0 tx=0).stop 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 msgr2=0x55886b6cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6cbe80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd3080012f0 tx=0x7fd308000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 shutdown_connections 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55886b6d58b0 0x55886b6d7d30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd1fc03d150 0x7fd1fc03f610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55886b5d5df0 0x55886b6d34d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 --2- 172.21.15.28:0/583326351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55886b5d2cd0 0x55886b6cbe80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 >> 172.21.15.28:0/583326351 conn(0x55886b5ce1b0 msgr2=0x55886b6c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 shutdown_connections 2024-09-19T11:23:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:27.957+0000 7fd317d0c800 1 -- 172.21.15.28:0/583326351 wait complete. 2024-09-19T11:23:27.968 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-09-19T11:23:27.968 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-09-19T11:23:27.968 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-09-19T11:23:27.968 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-09-19T11:23:28.053 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:28.053 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.053+0000 7f1703140800 1 Processor -- start 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.053+0000 7f1703140800 1 -- start start 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.053+0000 7f1703140800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558813ee3690 0x558813ee4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.053+0000 7f1703140800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558813ee3d40 con 0x558813ee48d0 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.053+0000 7f1703140800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x558813d6f7d0 con 0x558813ee3690 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.054+0000 7f16feeed640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558813ee3690 0x558813ee4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.054+0000 7f16feeed640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558813ee3690 0x558813ee4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34898/0 (socket says 172.21.15.28:34898) 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.054+0000 7f16fe6ec640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x558813ee48d0 0x558813ee9630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48420/0 (socket says 172.21.15.28:48420) 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.054+0000 7f16feeed640 1 -- 172.21.15.28:0/2475130652 learned_addr learned my addr 172.21.15.28:0/2475130652 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.054+0000 7f16feeed640 1 -- 172.21.15.28:0/2475130652 >> v1:172.21.15.22:6789/0 conn(0x558813ee48d0 legacy=0x558813ee9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.054+0000 7f16feeed640 1 -- 172.21.15.28:0/2475130652 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558813ee4d80 con 0x558813ee3690 2024-09-19T11:23:28.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f16feeed640 1 --2- 172.21.15.28:0/2475130652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813ee4390 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f16f000a6b0 tx=0x7f16f0047e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501811ea7821860a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f16fdeeb640 1 -- 172.21.15.28:0/2475130652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f000a200 con 0x558813ee3690 2024-09-19T11:23:28.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f16fdeeb640 1 -- 172.21.15.28:0/2475130652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f000a3a0 con 0x558813ee3690 2024-09-19T11:23:28.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f16fdeeb640 1 -- 172.21.15.28:0/2475130652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0008700 con 0x558813ee3690 2024-09-19T11:23:28.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f1703140800 1 -- 172.21.15.28:0/2475130652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 msgr2=0x558813ee4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f1703140800 1 --2- 172.21.15.28:0/2475130652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813ee4390 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f16f000a6b0 tx=0x7f16f0047e30 comp rx=0 tx=0).stop 2024-09-19T11:23:28.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f1703140800 1 -- 172.21.15.28:0/2475130652 shutdown_connections 2024-09-19T11:23:28.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f1703140800 1 --2- 172.21.15.28:0/2475130652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813ee4390 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f1703140800 1 -- 172.21.15.28:0/2475130652 >> 172.21.15.28:0/2475130652 conn(0x558813edea60 msgr2=0x558813ee0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f1703140800 1 -- 172.21.15.28:0/2475130652 shutdown_connections 2024-09-19T11:23:28.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.056+0000 7f1703140800 1 -- 172.21.15.28:0/2475130652 wait complete. 2024-09-19T11:23:28.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.072+0000 7f1703140800 1 Processor -- start 2024-09-19T11:23:28.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.072+0000 7f1703140800 1 -- start start 2024-09-19T11:23:28.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.072+0000 7f1703140800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fd20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.072+0000 7f1703140800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558813d7cdd0 con 0x558813ee3690 2024-09-19T11:23:28.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.072+0000 7f16feeed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fd20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.072+0000 7f16feeed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fd20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34902/0 (socket says 172.21.15.28:34902) 2024-09-19T11:23:28.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.072+0000 7f16feeed640 1 -- 172.21.15.28:0/431189287 learned_addr learned my addr 172.21.15.28:0/431189287 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.073+0000 7f16feeed640 1 -- 172.21.15.28:0/431189287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558813fd26a0 con 0x558813ee3690 2024-09-19T11:23:28.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.073+0000 7f16feeed640 1 --2- 172.21.15.28:0/431189287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fd20b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f16f0048360 tx=0x7f16f0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d25f3b578af0493 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.073+0000 7f15ece09640 1 -- 172.21.15.28:0/431189287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f000a200 con 0x558813ee3690 2024-09-19T11:23:28.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.073+0000 7f15ece09640 1 -- 172.21.15.28:0/431189287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f0065070 con 0x558813ee3690 2024-09-19T11:23:28.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.073+0000 7f15ece09640 1 -- 172.21.15.28:0/431189287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f00086e0 con 0x558813ee3690 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 -- 172.21.15.28:0/431189287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 msgr2=0x558813fd20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 --2- 172.21.15.28:0/431189287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fd20b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f16f0048360 tx=0x7f16f0051000 comp rx=0 tx=0).stop 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 -- 172.21.15.28:0/431189287 shutdown_connections 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 --2- 172.21.15.28:0/431189287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fd20b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 -- 172.21.15.28:0/431189287 >> 172.21.15.28:0/431189287 conn(0x558813edea60 msgr2=0x558813fcda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 -- 172.21.15.28:0/431189287 shutdown_connections 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 -- 172.21.15.28:0/431189287 wait complete. 2024-09-19T11:23:28.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 Processor -- start 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 -- start start 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fddf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.074+0000 7f1703140800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558813e56eb0 con 0x558813ee3690 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.075+0000 7f16feeed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fddf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.075+0000 7f16feeed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fddf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34920/0 (socket says 172.21.15.28:34920) 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.075+0000 7f16feeed640 1 -- 172.21.15.28:0/3333339461 learned_addr learned my addr 172.21.15.28:0/3333339461 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.075+0000 7f16feeed640 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558813fde490 con 0x558813ee3690 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.075+0000 7f16feeed640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fddf30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000c00 tx=0x7f16f0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.075+0000 7f15eb606640 1 -- 172.21.15.28:0/3333339461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f004fcc0 con 0x558813ee3690 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.075+0000 7f15eb606640 1 -- 172.21.15.28:0/3333339461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f006a400 con 0x558813ee3690 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.076+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558813fde720 con 0x558813ee3690 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.076+0000 7f15eb606640 1 -- 172.21.15.28:0/3333339461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f004fe60 con 0x558813ee3690 2024-09-19T11:23:28.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.076+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558813fdf330 con 0x558813ee3690 2024-09-19T11:23:28.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f15eb606640 1 -- 172.21.15.28:0/3333339461 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f16f006a720 con 0x558813ee3690 2024-09-19T11:23:28.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f15eb606640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15e403d000 0x7f15e403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f15eb606640 1 -- 172.21.15.28:0/3333339461 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f16f005d040 con 0x558813ee3690 2024-09-19T11:23:28.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f16fe6ec640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15e403d000 0x7f15e403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558813ee48d0 0x558813fe5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558813fe5ac0 con 0x558813ee48d0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558813ee48d0 0x558813fe5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.077+0000 7f16fe6ec640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15e403d000 0x7f15e403f4c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16f4001e30 tx=0x7f16f4001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558813ee48d0 0x558813fe5580 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f16ff6ee640 1 -- 172.21.15.28:0/3333339461 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16f800a4d0 con 0x558813ee48d0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558813fe5e60 con 0x558813ee48d0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f16ff6ee640 1 -- 172.21.15.28:0/3333339461 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f16f800a4d0 con 0x558813ee48d0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558813fe7920 0x558813fe9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558813fea530 con 0x558813fe7920 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f16feeed640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558813fe7920 0x558813fe9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.078+0000 7f16feeed640 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558813fe7920 0x558813fe9da0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.079+0000 7f16feeed640 1 -- 172.21.15.28:0/3333339461 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f16f0072920 con 0x558813fe7920 2024-09-19T11:23:28.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.079+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558813fe66b0 con 0x558813ee48d0 2024-09-19T11:23:28.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.079+0000 7f16ff6ee640 1 -- 172.21.15.28:0/3333339461 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f16f800a4d0 con 0x558813ee48d0 2024-09-19T11:23:28.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.079+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558813fe6a50 con 0x558813fe7920 2024-09-19T11:23:28.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f16feeed640 1 -- 172.21.15.28:0/3333339461 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f16f0072920 con 0x558813fe7920 2024-09-19T11:23:28.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f1703140800 1 Processor -- start 2024-09-19T11:23:28.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f1703140800 1 -- start start 2024-09-19T11:23:28.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f1703140800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ff8eb0 0x558813ff92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f1703140800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558813ddb4a0 con 0x558813ff8eb0 2024-09-19T11:23:28.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f16ff6ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ff8eb0 0x558813ff92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f16ff6ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ff8eb0 0x558813ff92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34924/0 (socket says 172.21.15.28:34924) 2024-09-19T11:23:28.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.080+0000 7f16ff6ee640 1 -- 172.21.15.28:0/1571531299 learned_addr learned my addr 172.21.15.28:0/1571531299 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.081+0000 7f16ff6ee640 1 -- 172.21.15.28:0/1571531299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558813ff9910 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.081+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ff8eb0 0x558813ff92d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x558813fdef30 tx=0x7f16f80167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.081+0000 7f15e8e01640 1 -- 172.21.15.28:0/1571531299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f8013640 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.081+0000 7f15e8e01640 1 -- 172.21.15.28:0/1571531299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f80137e0 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.081+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558813ffa140 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.081+0000 7f15e8e01640 1 -- 172.21.15.28:0/1571531299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f8013ae0 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.081+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558813ffa6e0 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.082+0000 7f15e8e01640 1 -- 172.21.15.28:0/1571531299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f16f8011040 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.082+0000 7f15e8e01640 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15d803d000 0x7f15d803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.082+0000 7f15e8e01640 1 -- 172.21.15.28:0/1571531299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f16f800e040 con 0x558813ff8eb0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.082+0000 7f16feeed640 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15d803d000 0x7f15d803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.082+0000 7f1703140800 1 Processor -- start 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f16feeed640 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15d803d000 0x7f15d803f4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16f00077e0 tx=0x7f16f0096cf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f1703140800 1 -- start start 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f1703140800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x558814005d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f1703140800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558813fd2a10 con 0x558814005910 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f16ff6ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x558814005d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f16ff6ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x558814005d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34928/0 (socket says 172.21.15.28:34928) 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f16ff6ee640 1 -- 172.21.15.28:0/2119227420 learned_addr learned my addr 172.21.15.28:0/2119227420 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f16ff6ee640 1 -- 172.21.15.28:0/2119227420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558814006300 con 0x558814005910 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/2119227420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x558814005d10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f16f80204e0 tx=0x7f16f801fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaf1f74d3b853d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f15e27fc640 1 -- 172.21.15.28:0/2119227420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f8051070 con 0x558814005910 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f15e27fc640 1 -- 172.21.15.28:0/2119227420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f801fdb0 con 0x558814005910 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.083+0000 7f15e27fc640 1 -- 172.21.15.28:0/2119227420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f8053440 con 0x558814005910 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 -- 172.21.15.28:0/2119227420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 msgr2=0x558814005d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 --2- 172.21.15.28:0/2119227420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x558814005d10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f16f80204e0 tx=0x7f16f801fc90 comp rx=0 tx=0).stop 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 -- 172.21.15.28:0/2119227420 shutdown_connections 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 --2- 172.21.15.28:0/2119227420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x558814005d10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 -- 172.21.15.28:0/2119227420 >> 172.21.15.28:0/2119227420 conn(0x558814001d30 msgr2=0x558813ffff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 -- 172.21.15.28:0/2119227420 shutdown_connections 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 -- 172.21.15.28:0/2119227420 wait complete. 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 Processor -- start 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 -- start start 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x55881400cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.084+0000 7f1703140800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558814006920 con 0x558814005910 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f16ff6ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x55881400cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f16ff6ee640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x55881400cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34932/0 (socket says 172.21.15.28:34932) 2024-09-19T11:23:28.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 learned_addr learned my addr 172.21.15.28:0/438970898 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55881400d460 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x55881400cf00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f16f80204e0 tx=0x7f16f8054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f15e0ff9640 1 -- 172.21.15.28:0/438970898 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f8054a00 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f15e0ff9640 1 -- 172.21.15.28:0/438970898 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16f80459d0 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f15e0ff9640 1 -- 172.21.15.28:0/438970898 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f8045cd0 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55881400d6f0 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.085+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55881400d6f0 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.086+0000 7f15e0ff9640 1 -- 172.21.15.28:0/438970898 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f16f8045130 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.086+0000 7f15e0ff9640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15c803d000 0x7f15c803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.086+0000 7f15e0ff9640 1 -- 172.21.15.28:0/438970898 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7f16f8034cc0 con 0x558814005910 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.086+0000 7f16feeed640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15c803d000 0x7f15c803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f16feeed640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15c803d000 0x7f15c803f4c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16f00054e0 tx=0x7f16f008a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558814018650 0x55881401ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55881401b260 con 0x558814018650 2024-09-19T11:23:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f16fe6ec640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558814018650 0x55881401ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f16fe6ec640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558814018650 0x55881401ab10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16f400f610 con 0x558814018650 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f1703140800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.087+0000 7f1703140800 20 realm 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x5588140179a0 con 0x558814018650 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16f400f610 con 0x558814018650 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55881401bd50 con 0x558814018650 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16f400f610 con 0x558814018650 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55881401c620 con 0x558814018650 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f16f400f610 con 0x558814018650 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55881401d040 0x55881401f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55881401fb70 con 0x55881401d040 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.088+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55881401d040 0x55881401f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55881401d040 0x55881401f430 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f1703140800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:28.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f1703140800 20 searching for the correct realm 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f16ff6ee640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f16f8015e20 0x7f16f803c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f8015e20 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.089+0000 7f16feeed640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f16f8015e20 0x7f16f803c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.090+0000 7f16feeed640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f16f8015e20 0x7f16f803c3f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.090+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.090+0000 7f16feeed640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f16f00a5040 0x7f16f0007850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.090+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.090+0000 7f16fe6ec640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f16f00a5040 0x7f16f0007850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.090+0000 7f16fe6ec640 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f16f00a5040 0x7f16f0007850 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f400f610 con 0x7f16f00a5040 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f400f610 con 0x558814018650 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x558814018650 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x7f16f00a5040 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.091+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x7f16f00a5040 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f8015e20 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x558814018650 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.092+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x7f16f00a5040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x558814018650 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f8015e20 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.093+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x7f16f00a5040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x558814018650 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.094+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f16f402b040 con 0x558814018650 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x7f16f00a5040 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f8015e20 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.095+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f402b040 con 0x558814018650 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.096+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f8015e20 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x55881401d040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f00a5040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x558814018650 2024-09-19T11:23:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.097+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x5588140214a0 con 0x7f16f8015e20 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814020530 con 0x558814018650 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814020d80 con 0x558814018650 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814021920 con 0x55881401d040 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 20 zone default found 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.098+0000 7f1703140800 4 Realm: () 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 zonegroup default 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814021ce0 con 0x7f16f00a5040 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814023b10 con 0x55881401d040 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814023f50 con 0x55881401d040 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 started sync module instance, tier type = 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.099+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814022fb0 con 0x7f16f8015e20 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.100+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.100+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94042939601824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814022c10 con 0x7f16f8015e20 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.101+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814024e80 con 0x558814018650 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.101+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94042939601824] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.102+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.102+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94042939607504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5588140252a0 con 0x558814018650 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.102+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814025c60 con 0x55881401d040 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.103+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94042939607504] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.103+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.103+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94042939611536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5588140252a0 con 0x55881401d040 2024-09-19T11:23:28.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.103+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x5588140266d0 con 0x7f16f8015e20 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.104+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94042939611536] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.104+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.104+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94042939608560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814025210 con 0x7f16f8015e20 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.104+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814027590 con 0x558814018650 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.105+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94042939608560] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.105+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.105+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94042939615936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814027590 con 0x558814018650 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.105+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814027fe0 con 0x7f16f00a5040 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.106+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.106+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94042939620080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814027f50 con 0x7f16f00a5040 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.106+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814028f30 con 0x7f16f00a5040 2024-09-19T11:23:28.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.106+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94042939615936] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.107+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94042939620080] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.107+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.107+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94042939623968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814028340 con 0x7f16f00a5040 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.107+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814029550 con 0x7f16f00a5040 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94042939623968] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(58 notify.7 [create] v31'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94042939617504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814029550 con 0x7f16f00a5040 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 20 add_watcher() i=0 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 20 add_watcher() i=1 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 20 add_watcher() i=2 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 20 add_watcher() i=3 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 20 add_watcher() i=4 2024-09-19T11:23:28.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 20 add_watcher() i=5 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.108+0000 7f1703140800 20 add_watcher() i=6 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.109+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94042939617504] v31'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.109+0000 7f1703140800 20 add_watcher() i=7 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.109+0000 7f1703140800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.109+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55881402c7b0 con 0x558814018650 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.110+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.110+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94042939626000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814029ae0 con 0x558814018650 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.110+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94042939626000] v31'148 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f15b57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 5 note: GC not initialized 2024-09-19T11:23:28.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f152ffff640 20 reqs_thread_entry: start 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 20 Filter name: none 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f152dffb640 20 reqs_thread_entry: start 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.111+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55881405f680 con 0x558814018650 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814061fb0 con 0x7f16f8015e20 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 20 disabling bucket name=myfoo 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x55881405ff90 con 0x7f16f8015e20 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.112+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814062960 con 0x7f16f00a5040 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(65 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f1703140800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f1703140800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:28.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f1703140800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f1703140800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f1703140800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f1703140800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.113+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~387 in=387b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881406a140 con 0x7f16f00a5040 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(66 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [delete,create,call,call,writefull 0~387,setxattr (123)] v31'8 uv8 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f1703140800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f1703140800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f1703140800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f1703140800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94042939863008 in=772b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814068ea0 con 0x7f16f00a5040 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== watch-notify(notify (1) cookie 94042939620080 notify 133143986191 ret 0) ==== 802+0+0 (crc 0 0 0) 0x558814062d50 con 0x7f16f00a5040 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f15e27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986191 cookie 94042939620080 notifier 4593 bl.length()=760 2024-09-19T11:23:28.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(67 notify.5 [notify cookie 94042939863008 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f15e27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f15e27fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f15e27fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.114+0000 7f15e27fc640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f15c4003ea0 con 0x7f16f00a5040 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.115+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.115+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== watch-notify(notify_complete (2) cookie 94042939863008 notify 133143986191 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558814061ea0 con 0x7f16f00a5040 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.115+0000 7f1703140800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_sources= new_sources= 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.115+0000 7f1703140800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential sources added=[] removed=[] 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.115+0000 7f1703140800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_dests= new_dests= 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.115+0000 7f1703140800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential dests added=[] removed=[] 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.115+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558814065000 con 0x558814018650 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v31'14 uv14 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f1703140800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f1703140800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f1703140800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94042939854064 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558814061110 con 0x558814018650 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94042939615936 notify 133143986192 ret 0) ==== 609+0+0 (crc 0 0 0) 0x558814061ea0 con 0x558814018650 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f15e27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 94042939615936 notifier 4593 bl.length()=567 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f15e27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f15e27fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f15e27fc640 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7f15c4003ea0 con 0x558814018650 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.116+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(70 notify.4 [notify cookie 94042939854064 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.117+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.117+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94042939854064 notify 133143986192 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f16f4018040 con 0x558814018650 2024-09-19T11:23:28.131 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:28.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.118+0000 7f15b57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:28.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.118+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94042939626000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x558814018650 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.119+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94042939626000] v31'149 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.119+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94042939601824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x7f16f8015e20 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.120+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94042939601824] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.120+0000 7f1703140800 20 remove_watcher() i=0 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.120+0000 7f1703140800 2 removed watcher, disabling cache 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.120+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94042939607504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x558814018650 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.121+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94042939607504] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.121+0000 7f1703140800 20 remove_watcher() i=1 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.121+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94042939611536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x55881401d040 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.122+0000 7f16ff6ee640 1 -- 172.21.15.28:0/438970898 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94042939611536] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f800a4d0 con 0x55881401d040 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.122+0000 7f1703140800 20 remove_watcher() i=2 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.122+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94042939608560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x7f16f8015e20 2024-09-19T11:23:28.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.122+0000 7f16feeed640 1 -- 172.21.15.28:0/438970898 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94042939608560] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f0072920 con 0x7f16f8015e20 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.123+0000 7f1703140800 20 remove_watcher() i=3 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.123+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94042939615936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x558814018650 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.123+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94042939615936] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x558814018650 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.124+0000 7f1703140800 20 remove_watcher() i=4 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.124+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94042939620080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x7f16f00a5040 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.124+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94042939620080] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.124+0000 7f1703140800 20 remove_watcher() i=5 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.124+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94042939623968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x7f16f00a5040 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.125+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94042939623968] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.125+0000 7f1703140800 20 remove_watcher() i=6 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.126+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94042939617504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x55881405d520 con 0x7f16f00a5040 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94042939617504] v31'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16f403b040 con 0x7f16f00a5040 2024-09-19T11:23:28.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 20 remove_watcher() i=7 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f16f00a5040 msgr2=0x7f16f0007850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f16f00a5040 0x7f16f0007850 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558814018650 msgr2=0x55881401ab10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558814018650 0x55881401ab10 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f16f8015e20 msgr2=0x7f16f803c3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f16f8015e20 0x7f16f803c3f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55881401d040 msgr2=0x55881401f430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55881401d040 0x55881401f430 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15c803d000 msgr2=0x7f15c803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15c803d000 0x7f15c803f4c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f16f00054e0 tx=0x7f16f008a000 comp rx=0 tx=0).stop 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 msgr2=0x55881400cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x55881400cf00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f16f80204e0 tx=0x7f16f8054790 comp rx=0 tx=0).stop 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f16fe6ec640 1 -- 172.21.15.28:0/438970898 reap_dead start 2024-09-19T11:23:28.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 shutdown_connections 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/438970898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558814005910 0x55881400cf00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 >> 172.21.15.28:0/438970898 conn(0x558814001d30 msgr2=0x558813ffff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 shutdown_connections 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/438970898 wait complete. 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15d803d000 msgr2=0x7f15d803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15d803d000 0x7f15d803f4c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16f00077e0 tx=0x7f16f0096cf0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ff8eb0 msgr2=0x558813ff92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ff8eb0 0x558813ff92d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x558813fdef30 tx=0x7f16f80167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 shutdown_connections 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15d803d000 0x7f15d803f4c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 --2- 172.21.15.28:0/1571531299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ff8eb0 0x558813ff92d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.127+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 >> 172.21.15.28:0/1571531299 conn(0x558813fed260 msgr2=0x558813fed640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 shutdown_connections 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/1571531299 wait complete. 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558813ee48d0 msgr2=0x558813fe5580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558813ee48d0 0x558813fe5580 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558813fe7920 msgr2=0x558813fe9da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558813fe7920 0x558813fe9da0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15e403d000 msgr2=0x7f15e403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15e403d000 0x7f15e403f4c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f16f4001e30 tx=0x7f16f4001390 comp rx=0 tx=0).stop 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 msgr2=0x558813fddf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fddf30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000c00 tx=0x7f16f0002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 shutdown_connections 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558813fe7920 0x558813fe9da0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f15e403d000 0x7f15e403f4c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558813ee48d0 0x558813fe5580 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 --2- 172.21.15.28:0/3333339461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558813ee3690 0x558813fddf30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 >> 172.21.15.28:0/3333339461 conn(0x558813edea60 msgr2=0x558813fd34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 shutdown_connections 2024-09-19T11:23:28.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.128+0000 7f1703140800 1 -- 172.21.15.28:0/3333339461 wait complete. 2024-09-19T11:23:28.142 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:28.144 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-09-19T11:23:28.144 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-09-19T11:23:28.144 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-09-19T11:23:28.189 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:28.190 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7f319800 1 Processor -- start 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7f319800 1 -- start start 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7f319800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558a75458ea0 0x558a7545b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7f319800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558a7545f5e0 con 0x558a7545b8c0 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7f319800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x558a752e37d0 con 0x558a75458ea0 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7ce0a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558a75458ea0 0x558a7545b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7c609640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x558a7545b8c0 0x558a7545dd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48422/0 (socket says 172.21.15.28:48422) 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7ce0a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558a75458ea0 0x558a7545b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34942/0 (socket says 172.21.15.28:34942) 2024-09-19T11:23:28.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.189+0000 7fbb7c609640 1 -- 172.21.15.28:0/4083191624 learned_addr learned my addr 172.21.15.28:0/4083191624 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:28.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.190+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/4083191624 >> v1:172.21.15.22:6789/0 conn(0x558a7545b8c0 legacy=0x558a7545dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.190+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/4083191624 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a75460810 con 0x558a75458ea0 2024-09-19T11:23:28.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.191+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/4083191624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7545b290 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c005fd0 tx=0x7fbb6c045c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bc545d959d8487 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7be08640 1 -- 172.21.15.28:0/4083191624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c058530 con 0x558a75458ea0 2024-09-19T11:23:28.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7be08640 1 -- 172.21.15.28:0/4083191624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6c0586d0 con 0x558a75458ea0 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7be08640 1 -- 172.21.15.28:0/4083191624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c0589d0 con 0x558a75458ea0 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7f319800 1 -- 172.21.15.28:0/4083191624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 msgr2=0x558a7545b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7f319800 1 --2- 172.21.15.28:0/4083191624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7545b290 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c005fd0 tx=0x7fbb6c045c30 comp rx=0 tx=0).stop 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7f319800 1 -- 172.21.15.28:0/4083191624 shutdown_connections 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7f319800 1 --2- 172.21.15.28:0/4083191624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7545b290 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7f319800 1 -- 172.21.15.28:0/4083191624 >> 172.21.15.28:0/4083191624 conn(0x558a75452a60 msgr2=0x558a75454ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7f319800 1 -- 172.21.15.28:0/4083191624 shutdown_connections 2024-09-19T11:23:28.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.192+0000 7fbb7f319800 1 -- 172.21.15.28:0/4083191624 wait complete. 2024-09-19T11:23:28.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.207+0000 7fbb7f319800 1 Processor -- start 2024-09-19T11:23:28.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.207+0000 7fbb7f319800 1 -- start start 2024-09-19T11:23:28.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.207+0000 7fbb7f319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a755483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.207+0000 7fbb7f319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a752f0dd0 con 0x558a75458ea0 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fbb7ce0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a755483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fbb7ce0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a755483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34946/0 (socket says 172.21.15.28:34946) 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/1222599179 learned_addr learned my addr 172.21.15.28:0/1222599179 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/1222599179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a75548990 con 0x558a75458ea0 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/1222599179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a755483a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00a3f0 tx=0x7fbb6c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed64e783c1a6fc27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fba6ae0d640 1 -- 172.21.15.28:0/1222599179 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c0589b0 con 0x558a75458ea0 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fba6ae0d640 1 -- 172.21.15.28:0/1222599179 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6c068070 con 0x558a75458ea0 2024-09-19T11:23:28.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.208+0000 7fba6ae0d640 1 -- 172.21.15.28:0/1222599179 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c058c80 con 0x558a75458ea0 2024-09-19T11:23:28.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 -- 172.21.15.28:0/1222599179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 msgr2=0x558a755483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1222599179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a755483a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c00a3f0 tx=0x7fbb6c051000 comp rx=0 tx=0).stop 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 -- 172.21.15.28:0/1222599179 shutdown_connections 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1222599179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a755483a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 -- 172.21.15.28:0/1222599179 >> 172.21.15.28:0/1222599179 conn(0x558a75452a60 msgr2=0x558a75543d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 -- 172.21.15.28:0/1222599179 shutdown_connections 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 -- 172.21.15.28:0/1222599179 wait complete. 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.209+0000 7fbb7f319800 1 Processor -- start 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7f319800 1 -- start start 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7f319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7554fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7f319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a7545fab0 con 0x558a75458ea0 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7ce0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7554fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7ce0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7554fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34960/0 (socket says 172.21.15.28:34960) 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/1287362439 learned_addr learned my addr 172.21.15.28:0/1287362439 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a75550290 con 0x558a75458ea0 2024-09-19T11:23:28.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.210+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7554fd30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c065040 tx=0x7fbb6c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.211+0000 7fba6960a640 1 -- 172.21.15.28:0/1287362439 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c063570 con 0x558a75458ea0 2024-09-19T11:23:28.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.211+0000 7fba6960a640 1 -- 172.21.15.28:0/1287362439 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6c063710 con 0x558a75458ea0 2024-09-19T11:23:28.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.211+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a75550520 con 0x558a75458ea0 2024-09-19T11:23:28.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.211+0000 7fba6960a640 1 -- 172.21.15.28:0/1287362439 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb6c063a10 con 0x558a75458ea0 2024-09-19T11:23:28.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.211+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a75550f20 con 0x558a75458ea0 2024-09-19T11:23:28.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.211+0000 7fba6960a640 1 -- 172.21.15.28:0/1287362439 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbb6c075020 con 0x558a75458ea0 2024-09-19T11:23:28.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fba6960a640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba6403d000 0x7fba6403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fba6960a640 1 -- 172.21.15.28:0/1287362439 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fbb6c098810 con 0x558a75458ea0 2024-09-19T11:23:28.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fbb7c609640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba6403d000 0x7fba6403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7545b8c0 0x558a75557380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75551ac0 con 0x558a7545b8c0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7545b8c0 0x558a75557380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fbb7c609640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba6403d000 0x7fba6403f4c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb64001d70 tx=0x7fbb640012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.212+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7545b8c0 0x558a75557380 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.213+0000 7fbb7d60b640 1 -- 172.21.15.28:0/1287362439 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb740083e0 con 0x558a7545b8c0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.213+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75557dc0 con 0x558a7545b8c0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.213+0000 7fbb7d60b640 1 -- 172.21.15.28:0/1287362439 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbb740083e0 con 0x558a7545b8c0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.213+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a755596d0 0x558a7555bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.213+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a7555c300 con 0x558a755596d0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.213+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a755596d0 0x558a7555bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.213+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a755596d0 0x558a7555bb70 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.214+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/1287362439 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbb6c077930 con 0x558a755596d0 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.214+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75558610 con 0x558a7545b8c0 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.214+0000 7fbb7d60b640 1 -- 172.21.15.28:0/1287362439 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbb740083e0 con 0x558a7545b8c0 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.214+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755589b0 con 0x558a755596d0 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.214+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/1287362439 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbb6c077930 con 0x558a755596d0 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7f319800 1 Processor -- start 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7f319800 1 -- start start 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7f319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a7556ac60 0x558a7556b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7f319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a7534f4a0 con 0x558a7556ac60 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7d60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a7556ac60 0x558a7556b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7d60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a7556ac60 0x558a7556b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34962/0 (socket says 172.21.15.28:34962) 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7d60b640 1 -- 172.21.15.28:0/233405157 learned_addr learned my addr 172.21.15.28:0/233405157 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.215+0000 7fbb7d60b640 1 -- 172.21.15.28:0/233405157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7556b6c0 con 0x558a7556ac60 2024-09-19T11:23:28.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.216+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a7556ac60 0x558a7556b080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x558a75551320 tx=0x7fbb7400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.216+0000 7fba627fc640 1 -- 172.21.15.28:0/233405157 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb7401c020 con 0x558a7556ac60 2024-09-19T11:23:28.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.216+0000 7fba627fc640 1 -- 172.21.15.28:0/233405157 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb74007c90 con 0x558a7556ac60 2024-09-19T11:23:28.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.216+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a7556bef0 con 0x558a7556ac60 2024-09-19T11:23:28.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.216+0000 7fba627fc640 1 -- 172.21.15.28:0/233405157 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb74018440 con 0x558a7556ac60 2024-09-19T11:23:28.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.216+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7556c490 con 0x558a7556ac60 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fba627fc640 1 -- 172.21.15.28:0/233405157 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbb7401f020 con 0x558a7556ac60 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fba627fc640 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5403cfb0 0x7fba5403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fba627fc640 1 -- 172.21.15.28:0/233405157 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fbb74055a00 con 0x558a7556ac60 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5403cfb0 0x7fba5403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fbb7f319800 1 Processor -- start 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5403cfb0 0x7fba5403f470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c096c20 tx=0x7fbb6c04ff70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fbb7f319800 1 -- start start 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fbb7f319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75458350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.217+0000 7fbb7f319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a75548d00 con 0x558a75457f50 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fbb7d60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75458350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fbb7d60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75458350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34978/0 (socket says 172.21.15.28:34978) 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fbb7d60b640 1 -- 172.21.15.28:0/646208100 learned_addr learned my addr 172.21.15.28:0/646208100 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fbb7d60b640 1 -- 172.21.15.28:0/646208100 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a75458890 con 0x558a75457f50 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/646208100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75458350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbb74053e10 tx=0x7fbb740157b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e987a0b4e874d500 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fba53fff640 1 -- 172.21.15.28:0/646208100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb740174d0 con 0x558a75457f50 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fba53fff640 1 -- 172.21.15.28:0/646208100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb74017670 con 0x558a75457f50 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.218+0000 7fba53fff640 1 -- 172.21.15.28:0/646208100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb7400b160 con 0x558a75457f50 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 -- 172.21.15.28:0/646208100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 msgr2=0x558a75458350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 --2- 172.21.15.28:0/646208100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75458350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbb74053e10 tx=0x7fbb740157b0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 -- 172.21.15.28:0/646208100 shutdown_connections 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 --2- 172.21.15.28:0/646208100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75458350 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 -- 172.21.15.28:0/646208100 >> 172.21.15.28:0/646208100 conn(0x558a75573ac0 msgr2=0x558a75571d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 -- 172.21.15.28:0/646208100 shutdown_connections 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 -- 172.21.15.28:0/646208100 wait complete. 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 Processor -- start 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 -- start start 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75580dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7f319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a75456e10 con 0x558a75457f50 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7d60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75580dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7d60b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75580dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:34984/0 (socket says 172.21.15.28:34984) 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.219+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 learned_addr learned my addr 172.21.15.28:0/72506287 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.220+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a75581330 con 0x558a75457f50 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.220+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75580dd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbb74015a80 tx=0x7fbb74053da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.220+0000 7fba527fc640 1 -- 172.21.15.28:0/72506287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb74013b10 con 0x558a75457f50 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.220+0000 7fba527fc640 1 -- 172.21.15.28:0/72506287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb74013cb0 con 0x558a75457f50 2024-09-19T11:23:28.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.220+0000 7fba527fc640 1 -- 172.21.15.28:0/72506287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb740208c0 con 0x558a75457f50 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.220+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a755815c0 con 0x558a75457f50 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.220+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a75582150 con 0x558a75457f50 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.221+0000 7fba527fc640 1 -- 172.21.15.28:0/72506287 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbb74016040 con 0x558a75457f50 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.221+0000 7fba527fc640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4803d000 0x7fba4803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.221+0000 7fba527fc640 1 -- 172.21.15.28:0/72506287 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6110+0+0 (secure 0 0 0) 0x7fbb74036c70 con 0x558a75457f50 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.221+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4803d000 0x7fba4803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.221+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4803d000 0x7fba4803f4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x558a75551500 tx=0x7fbb6c093000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7558c750 0x558a7558ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a7558f360 con 0x558a7558c750 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7c609640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7558c750 0x558a7558ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7c609640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7558c750 0x558a7558ec10 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb64009050 con 0x558a7558c750 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7f319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7f319800 20 realm 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.222+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a7558baa0 con 0x558a7558c750 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb64009050 con 0x558a7558c750 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a7558fe50 con 0x558a7558c750 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb64009050 con 0x558a7558c750 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75590720 con 0x558a7558c750 2024-09-19T11:23:28.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbb64009050 con 0x558a7558c750 2024-09-19T11:23:28.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:28.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a75591140 0x558a75593530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75593c70 con 0x558a75591140 2024-09-19T11:23:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.223+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a75591140 0x558a75593530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a75591140 0x558a75593530 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7f319800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7f319800 20 searching for the correct realm 2024-09-19T11:23:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7d60b640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb74021020 0x7fbb74020a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb74021020 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.224+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb74021020 0x7fbb74020a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.225+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb74021020 0x7fbb74020a60 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.225+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.225+0000 7fbb7ce0a640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb6c0a5040 0x7fbb6c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.225+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.225+0000 7fbb7c609640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb6c0a5040 0x7fbb6c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.225+0000 7fbb7c609640 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb6c0a5040 0x7fbb6c06c780 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64009050 con 0x7fbb6c0a5040 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64009050 con 0x558a7558c750 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.226+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb74021020 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.227+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb74021020 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 31 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.228+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 31 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(21 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.229+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 31 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(26 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.230+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb74021020 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(28 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(30 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.231+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.232+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(32 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.232+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb74021020 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.232+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(33 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.232+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a75591140 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.232+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.232+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(35 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x558a7558c750 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 31 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 31 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e31) -- 0x558a755955a0 con 0x7fbb74021020 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(37 [pgnls start_epoch 31 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75594630 con 0x558a7558c750 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75594e80 con 0x558a7558c750 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.233+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75595a20 con 0x558a75591140 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 zone default found 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 4 Realm: () 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:28.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 zonegroup default 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75595de0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755980c0 con 0x558a75591140 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.234+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a75598460 con 0x558a75591140 2024-09-19T11:23:28.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.235+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.235+0000 7fbb7f319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.235+0000 7fbb7f319800 20 started sync module instance, tier type = 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.235+0000 7fbb7f319800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.235+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a75597500 con 0x7fbb74021020 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.236+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(44 notify.0 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.236+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94053162643536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a75598870 con 0x7fbb74021020 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.236+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755990c0 con 0x558a7558c750 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.237+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94053162643536] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.237+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.237+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94053162650640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755994e0 con 0x558a7558c750 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.237+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a75599d30 con 0x558a75591140 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.238+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94053162650640] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.239+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.239+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94053162652176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a75599c60 con 0x558a75591140 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.239+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559a7b0 con 0x7fbb74021020 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.240+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(50 notify.3 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.240+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94053162656448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559b080 con 0x7fbb74021020 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.240+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559bba0 con 0x558a7558c750 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.240+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94053162652176] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.241+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.241+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94053162661488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559b080 con 0x558a7558c750 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.241+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559c610 con 0x7fbb6c0a5040 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.241+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94053162656448] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.242+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(54 notify.5 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.242+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94053162664192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559b080 con 0x7fbb6c0a5040 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.242+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559d050 con 0x7fbb6c0a5040 2024-09-19T11:23:28.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.242+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94053162661488] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.243+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94053162664192] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.243+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(56 notify.6 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.243+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94053162666912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559b080 con 0x7fbb6c0a5040 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.243+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559cfc0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(58 notify.7 [create] v31'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94053162667968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559d930 con 0x7fbb6c0a5040 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 20 add_watcher() i=0 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 20 add_watcher() i=1 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 20 add_watcher() i=2 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 20 add_watcher() i=3 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 20 add_watcher() i=4 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 20 add_watcher() i=5 2024-09-19T11:23:28.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94053162666912] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.244+0000 7fbb7f319800 20 add_watcher() i=6 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.245+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94053162667968] v31'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.245+0000 7fbb7f319800 20 add_watcher() i=7 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.245+0000 7fbb7f319800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.245+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a7559abd0 con 0x558a7558c750 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.245+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.245+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94053162669120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a7559abd0 con 0x558a7558c750 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.246+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94053162669120] v31'150 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.246+0000 7fba337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.246+0000 7fbb7f319800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.246+0000 7fbb7f319800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.246+0000 7fbb7f319800 5 note: GC not initialized 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fba327e4640 20 reqs_thread_entry: start 2024-09-19T11:23:28.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 20 Filter name: none 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fba2b7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755d3810 con 0x558a7558c750 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv14 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.247+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755d3270 con 0x7fbb74021020 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 20 enabling bucket name=myfoo 2024-09-19T11:23:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755d3270 con 0x7fbb74021020 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 : miss 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.248+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755d5950 con 0x7fbb6c0a5040 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(65 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv8 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7f319800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7f319800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:28.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7f319800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:28.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7f319800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:28.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7f319800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:28.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7f319800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 2024-09-19T11:23:28.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.249+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 8.e 8:770a8583:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.1:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~387 in=387b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755de270 con 0x7fbb6c0a5040 2024-09-19T11:23:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(66 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 [delete,create,call,call,writefull 0~387,setxattr (123)] v31'9 uv9 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7f319800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7f319800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7f319800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7f319800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1, ofs0, ns] 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 94053162933072 in=772b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755d8780 con 0x7fbb6c0a5040 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== watch-notify(notify (1) cookie 94053162664192 notify 133143986192 ret 0) ==== 802+0+0 (crc 0 0 0) 0x558a755d5d40 con 0x7fbb6c0a5040 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fba53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986192 cookie 94053162664192 notifier 4600 bl.length()=760 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fba53fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 info.flags=0x17 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fba53fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1 to cache LRU end 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fba53fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fba53fff640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fba40003ea0 con 0x7fbb6c0a5040 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.250+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(67 notify.5 [notify cookie 94053162933072 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.251+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.251+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== watch-notify(notify_complete (2) cookie 94053162933072 notify 133143986192 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558a755d3e10 con 0x7fbb6c0a5040 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.251+0000 7fbb7f319800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_sources= new_sources= 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.251+0000 7fbb7f319800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential sources added=[] removed=[] 2024-09-19T11:23:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.251+0000 7fbb7f319800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): orig_dests= new_dests= 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.251+0000 7fbb7f319800 20 handle_bi_update(): bucket=:myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.1]): potential dests added=[] removed=[] 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.251+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755ddb50 con 0x558a7558c750 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v31'15 uv15 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7f319800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7f319800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7f319800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94053162933072 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x558a755d76e0 con 0x558a7558c750 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94053162661488 notify 133143986193 ret 0) ==== 609+0+0 (crc 0 0 0) 0x558a755d3e10 con 0x558a7558c750 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fba537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 133143986193 cookie 94053162661488 notifier 4600 bl.length()=567 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fba537fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fba537fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fba537fe640 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e31) -- 0x7fb9a4003430 con 0x558a7558c750 2024-09-19T11:23:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(70 notify.4 [notify cookie 94053162933072 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.252+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94053162933072 notify 133143986193 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558a7545f5e0 con 0x558a7558c750 2024-09-19T11:23:28.268 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.253+0000 7fba337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.254+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94053162669120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x558a7558c750 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.254+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94053162669120] v31'151 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.254+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94053162643536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x7fbb74021020 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.255+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94053162643536] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.255+0000 7fbb7f319800 20 remove_watcher() i=0 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.255+0000 7fbb7f319800 2 removed watcher, disabling cache 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.255+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94053162650640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x558a7558c750 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.256+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94053162650640] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.256+0000 7fbb7f319800 20 remove_watcher() i=1 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.256+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94053162652176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x558a75591140 2024-09-19T11:23:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.257+0000 7fbb7d60b640 1 -- 172.21.15.28:0/72506287 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94053162652176] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb7403c070 con 0x558a75591140 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.257+0000 7fbb7f319800 20 remove_watcher() i=2 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.257+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94053162656448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x7fbb74021020 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.259+0000 7fbb7ce0a640 1 -- 172.21.15.28:0/72506287 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94053162656448] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6c093070 con 0x7fbb74021020 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.259+0000 7fbb7f319800 20 remove_watcher() i=3 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.259+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94053162661488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x558a7558c750 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.260+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94053162661488] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x558a7558c750 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.260+0000 7fbb7f319800 20 remove_watcher() i=4 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.260+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94053162664192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x7fbb6c0a5040 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.261+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94053162664192] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.261+0000 7fbb7f319800 20 remove_watcher() i=5 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.261+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94053162666912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x7fbb6c0a5040 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.262+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94053162666912] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.262+0000 7fbb7f319800 20 remove_watcher() i=6 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.262+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94053162667968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31) -- 0x558a755d1670 con 0x7fbb6c0a5040 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94053162667968] v31'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb64029040 con 0x7fbb6c0a5040 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 20 remove_watcher() i=7 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb6c0a5040 msgr2=0x7fbb6c06c780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb6c0a5040 0x7fbb6c06c780 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7558c750 msgr2=0x558a7558ec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7558c750 0x558a7558ec10 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb74021020 msgr2=0x7fbb74020a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb74021020 0x7fbb74020a60 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a75591140 msgr2=0x558a75593530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a75591140 0x558a75593530 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4803d000 msgr2=0x7fba4803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4803d000 0x7fba4803f4c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x558a75551500 tx=0x7fbb6c093000 comp rx=0 tx=0).stop 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 msgr2=0x558a75580dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/72506287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75457f50 0x558a75580dd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbb74015a80 tx=0x7fbb74053da0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 reap_dead start 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7c609640 1 -- 172.21.15.28:0/72506287 reap_dead start 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 shutdown_connections 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 >> 172.21.15.28:0/72506287 conn(0x558a75573ac0 msgr2=0x558a75571cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 shutdown_connections 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/72506287 wait complete. 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5403cfb0 msgr2=0x7fba5403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5403cfb0 0x7fba5403f470 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c096c20 tx=0x7fbb6c04ff70 comp rx=0 tx=0).stop 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a7556ac60 msgr2=0x558a7556b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.263+0000 7fbb7f319800 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a7556ac60 0x558a7556b080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x558a75551320 tx=0x7fbb7400bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 shutdown_connections 2024-09-19T11:23:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5403cfb0 0x7fba5403f470 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/233405157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a7556ac60 0x558a7556b080 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 >> 172.21.15.28:0/233405157 conn(0x558a7555f010 msgr2=0x558a7555f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 shutdown_connections 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/233405157 wait complete. 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7545b8c0 msgr2=0x558a75557380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7545b8c0 0x558a75557380 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a755596d0 msgr2=0x558a7555bb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a755596d0 0x558a7555bb70 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba6403d000 msgr2=0x7fba6403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba6403d000 0x7fba6403f4c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb64001d70 tx=0x7fbb640012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 msgr2=0x558a7554fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7554fd30 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c065040 tx=0x7fbb6c003330 comp rx=0 tx=0).stop 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 shutdown_connections 2024-09-19T11:23:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a755596d0 0x558a7555bb70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba6403d000 0x7fba6403f4c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a7545b8c0 0x558a75557380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 --2- 172.21.15.28:0/1287362439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a75458ea0 0x558a7554fd30 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 >> 172.21.15.28:0/1287362439 conn(0x558a75452a60 msgr2=0x558a75549790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 shutdown_connections 2024-09-19T11:23:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:28.264+0000 7fbb7f319800 1 -- 172.21.15.28:0/1287362439 wait complete. 2024-09-19T11:23:28.279 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:28.294 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-19T11:23:28.526 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-09-19T11:23:28.533 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:43.533 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-09-19T11:23:43.533 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-09-19T11:23:43.533 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-09-19T11:23:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:43.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.586+0000 7f0a5d319800 1 Processor -- start 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.586+0000 7f0a5d319800 1 -- start start 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.586+0000 7f0a5d319800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563971608d70 0x56397160e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.586+0000 7f0a5d319800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5639716080b0 con 0x563971607a00 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.586+0000 7f0a5d319800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5639714907d0 con 0x563971608d70 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.587+0000 7f0a5a602640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563971608d70 0x56397160e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.587+0000 7f0a5ae03640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x563971607a00 0x563971608660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55546/0 (socket says 172.21.15.28:55546) 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.587+0000 7f0a5a602640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563971608d70 0x56397160e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53098/0 (socket says 172.21.15.28:53098) 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.587+0000 7f0a5ae03640 1 -- 172.21.15.28:0/2602810420 learned_addr learned my addr 172.21.15.28:0/2602810420 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.587+0000 7f0a5a602640 1 -- 172.21.15.28:0/2602810420 >> v1:172.21.15.22:6789/0 conn(0x563971607a00 legacy=0x563971608660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.587+0000 7f0a59e01640 1 -- 172.21.15.28:0/2602810420 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1328546431 0 0) 0x5639716080b0 con 0x563971607a00 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.587+0000 7f0a5a602640 1 -- 172.21.15.28:0/2602810420 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563971609220 con 0x563971608d70 2024-09-19T11:23:43.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.589+0000 7f0a5a602640 1 --2- 172.21.15.28:0/2602810420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x56397160e910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400a470 tx=0x7f0a44045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691e3ad7dadafcdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.590+0000 7f0a59e01640 1 -- 172.21.15.28:0/2602810420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a44053570 con 0x563971608d70 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.590+0000 7f0a59e01640 1 -- 172.21.15.28:0/2602810420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a44058070 con 0x563971608d70 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.590+0000 7f0a59e01640 1 -- 172.21.15.28:0/2602810420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a44053880 con 0x563971608d70 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.591+0000 7f0a5d319800 1 -- 172.21.15.28:0/2602810420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 msgr2=0x56397160e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.591+0000 7f0a5d319800 1 --2- 172.21.15.28:0/2602810420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x56397160e910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400a470 tx=0x7f0a44045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.591+0000 7f0a5d319800 1 -- 172.21.15.28:0/2602810420 shutdown_connections 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.591+0000 7f0a5d319800 1 --2- 172.21.15.28:0/2602810420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x56397160e910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.591+0000 7f0a5d319800 1 -- 172.21.15.28:0/2602810420 >> 172.21.15.28:0/2602810420 conn(0x56397157ad70 msgr2=0x56397157b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.591+0000 7f0a5d319800 1 -- 172.21.15.28:0/2602810420 shutdown_connections 2024-09-19T11:23:43.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.591+0000 7f0a5d319800 1 -- 172.21.15.28:0/2602810420 wait complete. 2024-09-19T11:23:43.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.608+0000 7f0a5d319800 1 Processor -- start 2024-09-19T11:23:43.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.608+0000 7f0a5d319800 1 -- start start 2024-09-19T11:23:43.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.608+0000 7f0a5d319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971607a00 0x5639716f7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.608+0000 7f0a5d319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56397149ddd0 con 0x563971607a00 2024-09-19T11:23:43.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.609+0000 7f0a5ae03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971607a00 0x5639716f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.609+0000 7f0a5ae03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971607a00 0x5639716f7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53112/0 (socket says 172.21.15.28:53112) 2024-09-19T11:23:43.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.609+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3757825362 learned_addr learned my addr 172.21.15.28:0/3757825362 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.609+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3757825362 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639716f7920 con 0x563971607a00 2024-09-19T11:23:43.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.609+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/3757825362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971607a00 0x5639716f7330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a54001ea0 tx=0x7f0a5400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692c509a140fcea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0948e09640 1 -- 172.21.15.28:0/3757825362 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a54008070 con 0x563971607a00 2024-09-19T11:23:43.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0948e09640 1 -- 172.21.15.28:0/3757825362 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5400d500 con 0x563971607a00 2024-09-19T11:23:43.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0948e09640 1 -- 172.21.15.28:0/3757825362 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a5400d880 con 0x563971607a00 2024-09-19T11:23:43.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0a5d319800 1 -- 172.21.15.28:0/3757825362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971607a00 msgr2=0x5639716f7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3757825362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971607a00 0x5639716f7330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0a54001ea0 tx=0x7f0a5400b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0a5d319800 1 -- 172.21.15.28:0/3757825362 shutdown_connections 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3757825362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971607a00 0x5639716f7330 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.610+0000 7f0a5d319800 1 -- 172.21.15.28:0/3757825362 >> 172.21.15.28:0/3757825362 conn(0x56397157ad70 msgr2=0x5639716f2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5d319800 1 -- 172.21.15.28:0/3757825362 shutdown_connections 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5d319800 1 -- 172.21.15.28:0/3757825362 wait complete. 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5d319800 1 Processor -- start 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5d319800 1 -- start start 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5d319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x5639716ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5d319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56397160f060 con 0x563971608d70 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5ae03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x5639716ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5ae03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x5639716ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53116/0 (socket says 172.21.15.28:53116) 2024-09-19T11:23:43.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.611+0000 7f0a5ae03640 1 -- 172.21.15.28:0/155902037 learned_addr learned my addr 172.21.15.28:0/155902037 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.612+0000 7f0a5ae03640 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639716ff620 con 0x563971608d70 2024-09-19T11:23:43.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.612+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x5639716ff0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0a540012f0 tx=0x7f0a54000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.612+0000 7f0947606640 1 -- 172.21.15.28:0/155902037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a5400de50 con 0x563971608d70 2024-09-19T11:23:43.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.612+0000 7f0947606640 1 -- 172.21.15.28:0/155902037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5400f930 con 0x563971608d70 2024-09-19T11:23:43.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.612+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639716ff8b0 con 0x563971608d70 2024-09-19T11:23:43.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.612+0000 7f0947606640 1 -- 172.21.15.28:0/155902037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a5400fc30 con 0x563971608d70 2024-09-19T11:23:43.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.612+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639717002b0 con 0x563971608d70 2024-09-19T11:23:43.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.613+0000 7f0947606640 1 -- 172.21.15.28:0/155902037 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0a54017040 con 0x563971608d70 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0947606640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f094003d1f0 0x7f094003f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0947606640 1 -- 172.21.15.28:0/155902037 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f0a54015020 con 0x563971608d70 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0a5a602640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f094003d1f0 0x7f094003f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971700dd0 0x563971706710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971706ca0 con 0x563971700dd0 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0a5b604640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971700dd0 0x563971706710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0a5a602640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f094003d1f0 0x7f094003f6b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a44057000 tx=0x7f0a44052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.614+0000 7f0a5b604640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971700dd0 0x563971706710 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.615+0000 7f0a5b604640 1 -- 172.21.15.28:0/155902037 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x563971700dd0 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.615+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971700760 con 0x563971700dd0 2024-09-19T11:23:43.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.615+0000 7f0a5b604640 1 -- 172.21.15.28:0/155902037 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x563971700dd0 2024-09-19T11:23:43.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.615+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563971708fd0 0x56397170b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.615+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56397170ba90 con 0x563971708fd0 2024-09-19T11:23:43.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.615+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563971708fd0 0x56397170b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.616+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563971708fd0 0x56397170b3c0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.616+0000 7f0a5ae03640 1 -- 172.21.15.28:0/155902037 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0a5402d070 con 0x563971708fd0 2024-09-19T11:23:43.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.616+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971707e70 con 0x563971700dd0 2024-09-19T11:23:43.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.616+0000 7f0a5b604640 1 -- 172.21.15.28:0/155902037 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x563971700dd0 2024-09-19T11:23:43.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.616+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5639717082d0 con 0x563971708fd0 2024-09-19T11:23:43.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.617+0000 7f0a5ae03640 1 -- 172.21.15.28:0/155902037 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0a5402d070 con 0x563971708fd0 2024-09-19T11:23:43.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.617+0000 7f0a5d319800 1 Processor -- start 2024-09-19T11:23:43.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.617+0000 7f0a5d319800 1 -- start start 2024-09-19T11:23:43.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.617+0000 7f0a5d319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56397171a3b0 0x56397171a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.617+0000 7f0a5d319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5639714fc4a0 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0a5b604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56397171a3b0 0x56397171a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0a5b604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56397171a3b0 0x56397171a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53122/0 (socket says 172.21.15.28:53122) 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0a5b604640 1 -- 172.21.15.28:0/734855529 learned_addr learned my addr 172.21.15.28:0/734855529 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0a5b604640 1 -- 172.21.15.28:0/734855529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56397171ae10 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0a5b604640 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56397171a3b0 0x56397171a7d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5639716fff50 tx=0x7f0a4c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0944e01640 1 -- 172.21.15.28:0/734855529 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a4c013640 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0944e01640 1 -- 172.21.15.28:0/734855529 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c0137e0 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56397171b640 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.618+0000 7f0944e01640 1 -- 172.21.15.28:0/734855529 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a4c013ac0 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.619+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397171bbe0 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.619+0000 7f0944e01640 1 -- 172.21.15.28:0/734855529 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0a4c011040 con 0x56397171a3b0 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0944e01640 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f093403d000 0x7f093403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0944e01640 1 -- 172.21.15.28:0/734855529 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f0a4c00e040 con 0x56397171a3b0 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f093403d000 0x7f093403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0a5d319800 1 Processor -- start 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0a5d319800 1 -- start start 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f093403d000 0x7f093403f4c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x563971707150 tx=0x7f0a54052f30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0a5d319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x563971727270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.620+0000 7f0a5d319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5639716f7d50 con 0x563971726e70 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f0a5b604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x563971727270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f0a5b604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x563971727270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53124/0 (socket says 172.21.15.28:53124) 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f0a5b604640 1 -- 172.21.15.28:0/2101999886 learned_addr learned my addr 172.21.15.28:0/2101999886 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f0a5b604640 1 -- 172.21.15.28:0/2101999886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563971727860 con 0x563971726e70 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f0a5b604640 1 --2- 172.21.15.28:0/2101999886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x563971727270 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c020000 tx=0x7f0a4c0204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a05df5de700fdb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f093e7fc640 1 -- 172.21.15.28:0/2101999886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a4c021440 con 0x563971726e70 2024-09-19T11:23:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f093e7fc640 1 -- 172.21.15.28:0/2101999886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c052070 con 0x563971726e70 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.621+0000 7f093e7fc640 1 -- 172.21.15.28:0/2101999886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a4c0215e0 con 0x563971726e70 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 -- 172.21.15.28:0/2101999886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 msgr2=0x563971727270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 --2- 172.21.15.28:0/2101999886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x563971727270 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c020000 tx=0x7f0a4c0204d0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 -- 172.21.15.28:0/2101999886 shutdown_connections 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 --2- 172.21.15.28:0/2101999886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x563971727270 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 -- 172.21.15.28:0/2101999886 >> 172.21.15.28:0/2101999886 conn(0x563971723270 msgr2=0x5639717214b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 -- 172.21.15.28:0/2101999886 shutdown_connections 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 -- 172.21.15.28:0/2101999886 wait complete. 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 Processor -- start 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 -- start start 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x56397172e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.622+0000 7f0a5d319800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563971727e80 con 0x563971726e70 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f0a5b604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x56397172e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f0a5b604640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x56397172e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53134/0 (socket says 172.21.15.28:53134) 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 learned_addr learned my addr 172.21.15.28:0/3444010742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56397172e990 con 0x563971726e70 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f0a5b604640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x56397172e430 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0547e0 tx=0x7f0a4c054840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f093cff9640 1 -- 172.21.15.28:0/3444010742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a4c01fca0 con 0x563971726e70 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f093cff9640 1 -- 172.21.15.28:0/3444010742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c0560d0 con 0x563971726e70 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56397172ec20 con 0x563971726e70 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f093cff9640 1 -- 172.21.15.28:0/3444010742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a4c0563d0 con 0x563971726e70 2024-09-19T11:23:43.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.623+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397172f7b0 con 0x563971726e70 2024-09-19T11:23:43.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.624+0000 7f093cff9640 1 -- 172.21.15.28:0/3444010742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0a4c01f500 con 0x563971726e70 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f093cff9640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092403cfb0 0x7f092403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f093cff9640 1 -- 172.21.15.28:0/3444010742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f0a4c004600 con 0x563971726e70 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092403cfb0 0x7f092403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971739c80 0x56397173c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56397173c890 con 0x563971739c80 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f0a5a602640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971739c80 0x56397173c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.625+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092403cfb0 0x7f092403f470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c010 tx=0x7f0a5404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5a602640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971739c80 0x56397173c140 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 20 realm 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56397173ccc0 con 0x563971739c80 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56397173d590 con 0x563971739c80 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.626+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56397173d930 con 0x563971739c80 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.627+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.627+0000 7f0a5d319800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.627+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.627+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56397173e690 0x563971740a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.627+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971740fc0 con 0x56397173e690 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.627+0000 7f0a5b604640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56397173e690 0x563971740a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.627+0000 7f0a5b604640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56397173e690 0x563971740a80 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5d319800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5d319800 20 searching for the correct realm 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5b604640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a4c03c7f0 0x7f0a4c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a4c03c7f0 0x7f0a4c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.628+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a4c03c7f0 0x7f0a4c03b610 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.629+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.629+0000 7f0a5ae03640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a5403c070 0x7f0a54054150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.629+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a5403c070 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.629+0000 7f0a5a602640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a5403c070 0x7f0a54054150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.629+0000 7f0a5a602640 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a5403c070 0x7f0a54054150 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.630+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.630+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.630+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.630+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.630+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.630+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a5403c070 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a5403c070 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.631+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.632+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.632+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a5403c070 2024-09-19T11:23:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.632+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.632+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.633+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.633+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.633+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.633+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.633+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.633+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.633+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.634+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.634+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.634+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.634+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.634+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.634+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.634+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.635+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.635+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.635+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.635+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a5403c070 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.635+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.635+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a5403c070 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.636+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.637+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.637+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a5403c070 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.637+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.637+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.637+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.637+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x56397173e690 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x563971739c80 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563971742930 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.638+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971741a00 con 0x563971739c80 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5639717422a0 con 0x563971739c80 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971741640 con 0x56397173e690 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:43.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 zone default found 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 4 Realm: () 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 zonegroup default 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.639+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971741ed0 con 0x7f0a5403c070 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971744d20 con 0x56397173e690 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5639717451e0 con 0x56397173e690 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 20 started sync module instance, tier type = 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.640+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717441c0 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.641+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.641+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94804716570032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971745ad0 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.641+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971746320 con 0x563971739c80 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.643+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94804716570032] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.643+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.643+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94804716574240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717463c0 con 0x563971739c80 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.643+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971746ee0 con 0x56397173e690 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.644+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94804716574240] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.645+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.645+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94804716579248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971746ea0 con 0x56397173e690 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.645+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971747940 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.646+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94804716579248] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.646+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.646+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94804716582000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971747d60 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.646+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971748780 con 0x563971739c80 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.647+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94804716582000] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.647+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.647+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94804716585584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971748b70 con 0x563971739c80 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.647+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717495d0 con 0x7f0a5403c070 2024-09-19T11:23:43.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.648+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.648+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94804716589232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717495d0 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.648+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x56397174a550 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.648+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94804716585584] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.649+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94804716589232] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.650+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.650+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94804716593216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971749880 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.650+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x56397174aa20 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94804716593216] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94804716594448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x56397174a9e0 con 0x7f0a5403c070 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 20 add_watcher() i=0 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 20 add_watcher() i=1 2024-09-19T11:23:43.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 20 add_watcher() i=2 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 20 add_watcher() i=3 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 20 add_watcher() i=4 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 20 add_watcher() i=5 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.651+0000 7f0a5d319800 20 add_watcher() i=6 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.652+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94804716594448] v33'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.652+0000 7f0a5d319800 20 add_watcher() i=7 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.652+0000 7f0a5d319800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:43.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.652+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56397174de40 con 0x563971739c80 2024-09-19T11:23:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.652+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.652+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94804716576368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x56397174b330 con 0x563971739c80 2024-09-19T11:23:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.653+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94804716576368] v33'152 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.653+0000 7f09117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.654+0000 7f0a5d319800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.654+0000 7f0a5d319800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.654+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971750210 con 0x563971739c80 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.655+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.655+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971756990 con 0x7f0a5403c070 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.656+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.656+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971756e40 con 0x56397173e690 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.657+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.657+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971756e40 con 0x563971739c80 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.658+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'153 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.658+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757220 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.659+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'44 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.659+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757400 con 0x56397173e690 2024-09-19T11:23:43.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.660+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'29 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.660+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757400 con 0x563971739c80 2024-09-19T11:23:43.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.662+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.662+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757490 con 0x7f0a5403c070 2024-09-19T11:23:43.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.663+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'29 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.663+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757520 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.664+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.664+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717579d0 con 0x56397173e690 2024-09-19T11:23:43.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.665+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'24 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.665+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717579d0 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.666+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'45 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.666+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717579d0 con 0x7f0a5403c070 2024-09-19T11:23:43.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.668+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'20 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.668+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717579d0 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.669+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.669+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5639717579d0 con 0x7f0a5403c070 2024-09-19T11:23:43.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.670+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'36 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.670+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757a60 con 0x563971739c80 2024-09-19T11:23:43.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.671+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'154 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.671+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757a60 con 0x56397173e690 2024-09-19T11:23:43.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.672+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'38 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.672+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757b20 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.673+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'46 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.673+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757b20 con 0x56397173e690 2024-09-19T11:23:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.674+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.674+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757af0 con 0x563971739c80 2024-09-19T11:23:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.675+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.675+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757af0 con 0x563971739c80 2024-09-19T11:23:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.676+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'155 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.676+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757ee0 con 0x56397173e690 2024-09-19T11:23:43.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.677+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'39 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.677+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757cc0 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.678+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'28 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.678+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757cc0 con 0x56397173e690 2024-09-19T11:23:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.679+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.679+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757fa0 con 0x56397173e690 2024-09-19T11:23:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.680+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.680+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x56397173e690 2024-09-19T11:23:43.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.681+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.681+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x56397173e690 2024-09-19T11:23:43.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.682+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'30 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.682+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x7f0a5403c070 2024-09-19T11:23:43.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.684+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'21 uv20 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.684+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x56397173e690 2024-09-19T11:23:43.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.685+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.685+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x7f0a5403c070 2024-09-19T11:23:43.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.686+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'30 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.686+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x56397173e690 2024-09-19T11:23:43.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.687+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'40 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.687+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x7f0a5403c070 2024-09-19T11:23:43.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.688+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'37 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.688+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971757f10 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.689+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.689+0000 7f088bfff640 20 reqs_thread_entry: start 2024-09-19T11:23:43.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.689+0000 7f0a5d319800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:43.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.689+0000 7f0a5d319800 20 Filter name: none 2024-09-19T11:23:43.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.690+0000 7f0889ffb640 20 reqs_thread_entry: start 2024-09-19T11:23:43.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.690+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.690+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.690+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756400 con 0x563971739c80 2024-09-19T11:23:43.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.690+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.690+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756400 con 0x563971739c80 2024-09-19T11:23:43.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5639717563e0 con 0x7f0a5403c070 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.691+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.692+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.692+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.692+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.692+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.692+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.692+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv13 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv13 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv13 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.693+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.694+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv9 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.694+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.694+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.694+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.694+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.694+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv8 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.695+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.696+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.696+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.696+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.696+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.696+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv8 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.696+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.697+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv9 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.698+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.699+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.699+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.699+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.699+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.699+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.699+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.700+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.701+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.701+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.701+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.701+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv12 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv12 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv15 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.702+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.703+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.703+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.703+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.703+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.703+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 38 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.703+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 39 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv13 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.704+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 40 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 41 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 42 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.705+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.706+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 43 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 44 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 45 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.707+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 36 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 37 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 46 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.708+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 47 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 48 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 49 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 50 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.709+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 51 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 52 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 53 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 54 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.710+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.711+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 55 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.711+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.711+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 56 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv4 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.711+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.711+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 57 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.711+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.712+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.712+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.712+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.712+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971756000 con 0x7f0a5403c070 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== osd_op_reply(174 gc.26 [call out=1702b] v0'0 uv20 ondisk = 0) ==== 149+0+1702 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x56397173e690 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 58 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971782c50 con 0x56397173e690 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 59 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x56397173e690 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 60 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.713+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x7f0a5403c070 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.714+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.714+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971782c50 con 0x7f0a5403c070 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.714+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv4 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.714+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x7f0a5403c070 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.714+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.714+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x56397173e690 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 61 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971782c50 con 0x56397173e690 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 62 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv9 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x56397173e690 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 63 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x7f0a5403c070 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv6 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.715+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971782c50 con 0x7f0a5403c070 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.716+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv6 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.716+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x7f0a5403c070 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.716+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv6 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.716+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.716+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 38 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.716+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971782c50 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.717+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 39 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.717+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563971783c60 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.717+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 40 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.718+0000 7f09117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.718+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94804716576368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755700 con 0x563971739c80 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.719+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 94804716576368] v33'156 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.719+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94804716570032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.720+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 41 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 94804716570032] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.720+0000 7f0a5d319800 20 remove_watcher() i=0 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.720+0000 7f0a5d319800 2 removed watcher, disabling cache 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.720+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94804716574240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x563971739c80 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.721+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 94804716574240] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.721+0000 7f0a5d319800 20 remove_watcher() i=1 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.721+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94804716579248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x56397173e690 2024-09-19T11:23:43.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.722+0000 7f0a5b604640 1 -- 172.21.15.28:0/3444010742 <== osd.3 v2:172.21.15.28:6808/3027134945 64 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 94804716579248] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a4c00a4d0 con 0x56397173e690 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.722+0000 7f0a5d319800 20 remove_watcher() i=2 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.722+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94804716582000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.723+0000 7f0a5ae03640 1 -- 172.21.15.28:0/3444010742 <== osd.2 v2:172.21.15.28:6800/2071226075 42 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 94804716582000] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a5402d070 con 0x7f0a4c03c7f0 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.723+0000 7f0a5d319800 20 remove_watcher() i=3 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.723+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94804716585584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x563971739c80 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.724+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.1 v2:172.21.15.28:6816/656119574 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 94804716585584] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x563971739c80 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.724+0000 7f0a5d319800 20 remove_watcher() i=4 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.724+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94804716589232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x7f0a5403c070 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.725+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 94804716589232] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.725+0000 7f0a5d319800 20 remove_watcher() i=5 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.725+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94804716593216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x7f0a5403c070 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.727+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 94804716593216] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.727+0000 7f0a5d319800 20 remove_watcher() i=6 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.727+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94804716594448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563971755fb0 con 0x7f0a5403c070 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 94804716594448] v33'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a44051070 con 0x7f0a5403c070 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 20 remove_watcher() i=7 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a5403c070 msgr2=0x7f0a54054150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a5403c070 0x7f0a54054150 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971739c80 msgr2=0x56397173c140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971739c80 0x56397173c140 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a4c03c7f0 msgr2=0x7f0a4c03b610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a4c03c7f0 0x7f0a4c03b610 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56397173e690 msgr2=0x563971740a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x56397173e690 0x563971740a80 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092403cfb0 msgr2=0x7f092403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092403cfb0 0x7f092403f470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c010 tx=0x7f0a5404f000 comp rx=0 tx=0).stop 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 msgr2=0x56397172e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 --2- 172.21.15.28:0/3444010742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971726e70 0x56397172e430 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0547e0 tx=0x7f0a4c054840 comp rx=0 tx=0).stop 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5a602640 1 -- 172.21.15.28:0/3444010742 reap_dead start 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 shutdown_connections 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 >> 172.21.15.28:0/3444010742 conn(0x563971723270 msgr2=0x563971721480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.728+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 shutdown_connections 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/3444010742 wait complete. 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f093403d000 msgr2=0x7f093403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f093403d000 0x7f093403f4c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x563971707150 tx=0x7f0a54052f30 comp rx=0 tx=0).stop 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56397171a3b0 msgr2=0x56397171a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56397171a3b0 0x56397171a7d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5639716fff50 tx=0x7f0a4c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 shutdown_connections 2024-09-19T11:23:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f093403d000 0x7f093403f4c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/734855529 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56397171a3b0 0x56397171a7d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 >> 172.21.15.28:0/734855529 conn(0x56397170e760 msgr2=0x56397170eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 shutdown_connections 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/734855529 wait complete. 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971700dd0 msgr2=0x563971706710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971700dd0 0x563971706710 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563971708fd0 msgr2=0x56397170b3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563971708fd0 0x56397170b3c0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f094003d1f0 msgr2=0x7f094003f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f094003d1f0 0x7f094003f6b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0a44057000 tx=0x7f0a44052000 comp rx=0 tx=0).stop 2024-09-19T11:23:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 msgr2=0x5639716ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x5639716ff0c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0a540012f0 tx=0x7f0a54000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 shutdown_connections 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563971708fd0 0x56397170b3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f094003d1f0 0x7f094003f6b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563971700dd0 0x563971706710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 --2- 172.21.15.28:0/155902037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563971608d70 0x5639716ff0c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 >> 172.21.15.28:0/155902037 conn(0x56397157ad70 msgr2=0x5639716f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.729+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 shutdown_connections 2024-09-19T11:23:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.730+0000 7f0a5d319800 1 -- 172.21.15.28:0/155902037 wait complete. 2024-09-19T11:23:43.741 INFO:teuthology.orchestra.run.smithi028.stdout:[{"tag":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.18292526871366832926\u0000","time":"2024-09-19T11:23:43.531487+0000","objs":[{"pool":"default.rgw.buckets.data","oid":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_1","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_2","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_3","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_4","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_5","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_6","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_7","key":"","instance":""}]}] 2024-09-19T11:23:43.741 DEBUG:tasks.util.rgw: json result: [{'tag': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.18292526871366832926\x00', 'time': '2024-09-19T11:23:43.531487+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_7', 'key': '', 'instance': ''}]}] 2024-09-19T11:23:43.742 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-09-19T11:23:43.742 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-09-19T11:23:43.742 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-09-19T11:23:43.785 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:43.785 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.784+0000 7fbaec146800 1 Processor -- start 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbaec146800 1 -- start start 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbaec146800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c97de2ab50 0x55c97de306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbaec146800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c97de29f30 con 0x55c97de29880 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbaec146800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55c97dcb47d0 con 0x55c97de2ab50 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbae9436640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c97de2ab50 0x55c97de306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbae9c37640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55c97de29880 0x55c97de2a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55562/0 (socket says 172.21.15.28:55562) 2024-09-19T11:23:43.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbae9c37640 1 -- 172.21.15.28:0/3949929963 learned_addr learned my addr 172.21.15.28:0/3949929963 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:43.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbae9436640 1 -- 172.21.15.28:0/3949929963 >> v1:172.21.15.22:6789/0 conn(0x55c97de29880 legacy=0x55c97de2a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.785+0000 7fbae8c35640 1 -- 172.21.15.28:0/3949929963 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4289756572 0 0) 0x55c97de29f30 con 0x55c97de29880 2024-09-19T11:23:43.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.786+0000 7fbae9436640 1 -- 172.21.15.28:0/3949929963 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97de2b000 con 0x55c97de2ab50 2024-09-19T11:23:43.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.787+0000 7fbae9436640 1 --2- 172.21.15.28:0/3949929963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de2ab50 0x55c97de306f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbae000a470 tx=0x7fbae0045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee8e4a7fc55681a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbae8c35640 1 -- 172.21.15.28:0/3949929963 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae0053570 con 0x55c97de2ab50 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbae8c35640 1 -- 172.21.15.28:0/3949929963 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae0058070 con 0x55c97de2ab50 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbae8c35640 1 -- 172.21.15.28:0/3949929963 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae0053880 con 0x55c97de2ab50 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbaec146800 1 -- 172.21.15.28:0/3949929963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de2ab50 msgr2=0x55c97de306f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbaec146800 1 --2- 172.21.15.28:0/3949929963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de2ab50 0x55c97de306f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbae000a470 tx=0x7fbae0045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbaec146800 1 -- 172.21.15.28:0/3949929963 shutdown_connections 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbaec146800 1 --2- 172.21.15.28:0/3949929963 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de2ab50 0x55c97de306f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbaec146800 1 -- 172.21.15.28:0/3949929963 >> 172.21.15.28:0/3949929963 conn(0x55c97de24c50 msgr2=0x55c97de270b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbaec146800 1 -- 172.21.15.28:0/3949929963 shutdown_connections 2024-09-19T11:23:43.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.788+0000 7fbaec146800 1 -- 172.21.15.28:0/3949929963 wait complete. 2024-09-19T11:23:43.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.803+0000 7fbaec146800 1 Processor -- start 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.803+0000 7fbaec146800 1 -- start start 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.803+0000 7fbaec146800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.803+0000 7fbaec146800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c97dcc1dd0 con 0x55c97de29880 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fbae9c37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fbae9c37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df190f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53148/0 (socket says 172.21.15.28:53148) 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fbae9c37640 1 -- 172.21.15.28:0/2830403611 learned_addr learned my addr 172.21.15.28:0/2830403611 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fbae9c37640 1 -- 172.21.15.28:0/2830403611 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97df196e0 con 0x55c97de29880 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fbae9c37640 1 --2- 172.21.15.28:0/2830403611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df190f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbadc001ea0 tx=0x7fbadc00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8abdd33f3f7f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fb9d7e07640 1 -- 172.21.15.28:0/2830403611 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc008070 con 0x55c97de29880 2024-09-19T11:23:43.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fb9d7e07640 1 -- 172.21.15.28:0/2830403611 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbadc00d500 con 0x55c97de29880 2024-09-19T11:23:43.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.804+0000 7fb9d7e07640 1 -- 172.21.15.28:0/2830403611 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc00d880 con 0x55c97de29880 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 -- 172.21.15.28:0/2830403611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 msgr2=0x55c97df190f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 --2- 172.21.15.28:0/2830403611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df190f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbadc001ea0 tx=0x7fbadc00b6a0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 -- 172.21.15.28:0/2830403611 shutdown_connections 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 --2- 172.21.15.28:0/2830403611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df190f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 -- 172.21.15.28:0/2830403611 >> 172.21.15.28:0/2830403611 conn(0x55c97de24c50 msgr2=0x55c97df14a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 -- 172.21.15.28:0/2830403611 shutdown_connections 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 -- 172.21.15.28:0/2830403611 wait complete. 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 Processor -- start 2024-09-19T11:23:43.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.805+0000 7fbaec146800 1 -- start start 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.806+0000 7fbaec146800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df20c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.806+0000 7fbaec146800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c97de30e40 con 0x55c97de29880 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.806+0000 7fbae9c37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.806+0000 7fbae9c37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df20c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53164/0 (socket says 172.21.15.28:53164) 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.806+0000 7fbae9c37640 1 -- 172.21.15.28:0/4027793003 learned_addr learned my addr 172.21.15.28:0/4027793003 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.806+0000 7fbae9c37640 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97df211a0 con 0x55c97de29880 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.806+0000 7fbae9c37640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df20c40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0012f0 tx=0x7fbadc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.807+0000 7fb9d6604640 1 -- 172.21.15.28:0/4027793003 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc00f9d0 con 0x55c97de29880 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.807+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c97df21430 con 0x55c97de29880 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.807+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97df21e30 con 0x55c97de29880 2024-09-19T11:23:43.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.807+0000 7fb9d6604640 1 -- 172.21.15.28:0/4027793003 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbadc00fb70 con 0x55c97de29880 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.807+0000 7fb9d6604640 1 -- 172.21.15.28:0/4027793003 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbadc02b060 con 0x55c97de29880 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fb9d6604640 1 -- 172.21.15.28:0/4027793003 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbadc017040 con 0x55c97de29880 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fb9d6604640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9d003d000 0x7fb9d003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fb9d6604640 1 -- 172.21.15.28:0/4027793003 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbadc015020 con 0x55c97de29880 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fbae9436640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9d003d000 0x7fb9d003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97de2ab50 0x55c97df28290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df287d0 con 0x55c97de2ab50 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fbaea438640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97de2ab50 0x55c97df28290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fbae9436640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9d003d000 0x7fb9d003f4c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbae0057000 tx=0x7fbae0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.808+0000 7fbaea438640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97de2ab50 0x55c97df28290 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.809+0000 7fbaea438640 1 -- 172.21.15.28:0/4027793003 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97de2ab50 2024-09-19T11:23:43.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.809+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df28b70 con 0x55c97de2ab50 2024-09-19T11:23:43.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.809+0000 7fbaea438640 1 -- 172.21.15.28:0/4027793003 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97de2ab50 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.809+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df2a630 0x55c97df2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.809+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df2d230 con 0x55c97df2a630 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.809+0000 7fbae9c37640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df2a630 0x55c97df2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.809+0000 7fbae9c37640 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df2a630 0x55c97df2caa0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.810+0000 7fbae9c37640 1 -- 172.21.15.28:0/4027793003 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbadc02d070 con 0x55c97df2a630 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.810+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df293c0 con 0x55c97de2ab50 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.810+0000 7fbaea438640 1 -- 172.21.15.28:0/4027793003 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97de2ab50 2024-09-19T11:23:43.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.810+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df29760 con 0x55c97df2a630 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.810+0000 7fbae9c37640 1 -- 172.21.15.28:0/4027793003 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbadc02d070 con 0x55c97df2a630 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaec146800 1 Processor -- start 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaec146800 1 -- start start 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaec146800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97dd9f180 0x55c97dd9d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaec146800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c97dd204a0 con 0x55c97dd9f180 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaea438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97dd9f180 0x55c97dd9d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaea438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97dd9f180 0x55c97dd9d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53172/0 (socket says 172.21.15.28:53172) 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaea438640 1 -- 172.21.15.28:0/3124710886 learned_addr learned my addr 172.21.15.28:0/3124710886 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.811+0000 7fbaea438640 1 -- 172.21.15.28:0/3124710886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97dd9f6a0 con 0x55c97dd9f180 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.812+0000 7fbaea438640 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97dd9f180 0x55c97dd9d9f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55c97df21c40 tx=0x7fbae40167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.812+0000 7fb9cf7fe640 1 -- 172.21.15.28:0/3124710886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4013640 con 0x55c97dd9f180 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.812+0000 7fb9cf7fe640 1 -- 172.21.15.28:0/3124710886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae40137e0 con 0x55c97dd9f180 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.812+0000 7fb9cf7fe640 1 -- 172.21.15.28:0/3124710886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4013ae0 con 0x55c97dd9f180 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.812+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c97dd9df30 con 0x55c97dd9f180 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.812+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97dd9e480 con 0x55c97dd9f180 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fb9cf7fe640 1 -- 172.21.15.28:0/3124710886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbae4011040 con 0x55c97dd9f180 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fb9cf7fe640 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9c403d020 0x7fb9c403f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fb9cf7fe640 1 -- 172.21.15.28:0/3124710886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbae400e040 con 0x55c97dd9f180 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fbae9c37640 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9c403d020 0x7fb9c403f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fbaec146800 1 Processor -- start 2024-09-19T11:23:43.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fbaec146800 1 -- start start 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fbae9c37640 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9c403d020 0x7fb9c403f4e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55c97df22d10 tx=0x7fbadc001440 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.813+0000 7fbaec146800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df4acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fbaec146800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c97df19b10 con 0x55c97df4a8f0 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fbaea438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df4acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fbaea438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df4acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53184/0 (socket says 172.21.15.28:53184) 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fbaea438640 1 -- 172.21.15.28:0/2633468909 learned_addr learned my addr 172.21.15.28:0/2633468909 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fbaea438640 1 -- 172.21.15.28:0/2633468909 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97df4b2c0 con 0x55c97df4a8f0 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fbaea438640 1 --2- 172.21.15.28:0/2633468909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df4acd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbae4020000 tx=0x7fbae401f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e550b44915993575 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fb9cd7fa640 1 -- 172.21.15.28:0/2633468909 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4021a90 con 0x55c97df4a8f0 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fb9cd7fa640 1 -- 172.21.15.28:0/2633468909 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4052070 con 0x55c97df4a8f0 2024-09-19T11:23:43.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.814+0000 7fb9cd7fa640 1 -- 172.21.15.28:0/2633468909 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4021570 con 0x55c97df4a8f0 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 -- 172.21.15.28:0/2633468909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 msgr2=0x55c97df4acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 --2- 172.21.15.28:0/2633468909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df4acd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbae4020000 tx=0x7fbae401f400 comp rx=0 tx=0).stop 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 -- 172.21.15.28:0/2633468909 shutdown_connections 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 --2- 172.21.15.28:0/2633468909 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df4acd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 -- 172.21.15.28:0/2633468909 >> 172.21.15.28:0/2633468909 conn(0x55c97df46cf0 msgr2=0x55c97df44f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 -- 172.21.15.28:0/2633468909 shutdown_connections 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 -- 172.21.15.28:0/2633468909 wait complete. 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 Processor -- start 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 -- start start 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df51e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaec146800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c97df4b900 con 0x55c97df4a8f0 2024-09-19T11:23:43.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaea438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df51e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaea438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df51e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53188/0 (socket says 172.21.15.28:53188) 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.815+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 learned_addr learned my addr 172.21.15.28:0/1974599229 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.816+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97df523c0 con 0x55c97df4a8f0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.816+0000 7fbaea438640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df51e60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbae4054800 tx=0x7fbae4054860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.816+0000 7fb9bb7fe640 1 -- 172.21.15.28:0/1974599229 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae4020bd0 con 0x55c97df4a8f0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.816+0000 7fb9bb7fe640 1 -- 172.21.15.28:0/1974599229 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4020d70 con 0x55c97df4a8f0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.816+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c97df52650 con 0x55c97df4a8f0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.816+0000 7fb9bb7fe640 1 -- 172.21.15.28:0/1974599229 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbae401fa90 con 0x55c97df4a8f0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.816+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97df52fd0 con 0x55c97df4a8f0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.817+0000 7fb9bb7fe640 1 -- 172.21.15.28:0/1974599229 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbae401fc30 con 0x55c97df4a8f0 2024-09-19T11:23:43.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.817+0000 7fb9bb7fe640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9b403d050 0x7fb9b403f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.817+0000 7fb9bb7fe640 1 -- 172.21.15.28:0/1974599229 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbae4034d50 con 0x55c97df4a8f0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.817+0000 7fbae9c37640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9b403d050 0x7fb9b403f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.818+0000 7fbae9c37640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9b403d050 0x7fb9b403f510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbadc01ac20 tx=0x7fbadc01bf90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.818+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.818+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97df5d780 0x55c97df5fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.818+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df60390 con 0x55c97df5d780 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.818+0000 7fbae9436640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97df5d780 0x55c97df5fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.818+0000 7fbae9436640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97df5d780 0x55c97df5fc40 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 realm 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df607c0 con 0x55c97df5d780 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df61090 con 0x55c97df5d780 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df61430 con 0x55c97df5d780 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df62190 0x55c97df64580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.819+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df64b90 con 0x55c97df62190 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaea438640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df62190 0x55c97df64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaea438640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df62190 0x55c97df64580 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaec146800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaec146800 20 searching for the correct realm 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaea438640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbae4046040 0x7fbae4046810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.820+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.821+0000 7fbae9c37640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbae4046040 0x7fbae4046810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.821+0000 7fbae9c37640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbae4046040 0x7fbae4046810 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.821+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.821+0000 7fbae9c37640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbadc046030 0x7fbadc026430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:43.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.821+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbadc046030 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.821+0000 7fbae9436640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbadc046030 0x7fbadc026430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.822+0000 7fbae9436640 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbadc046030 0x7fbadc026430 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.822+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.822+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.822+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.822+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.822+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.822+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbadc046030 2024-09-19T11:23:43.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.823+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.823+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbadc046030 2024-09-19T11:23:43.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.823+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.823+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.823+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.823+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.823+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.824+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.824+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.824+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbadc046030 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.824+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.824+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.824+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.824+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.825+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.826+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbadc046030 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.827+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbadc046030 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.828+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.828+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.828+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.828+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbadc046030 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.828+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.828+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df62190 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x55c97df5d780 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.829+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c97df664c0 con 0x7fbae4046040 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df65550 con 0x55c97df5d780 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df65da0 con 0x55c97df5d780 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.830+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df66940 con 0x55c97df62190 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 zone default found 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 4 Realm: () 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 zonegroup default 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df66d20 con 0x7fbadc046030 2024-09-19T11:23:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df68b50 con 0x55c97df62190 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 rados->read ofs=0 len=0 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df68fa0 con 0x55c97df62190 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 started sync module instance, tier type = 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.831+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df67ff0 con 0x7fbae4046040 2024-09-19T11:23:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.833+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.833+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94323890095120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df67c50 con 0x7fbae4046040 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.833+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df69ed0 con 0x55c97df5d780 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.834+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94323890095120] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.834+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.834+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94323890100768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6a2f0 con 0x55c97df5d780 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.834+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6acb0 con 0x55c97df62190 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.835+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94323890100768] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.835+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.835+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94323890104800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6a2f0 con 0x55c97df62190 2024-09-19T11:23:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.835+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6b700 con 0x7fbae4046040 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.836+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94323890104800] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.836+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.836+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94323890101824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6a260 con 0x7fbae4046040 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.836+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6c5c0 con 0x55c97df5d780 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.837+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94323890101824] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.837+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.837+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94323890109168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6c5c0 con 0x55c97df5d780 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.837+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6cfc0 con 0x7fbadc046030 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.838+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94323890109168] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.838+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.839+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94323890113232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6c5c0 con 0x7fbadc046030 2024-09-19T11:23:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.839+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6da30 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.840+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.840+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94323890115872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6dea0 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.840+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6e980 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.840+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94323890113232] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94323890115872] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94323890119824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6d9f0 con 0x7fbadc046030 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 20 add_watcher() i=0 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 20 add_watcher() i=1 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 20 add_watcher() i=2 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 20 add_watcher() i=3 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 20 add_watcher() i=4 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 20 add_watcher() i=5 2024-09-19T11:23:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbaec146800 20 add_watcher() i=6 2024-09-19T11:23:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.841+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94323890119824] v33'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.842+0000 7fbaec146800 20 add_watcher() i=7 2024-09-19T11:23:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.842+0000 7fbaec146800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.842+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97df71a80 con 0x55c97df5d780 2024-09-19T11:23:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.842+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.842+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94323890110736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6ef00 con 0x55c97df5d780 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.843+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94323890110736] v33'157 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.843+0000 7fb99b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.843+0000 7fbaec146800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.843+0000 7fbaec146800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.843+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7a170 con 0x55c97df5d780 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.844+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.845+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7a170 con 0x7fbadc046030 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.846+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.846+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7ab90 con 0x55c97df62190 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.847+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.847+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7ac20 con 0x55c97df5d780 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.848+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'158 uv13 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.848+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b020 con 0x7fbae4046040 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.849+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'47 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.849+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b590 con 0x55c97df62190 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.850+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'31 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.850+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7ba40 con 0x55c97df5d780 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.851+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.851+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7a170 con 0x7fbadc046030 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.852+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'31 uv8 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.852+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7a170 con 0x7fbae4046040 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.853+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.853+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b870 con 0x55c97df62190 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.854+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'25 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.854+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b7b0 con 0x7fbae4046040 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.855+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'48 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.855+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b840 con 0x7fbadc046030 2024-09-19T11:23:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.856+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'21 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.856+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b840 con 0x7fbae4046040 2024-09-19T11:23:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.857+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.857+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b840 con 0x7fbadc046030 2024-09-19T11:23:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.858+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'38 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.858+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b840 con 0x55c97df5d780 2024-09-19T11:23:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.859+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'159 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.859+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7b840 con 0x55c97df62190 2024-09-19T11:23:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.860+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'41 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.860+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bf10 con 0x7fbae4046040 2024-09-19T11:23:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.861+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'49 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.861+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.862+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.862+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df5d780 2024-09-19T11:23:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.863+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.863+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df5d780 2024-09-19T11:23:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.864+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'160 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.864+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.865+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'42 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.865+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x7fbae4046040 2024-09-19T11:23:43.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.866+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'29 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.867+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.867+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.868+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.868+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.868+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.869+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.869+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.870+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'32 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.870+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x7fbadc046030 2024-09-19T11:23:43.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.871+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'22 uv20 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.871+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.872+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.872+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x7fbadc046030 2024-09-19T11:23:43.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.874+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'32 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.874+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x55c97df62190 2024-09-19T11:23:43.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.875+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'43 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.875+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x7fbadc046030 2024-09-19T11:23:43.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.876+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'39 uv6 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.876+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df7bef0 con 0x7fbae4046040 2024-09-19T11:23:43.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.877+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.877+0000 7fb99a7fc640 20 reqs_thread_entry: start 2024-09-19T11:23:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.877+0000 7fbaec146800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.877+0000 7fbaec146800 20 Filter name: none 2024-09-19T11:23:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.877+0000 7fb90ffff640 20 reqs_thread_entry: start 2024-09-19T11:23:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.878+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-09-19T11:23:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.878+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:94 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.879+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(94 gc.8 [call] v33'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.879+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.879+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(95 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.879+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.879+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:96 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.879+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(96 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.879+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:97 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.880+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(97 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.880+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.880+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.880+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(98 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.880+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.880+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:99 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.881+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(99 gc.8 [call] v33'22 uv22 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.881+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-09-19T11:23:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.881+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:100 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.882+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(100 gc.9 [call] v33'26 uv26 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.882+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:101 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.882+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(101 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.882+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.882+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:102 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(102 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:103 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(103 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:104 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(104 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.883+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:105 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.884+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(105 gc.9 [call] v33'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.884+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-09-19T11:23:43.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.884+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:106 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.885+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(106 gc.10 [call] v33'50 uv50 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.885+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:107 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(107 gc.10 [call out=15b] v0'0 uv50 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:108 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(108 gc.10 [call out=24b] v0'0 uv50 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:109 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(109 gc.10 [call out=15b] v0'0 uv50 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:110 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== osd_op_reply(110 gc.10 [call out=15b] v0'0 uv50 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.886+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:111 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.887+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(111 gc.10 [call] v33'51 uv51 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.887+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-09-19T11:23:43.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.887+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:112 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.888+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(112 gc.11 [call] v33'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.888+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:113 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(113 gc.11 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:114 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbadc046030 2024-09-19T11:23:43.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(114 gc.11 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:115 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(115 gc.11 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.889+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:116 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.890+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(116 gc.11 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.890+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.890+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:117 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.891+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(117 gc.11 [call] v33'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.891+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-09-19T11:23:43.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.891+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:118 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(118 gc.12 [call] v33'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:119 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== osd_op_reply(119 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:120 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(120 gc.12 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:121 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(121 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.893+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:122 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.894+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 36 ==== osd_op_reply(122 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.894+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.894+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:123 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.894+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 37 ==== osd_op_reply(123 gc.12 [call] v33'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.895+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-09-19T11:23:43.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.895+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:124 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.896+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(124 gc.13 [call] v33'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.896+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:125 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.896+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(125 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.896+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.896+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:126 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbadc046030 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.896+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(126 gc.13 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.896+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:127 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.897+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(127 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.897+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.897+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:128 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.897+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(128 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.897+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.897+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:129 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.898+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(129 gc.13 [call] v33'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.898+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-09-19T11:23:43.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.898+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:130 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.899+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(130 gc.14 [call] v33'161 uv161 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.899+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:131 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.899+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(131 gc.14 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.899+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.899+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:132 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.899+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(132 gc.14 [call out=24b] v0'0 uv161 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.899+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:133 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.900+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(133 gc.14 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.900+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.900+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:134 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.900+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(134 gc.14 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.900+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.900+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:135 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.901+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(135 gc.14 [call] v33'162 uv162 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.901+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-09-19T11:23:43.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.901+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:136 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(136 gc.15 [call] v33'44 uv44 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:137 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(137 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:138 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(138 gc.15 [call out=24b] v0'0 uv44 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.902+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.903+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.903+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.903+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 38 ==== osd_op_reply(140 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.903+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.903+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.904+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 39 ==== osd_op_reply(141 gc.15 [call] v33'45 uv45 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.904+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-09-19T11:23:43.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.904+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.905+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 38 ==== osd_op_reply(142 gc.16 [call] v33'52 uv52 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.905+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.905+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 39 ==== osd_op_reply(143 gc.16 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.905+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.905+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.905+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 40 ==== osd_op_reply(144 gc.16 [call out=24b] v0'0 uv52 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.905+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:145 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.906+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 41 ==== osd_op_reply(145 gc.16 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.906+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.906+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:146 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.906+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 42 ==== osd_op_reply(146 gc.16 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.906+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.906+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:147 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.907+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 43 ==== osd_op_reply(147 gc.16 [call] v33'53 uv53 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.907+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-09-19T11:23:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.907+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:148 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 40 ==== osd_op_reply(148 gc.17 [call] v33'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:149 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 41 ==== osd_op_reply(149 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:150 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 42 ==== osd_op_reply(150 gc.17 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:151 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 43 ==== osd_op_reply(151 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.908+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:152 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.909+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 44 ==== osd_op_reply(152 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.909+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.909+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:153 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.910+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 45 ==== osd_op_reply(153 gc.17 [call] v33'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.910+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-09-19T11:23:43.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.910+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:154 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.911+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(154 gc.18 [call] v33'28 uv28 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.911+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:155 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.911+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(155 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.911+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.911+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:156 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df5d780 2024-09-19T11:23:43.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.911+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(156 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.911+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:157 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.912+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(157 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.912+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.912+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:158 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.912+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(158 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.912+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.912+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:159 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.913+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(159 gc.18 [call] v33'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.913+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-09-19T11:23:43.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.913+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:160 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.914+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(160 gc.19 [call] v33'163 uv163 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.914+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:161 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.914+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(161 gc.19 [call out=15b] v0'0 uv163 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.914+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.914+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:162 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df5d780 2024-09-19T11:23:43.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 41 ==== osd_op_reply(162 gc.19 [call out=24b] v0'0 uv163 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:163 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 42 ==== osd_op_reply(163 gc.19 [call out=15b] v0'0 uv163 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:164 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 43 ==== osd_op_reply(164 gc.19 [call out=15b] v0'0 uv163 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.915+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:165 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df5d780 2024-09-19T11:23:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.916+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 44 ==== osd_op_reply(165 gc.19 [call] v33'164 uv164 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.916+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-09-19T11:23:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.917+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:166 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 46 ==== osd_op_reply(166 gc.20 [call] v33'46 uv46 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:167 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 47 ==== osd_op_reply(167 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:168 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 48 ==== osd_op_reply(168 gc.20 [call out=24b] v0'0 uv46 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:169 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 49 ==== osd_op_reply(169 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.918+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:170 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.919+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 50 ==== osd_op_reply(170 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.919+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.919+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:171 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.920+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 51 ==== osd_op_reply(171 gc.20 [call] v33'47 uv47 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.920+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-09-19T11:23:43.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.920+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:172 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.920+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 44 ==== osd_op_reply(172 gc.21 [call] v33'30 uv30 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:173 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 45 ==== osd_op_reply(173 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:174 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 46 ==== osd_op_reply(174 gc.21 [call out=24b] v0'0 uv30 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:175 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 47 ==== osd_op_reply(175 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:176 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 48 ==== osd_op_reply(176 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.921+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:177 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbae4046040 2024-09-19T11:23:43.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.922+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 49 ==== osd_op_reply(177 gc.21 [call] v33'31 uv31 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.922+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-09-19T11:23:43.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.922+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:178 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.923+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 52 ==== osd_op_reply(178 gc.22 [call] v33'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.923+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:179 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 53 ==== osd_op_reply(179 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:180 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 54 ==== osd_op_reply(180 gc.22 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:181 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 55 ==== osd_op_reply(181 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:182 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 56 ==== osd_op_reply(182 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.924+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:183 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.925+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 57 ==== osd_op_reply(183 gc.22 [call] v33'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.925+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-09-19T11:23:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.925+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:184 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.926+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 58 ==== osd_op_reply(184 gc.23 [call] v33'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.926+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:185 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.926+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 59 ==== osd_op_reply(185 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.926+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.926+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:186 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 60 ==== osd_op_reply(186 gc.23 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:187 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 61 ==== osd_op_reply(187 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:188 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 62 ==== osd_op_reply(188 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.927+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:189 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.928+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 63 ==== osd_op_reply(189 gc.23 [call] v33'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.928+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-09-19T11:23:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.928+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:190 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.929+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 64 ==== osd_op_reply(190 gc.24 [call] v33'16 uv16 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.929+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:191 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.929+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 65 ==== osd_op_reply(191 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.929+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.929+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:192 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 66 ==== osd_op_reply(192 gc.24 [call out=24b] v0'0 uv16 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:193 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 67 ==== osd_op_reply(193 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:194 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 68 ==== osd_op_reply(194 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.930+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:195 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.931+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 69 ==== osd_op_reply(195 gc.24 [call] v33'17 uv17 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.931+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-09-19T11:23:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.931+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:196 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.932+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 70 ==== osd_op_reply(196 gc.25 [call] v33'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.932+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:197 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.932+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 71 ==== osd_op_reply(197 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.932+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.932+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:198 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x55c97df62190 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.932+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 72 ==== osd_op_reply(198 gc.25 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.932+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:199 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.933+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 73 ==== osd_op_reply(199 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.933+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.933+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:200 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.933+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 74 ==== osd_op_reply(200 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.933+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.933+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:201 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x55c97df62190 2024-09-19T11:23:43.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.934+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 75 ==== osd_op_reply(201 gc.25 [call] v33'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.934+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-09-19T11:23:43.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.934+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:202 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.935+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(202 gc.26 [call] v33'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.935+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:203 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.935+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(203 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.935+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.935+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:204 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa69d0 con 0x7fbadc046030 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.935+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== osd_op_reply(204 gc.26 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.935+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:205 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(205 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:206 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa79e0 con 0x7fbadc046030 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== osd_op_reply(206 gc.26 [call out=1702b] v0'0 uv23 ondisk = 0) ==== 149+0+1702 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 20 garbage collection: RGWGC::process iterating over entry tag='293899f1-2738-4b64-a8e1-a18ff039d626.4229.18292526871366832926', time=2024-09-19T11:23:43.531487+0000, chain.objs.size()=7 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_1 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:207 3.18s0 3:1b6bc5cf:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_1:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfa5c80 con 0x55c97df62190 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_2 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:208 3.3cs0 3:3e500a34:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_2:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaacf0 con 0x7fbae4046040 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_3 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:209 3.16s0 3:691ff561:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_3:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad6a0 con 0x7fbae4046040 2024-09-19T11:23:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_4 2024-09-19T11:23:43.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:210 3.3fs0 3:fe31f92e:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_4:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaff30 con 0x7fbae4046040 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_5 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:211 3.9s0 3:9399d41a:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_5:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad6a0 con 0x55c97df62190 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_6 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:212 3.22s0 3:46976b19:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_6:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfb4cd0 con 0x55c97df62190 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_7 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.936+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:213 3.12s0 3:4b97665a:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_7:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfb4cd0 con 0x7fbae4046040 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.939+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 50 ==== osd_op_reply(208 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_2 [call] v33'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.939+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 51 ==== osd_op_reply(213 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_7 [call] v33'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.939+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 52 ==== osd_op_reply(210 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_4 [call] v33'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.939+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 76 ==== osd_op_reply(207 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_1 [call] v33'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.940+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 77 ==== osd_op_reply(212 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_6 [call] v33'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.940+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 78 ==== osd_op_reply(211 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_5 [call] v33'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.940+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 53 ==== osd_op_reply(209 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1__shadow_.tFEk4nzvot1VebYWWWqt8wvdLBPjgcg_3 [call] v33'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.940+0000 7fbaec146800 5 garbage collection: RGWGC::process removing entries, marker: 2024-09-19T11:23:43.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.940+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:214 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.942+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(214 gc.26 [call] v33'24 uv24 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.943+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:215 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.943+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== osd_op_reply(215 gc.26 [call] v33'25 uv25 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.943+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-09-19T11:23:43.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.943+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:216 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.944+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 79 ==== osd_op_reply(216 gc.27 [call] v33'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:217 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 80 ==== osd_op_reply(217 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:218 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 81 ==== osd_op_reply(218 gc.27 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:219 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 82 ==== osd_op_reply(219 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.945+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:220 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.946+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 83 ==== osd_op_reply(220 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.946+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.946+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:221 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.947+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 84 ==== osd_op_reply(221 gc.27 [call] v33'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.947+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-09-19T11:23:43.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.947+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:222 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(222 gc.28 [call] v33'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:223 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== osd_op_reply(223 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:224 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(224 gc.28 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:225 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== osd_op_reply(225 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.948+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:226 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.949+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== osd_op_reply(226 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.949+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.949+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:227 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.950+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 46 ==== osd_op_reply(227 gc.28 [call] v33'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.950+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-09-19T11:23:43.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.950+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:228 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 85 ==== osd_op_reply(228 gc.29 [call] v33'48 uv48 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:229 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 86 ==== osd_op_reply(229 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:230 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 87 ==== osd_op_reply(230 gc.29 [call out=24b] v0'0 uv48 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:231 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 88 ==== osd_op_reply(231 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.951+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:232 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.952+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 89 ==== osd_op_reply(232 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.952+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.952+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:233 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x55c97df62190 2024-09-19T11:23:43.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.953+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 90 ==== osd_op_reply(233 gc.29 [call] v33'49 uv49 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.953+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-09-19T11:23:43.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.953+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:234 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.953+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 47 ==== osd_op_reply(234 gc.30 [call] v33'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.953+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:235 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 48 ==== osd_op_reply(235 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:236 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 49 ==== osd_op_reply(236 gc.30 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:237 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 50 ==== osd_op_reply(237 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:238 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 51 ==== osd_op_reply(238 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.954+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:239 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbadc046030 2024-09-19T11:23:43.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.955+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 52 ==== osd_op_reply(239 gc.30 [call] v33'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.955+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-09-19T11:23:43.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.955+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:240 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfad190 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.956+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 54 ==== osd_op_reply(240 gc.31 [call] v33'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.956+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:241 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 55 ==== osd_op_reply(241 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:242 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 56 ==== osd_op_reply(242 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:243 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 57 ==== osd_op_reply(243 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:244 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 58 ==== osd_op_reply(244 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.957+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:245 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.958+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 59 ==== osd_op_reply(245 gc.31 [call] v33'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.958+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-09-19T11:23:43.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.958+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:246 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.959+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 45 ==== osd_op_reply(246 gc.0 [call] v33'26 uv26 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.959+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:247 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 46 ==== osd_op_reply(247 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:248 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 47 ==== osd_op_reply(248 gc.0 [call out=24b] v0'0 uv26 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:249 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 48 ==== osd_op_reply(249 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:250 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 49 ==== osd_op_reply(250 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.960+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:251 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.961+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 50 ==== osd_op_reply(251 gc.0 [call] v33'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.961+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-09-19T11:23:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.961+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:252 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.962+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 53 ==== osd_op_reply(252 gc.1 [call] v33'20 uv20 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.962+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:253 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 54 ==== osd_op_reply(253 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:254 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 55 ==== osd_op_reply(254 gc.1 [call out=24b] v0'0 uv20 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:255 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 56 ==== osd_op_reply(255 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:256 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 57 ==== osd_op_reply(256 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.963+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:257 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.964+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 58 ==== osd_op_reply(257 gc.1 [call] v33'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.964+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-09-19T11:23:43.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.964+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:258 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.965+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 91 ==== osd_op_reply(258 gc.2 [call] v33'20 uv20 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.965+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:259 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 92 ==== osd_op_reply(259 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:260 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 93 ==== osd_op_reply(260 gc.2 [call out=24b] v0'0 uv20 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:261 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 94 ==== osd_op_reply(261 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:262 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 95 ==== osd_op_reply(262 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.966+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:263 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.967+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 96 ==== osd_op_reply(263 gc.2 [call] v33'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.967+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-09-19T11:23:43.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.967+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:264 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.968+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 51 ==== osd_op_reply(264 gc.3 [call] v33'165 uv165 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.968+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:265 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.968+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 52 ==== osd_op_reply(265 gc.3 [call out=15b] v0'0 uv165 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.968+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.968+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:266 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 53 ==== osd_op_reply(266 gc.3 [call out=24b] v0'0 uv165 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:267 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 54 ==== osd_op_reply(267 gc.3 [call out=15b] v0'0 uv165 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:268 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 55 ==== osd_op_reply(268 gc.3 [call out=15b] v0'0 uv165 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.969+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:269 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.970+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 56 ==== osd_op_reply(269 gc.3 [call] v33'166 uv166 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.970+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-09-19T11:23:43.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.970+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:270 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.971+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 60 ==== osd_op_reply(270 gc.4 [call] v33'54 uv54 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.971+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:271 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.971+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 61 ==== osd_op_reply(271 gc.4 [call out=15b] v0'0 uv54 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.971+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.971+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:272 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x7fbae4046040 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 62 ==== osd_op_reply(272 gc.4 [call out=24b] v0'0 uv54 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:273 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 63 ==== osd_op_reply(273 gc.4 [call out=15b] v0'0 uv54 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:274 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 64 ==== osd_op_reply(274 gc.4 [call out=15b] v0'0 uv54 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.972+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:275 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbae4046040 2024-09-19T11:23:43.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.973+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 65 ==== osd_op_reply(275 gc.4 [call] v33'55 uv55 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.973+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-09-19T11:23:43.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.973+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:276 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.974+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 97 ==== osd_op_reply(276 gc.5 [call] v33'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.974+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:277 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.974+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 98 ==== osd_op_reply(277 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.974+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.974+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:278 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x55c97df62190 2024-09-19T11:23:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.974+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 99 ==== osd_op_reply(278 gc.5 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.974+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:279 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.975+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 100 ==== osd_op_reply(279 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.975+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.975+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:280 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.975+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 101 ==== osd_op_reply(280 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.975+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.975+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:281 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df62190 2024-09-19T11:23:43.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.976+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 102 ==== osd_op_reply(281 gc.5 [call] v33'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.976+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-09-19T11:23:43.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.976+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:282 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.977+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 57 ==== osd_op_reply(282 gc.6 [call] v33'22 uv22 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.977+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:283 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 58 ==== osd_op_reply(283 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:284 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 59 ==== osd_op_reply(284 gc.6 [call out=24b] v0'0 uv22 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:285 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 60 ==== osd_op_reply(285 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:286 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 61 ==== osd_op_reply(286 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.978+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:287 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x55c97df5d780 2024-09-19T11:23:43.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.979+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 62 ==== osd_op_reply(287 gc.6 [call] v33'23 uv23 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.979+0000 7fbaec146800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-09-19T11:23:43.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.979+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:288 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.980+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 59 ==== osd_op_reply(288 gc.7 [call] v33'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.980+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:289 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 60 ==== osd_op_reply(289 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:290 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfae960 con 0x7fbadc046030 2024-09-19T11:23:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 61 ==== osd_op_reply(290 gc.7 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:291 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 62 ==== osd_op_reply(291 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:23:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:292 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 63 ==== osd_op_reply(292 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbaec146800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:23:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.981+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:293 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97dfaf970 con 0x7fbadc046030 2024-09-19T11:23:43.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.982+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 64 ==== osd_op_reply(293 gc.7 [call] v33'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.983+0000 7fb99b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:43.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.983+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94323890110736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df79ad0 con 0x55c97df5d780 2024-09-19T11:23:43.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.984+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 63 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 94323890110736] v33'167 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.984+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94323890095120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6c110 con 0x7fbae4046040 2024-09-19T11:23:43.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.985+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 66 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 94323890095120] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.985+0000 7fbaec146800 20 remove_watcher() i=0 2024-09-19T11:23:43.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.985+0000 7fbaec146800 2 removed watcher, disabling cache 2024-09-19T11:23:43.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.985+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94323890100768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df68410 con 0x55c97df5d780 2024-09-19T11:23:43.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.986+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 64 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 94323890100768] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.986+0000 7fbaec146800 20 remove_watcher() i=1 2024-09-19T11:23:43.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.986+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94323890104800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df68410 con 0x55c97df62190 2024-09-19T11:23:43.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.987+0000 7fbaea438640 1 -- 172.21.15.28:0/1974599229 <== osd.3 v2:172.21.15.28:6808/3027134945 103 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 94323890104800] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae400a4d0 con 0x55c97df62190 2024-09-19T11:23:43.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.987+0000 7fbaec146800 20 remove_watcher() i=2 2024-09-19T11:23:43.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.987+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94323890101824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6a9e0 con 0x7fbae4046040 2024-09-19T11:23:43.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.988+0000 7fbae9c37640 1 -- 172.21.15.28:0/1974599229 <== osd.2 v2:172.21.15.28:6800/2071226075 67 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 94323890101824] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbadc03a070 con 0x7fbae4046040 2024-09-19T11:23:43.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.988+0000 7fbaec146800 20 remove_watcher() i=3 2024-09-19T11:23:43.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.988+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94323890109168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df68410 con 0x55c97df5d780 2024-09-19T11:23:43.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.989+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.1 v2:172.21.15.28:6816/656119574 65 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 94323890109168] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x55c97df5d780 2024-09-19T11:23:43.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.989+0000 7fbaec146800 20 remove_watcher() i=4 2024-09-19T11:23:43.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.989+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94323890113232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df68410 con 0x7fbadc046030 2024-09-19T11:23:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.990+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 65 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 94323890113232] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.990+0000 7fbaec146800 20 remove_watcher() i=5 2024-09-19T11:23:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.990+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94323890115872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6cad0 con 0x7fbadc046030 2024-09-19T11:23:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.991+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 66 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 94323890115872] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.991+0000 7fbaec146800 20 remove_watcher() i=6 2024-09-19T11:23:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.991+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94323890119824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c97df6cad0 con 0x7fbadc046030 2024-09-19T11:23:43.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 <== osd.0 v2:172.21.15.22:6800/3144434255 67 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 94323890119824] v33'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbae0051070 con 0x7fbadc046030 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 20 remove_watcher() i=7 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbadc046030 msgr2=0x7fbadc026430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbadc046030 0x7fbadc026430 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97df5d780 msgr2=0x55c97df5fc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97df5d780 0x55c97df5fc40 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbae4046040 msgr2=0x7fbae4046810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbae4046040 0x7fbae4046810 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df62190 msgr2=0x55c97df64580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.992+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df62190 0x55c97df64580 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9b403d050 msgr2=0x7fb9b403f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9b403d050 0x7fb9b403f510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbadc01ac20 tx=0x7fbadc01bf90 comp rx=0 tx=0).stop 2024-09-19T11:23:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 msgr2=0x55c97df51e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 --2- 172.21.15.28:0/1974599229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97df4a8f0 0x55c97df51e60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbae4054800 tx=0x7fbae4054860 comp rx=0 tx=0).stop 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbae9436640 1 -- 172.21.15.28:0/1974599229 reap_dead start 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 shutdown_connections 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 >> 172.21.15.28:0/1974599229 conn(0x55c97df46cf0 msgr2=0x55c97df44f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 shutdown_connections 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/1974599229 wait complete. 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9c403d020 msgr2=0x7fb9c403f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9c403d020 0x7fb9c403f4e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x55c97df22d10 tx=0x7fbadc001440 comp rx=0 tx=0).stop 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97dd9f180 msgr2=0x55c97dd9d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97dd9f180 0x55c97dd9d9f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x55c97df21c40 tx=0x7fbae40167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 shutdown_connections 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9c403d020 0x7fb9c403f4e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 --2- 172.21.15.28:0/3124710886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97dd9f180 0x55c97dd9d9f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 >> 172.21.15.28:0/3124710886 conn(0x55c97df2ff70 msgr2=0x55c97df30350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 shutdown_connections 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.993+0000 7fbaec146800 1 -- 172.21.15.28:0/3124710886 wait complete. 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97de2ab50 msgr2=0x55c97df28290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97de2ab50 0x55c97df28290 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df2a630 msgr2=0x55c97df2caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df2a630 0x55c97df2caa0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9d003d000 msgr2=0x7fb9d003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9d003d000 0x7fb9d003f4c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbae0057000 tx=0x7fbae0052000 comp rx=0 tx=0).stop 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 msgr2=0x55c97df20c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df20c40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0012f0 tx=0x7fbadc000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 shutdown_connections 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c97df2a630 0x55c97df2caa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb9d003d000 0x7fb9d003f4c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c97de2ab50 0x55c97df28290 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 --2- 172.21.15.28:0/4027793003 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c97de29880 0x55c97df20c40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 >> 172.21.15.28:0/4027793003 conn(0x55c97de24c50 msgr2=0x55c97df1a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 shutdown_connections 2024-09-19T11:23:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:43.994+0000 7fbaec146800 1 -- 172.21.15.28:0/4027793003 wait complete. 2024-09-19T11:23:44.004 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-09-19T11:23:44.004 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-09-19T11:23:44.004 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-09-19T11:23:44.090 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:44.090 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:44.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f2542528800 1 Processor -- start 2024-09-19T11:23:44.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f2542528800 1 -- start start 2024-09-19T11:23:44.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f2542528800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5636f266d890 0x5636f266e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f2542528800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5636f266df40 con 0x5636f266e990 2024-09-19T11:23:44.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f2542528800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5636f24f87d0 con 0x5636f266d890 2024-09-19T11:23:44.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f253e24b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5636f266d890 0x5636f266e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f253da4a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5636f266e990 0x5636f26736f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55564/0 (socket says 172.21.15.28:55564) 2024-09-19T11:23:44.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.090+0000 7f253da4a640 1 -- 172.21.15.28:0/3073567543 learned_addr learned my addr 172.21.15.28:0/3073567543 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:44.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.091+0000 7f253e24b640 1 -- 172.21.15.28:0/3073567543 >> v1:172.21.15.22:6789/0 conn(0x5636f266e990 legacy=0x5636f26736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.091+0000 7f253e24b640 1 -- 172.21.15.28:0/3073567543 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f266ee40 con 0x5636f266d890 2024-09-19T11:23:44.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.092+0000 7f253e24b640 1 --2- 172.21.15.28:0/3073567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f266e450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f253000a470 tx=0x7f2530045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07b4f977d62ff0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.092+0000 7f253d249640 1 -- 172.21.15.28:0/3073567543 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2530053570 con 0x5636f266d890 2024-09-19T11:23:44.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f253d249640 1 -- 172.21.15.28:0/3073567543 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2530058070 con 0x5636f266d890 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f253d249640 1 -- 172.21.15.28:0/3073567543 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2530053880 con 0x5636f266d890 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f2542528800 1 -- 172.21.15.28:0/3073567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 msgr2=0x5636f266e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f2542528800 1 --2- 172.21.15.28:0/3073567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f266e450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f253000a470 tx=0x7f2530045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f2542528800 1 -- 172.21.15.28:0/3073567543 shutdown_connections 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f2542528800 1 --2- 172.21.15.28:0/3073567543 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f266e450 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f2542528800 1 -- 172.21.15.28:0/3073567543 >> 172.21.15.28:0/3073567543 conn(0x5636f2668c60 msgr2=0x5636f266b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f2542528800 1 -- 172.21.15.28:0/3073567543 shutdown_connections 2024-09-19T11:23:44.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.093+0000 7f2542528800 1 -- 172.21.15.28:0/3073567543 wait complete. 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.108+0000 7f2542528800 1 Processor -- start 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.108+0000 7f2542528800 1 -- start start 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.108+0000 7f2542528800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f25e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.108+0000 7f2542528800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5636f2505dd0 con 0x5636f266d890 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.108+0000 7f253e24b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f25e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.108+0000 7f253e24b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f25e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53206/0 (socket says 172.21.15.28:53206) 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.109+0000 7f253e24b640 1 -- 172.21.15.28:0/2850729542 learned_addr learned my addr 172.21.15.28:0/2850729542 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.109+0000 7f253e24b640 1 -- 172.21.15.28:0/2850729542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f25e3750 con 0x5636f266d890 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.109+0000 7f253e24b640 1 --2- 172.21.15.28:0/2850729542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f25e3160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2530025c20 tx=0x7f2530051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7288856477baba47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.109+0000 7f242c608640 1 -- 172.21.15.28:0/2850729542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f253005c580 con 0x5636f266d890 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.109+0000 7f242c608640 1 -- 172.21.15.28:0/2850729542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2530063030 con 0x5636f266d890 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.109+0000 7f242c608640 1 -- 172.21.15.28:0/2850729542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f253005c850 con 0x5636f266d890 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 -- 172.21.15.28:0/2850729542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 msgr2=0x5636f25e3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 --2- 172.21.15.28:0/2850729542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f25e3160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2530025c20 tx=0x7f2530051000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 -- 172.21.15.28:0/2850729542 shutdown_connections 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 --2- 172.21.15.28:0/2850729542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f25e3160 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 -- 172.21.15.28:0/2850729542 >> 172.21.15.28:0/2850729542 conn(0x5636f2668c60 msgr2=0x5636f2758c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 -- 172.21.15.28:0/2850729542 shutdown_connections 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 -- 172.21.15.28:0/2850729542 wait complete. 2024-09-19T11:23:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 Processor -- start 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.110+0000 7f2542528800 1 -- start start 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f2542528800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f2766f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f2542528800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5636f2674e10 con 0x5636f266d890 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f253e24b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f2766f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f253e24b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f2766f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53222/0 (socket says 172.21.15.28:53222) 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f253e24b640 1 -- 172.21.15.28:0/1733005441 learned_addr learned my addr 172.21.15.28:0/1733005441 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f253e24b640 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f27674b0 con 0x5636f266d890 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f253e24b640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f2766f50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2530045f50 tx=0x7f2530002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f242ae05640 1 -- 172.21.15.28:0/1733005441 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2530054ad0 con 0x5636f266d890 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f242ae05640 1 -- 172.21.15.28:0/1733005441 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2530054c70 con 0x5636f266d890 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f242ae05640 1 -- 172.21.15.28:0/1733005441 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2530068420 con 0x5636f266d890 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.111+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5636f2767740 con 0x5636f266d890 2024-09-19T11:23:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.112+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f2768140 con 0x5636f266d890 2024-09-19T11:23:44.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.112+0000 7f242ae05640 1 -- 172.21.15.28:0/1733005441 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f25300685c0 con 0x5636f266d890 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f242ae05640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f242403d000 0x7f242403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f242ae05640 1 -- 172.21.15.28:0/1733005441 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f2530070030 con 0x5636f266d890 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f253da4a640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f242403d000 0x7f242403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f266e990 0x5636f276e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f276eae0 con 0x5636f266e990 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f253ea4c640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f266e990 0x5636f276e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f253da4a640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f242403d000 0x7f242403f4c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2534001d70 tx=0x7f25340012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f253ea4c640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f266e990 0x5636f276e5a0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.113+0000 7f253ea4c640 1 -- 172.21.15.28:0/1733005441 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f266e990 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.114+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f276ee80 con 0x5636f266e990 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.114+0000 7f253ea4c640 1 -- 172.21.15.28:0/1733005441 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f266e990 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.114+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f2770940 0x5636f2772de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.114+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f2773570 con 0x5636f2770940 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.114+0000 7f253e24b640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f2770940 0x5636f2772de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.114+0000 7f253e24b640 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f2770940 0x5636f2772de0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.115+0000 7f253e24b640 1 -- 172.21.15.28:0/1733005441 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2530051070 con 0x5636f2770940 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.115+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f276f6d0 con 0x5636f266e990 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.115+0000 7f253ea4c640 1 -- 172.21.15.28:0/1733005441 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f266e990 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.115+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f276fa70 con 0x5636f2770940 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.115+0000 7f253e24b640 1 -- 172.21.15.28:0/1733005441 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2530051070 con 0x5636f2770940 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.115+0000 7f2542528800 1 Processor -- start 2024-09-19T11:23:44.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f2542528800 1 -- start start 2024-09-19T11:23:44.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f2542528800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f2781ed0 0x5636f27822f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f2542528800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5636f25644a0 con 0x5636f2781ed0 2024-09-19T11:23:44.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f253ea4c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f2781ed0 0x5636f27822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f253ea4c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f2781ed0 0x5636f27822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53226/0 (socket says 172.21.15.28:53226) 2024-09-19T11:23:44.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f253ea4c640 1 -- 172.21.15.28:0/646160882 learned_addr learned my addr 172.21.15.28:0/646160882 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f253ea4c640 1 -- 172.21.15.28:0/646160882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f2782930 con 0x5636f2781ed0 2024-09-19T11:23:44.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.116+0000 7f253ea4c640 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f2781ed0 0x5636f27822f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x5636f2768310 tx=0x7f25380167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.117+0000 7f2423fff640 1 -- 172.21.15.28:0/646160882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2538013640 con 0x5636f2781ed0 2024-09-19T11:23:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.117+0000 7f2423fff640 1 -- 172.21.15.28:0/646160882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25380137e0 con 0x5636f2781ed0 2024-09-19T11:23:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.117+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5636f2783160 con 0x5636f2781ed0 2024-09-19T11:23:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.117+0000 7f2423fff640 1 -- 172.21.15.28:0/646160882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2538013b00 con 0x5636f2781ed0 2024-09-19T11:23:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.117+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f2783700 con 0x5636f2781ed0 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f2423fff640 1 -- 172.21.15.28:0/646160882 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2538011040 con 0x5636f2781ed0 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f2423fff640 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f241803d000 0x7f241803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f2423fff640 1 -- 172.21.15.28:0/646160882 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f253800e040 con 0x5636f2781ed0 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f253e24b640 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f241803d000 0x7f241803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f2542528800 1 Processor -- start 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f253e24b640 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f241803d000 0x7f241803f4c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5636f2769050 tx=0x7f2530068910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f2542528800 1 -- start start 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f2542528800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f278ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.118+0000 7f2542528800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5636f25e1ba0 con 0x5636f278e950 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f253ea4c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f278ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f253ea4c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f278ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53238/0 (socket says 172.21.15.28:53238) 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f253ea4c640 1 -- 172.21.15.28:0/1836017641 learned_addr learned my addr 172.21.15.28:0/1836017641 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f253ea4c640 1 -- 172.21.15.28:0/1836017641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f278f340 con 0x5636f278e950 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f253ea4c640 1 --2- 172.21.15.28:0/1836017641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f278ed50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f253800afe0 tx=0x7f2538020f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137cddf507acdd76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f2421ffb640 1 -- 172.21.15.28:0/1836017641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2538013ca0 con 0x5636f278e950 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f2421ffb640 1 -- 172.21.15.28:0/1836017641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2538052070 con 0x5636f278e950 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.119+0000 7f2421ffb640 1 -- 172.21.15.28:0/1836017641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f253802a3f0 con 0x5636f278e950 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 -- 172.21.15.28:0/1836017641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 msgr2=0x5636f278ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 --2- 172.21.15.28:0/1836017641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f278ed50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f253800afe0 tx=0x7f2538020f00 comp rx=0 tx=0).stop 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 -- 172.21.15.28:0/1836017641 shutdown_connections 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 --2- 172.21.15.28:0/1836017641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f278ed50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 -- 172.21.15.28:0/1836017641 >> 172.21.15.28:0/1836017641 conn(0x5636f278ad50 msgr2=0x5636f2788f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 -- 172.21.15.28:0/1836017641 shutdown_connections 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 -- 172.21.15.28:0/1836017641 wait complete. 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 Processor -- start 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 -- start start 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f2795f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.120+0000 7f2542528800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5636f278f960 con 0x5636f278e950 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f253ea4c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f2795f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f253ea4c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f2795f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53248/0 (socket says 172.21.15.28:53248) 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 learned_addr learned my addr 172.21.15.28:0/4068048275 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f2796460 con 0x5636f278e950 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f253ea4c640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f2795f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2538020f40 tx=0x7f25380547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f240ffff640 1 -- 172.21.15.28:0/4068048275 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2538045c00 con 0x5636f278e950 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f240ffff640 1 -- 172.21.15.28:0/4068048275 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253804de50 con 0x5636f278e950 2024-09-19T11:23:44.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5636f27966f0 con 0x5636f278e950 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f240ffff640 1 -- 172.21.15.28:0/4068048275 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2538054960 con 0x5636f278e950 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.121+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f2797280 con 0x5636f278e950 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.122+0000 7f240ffff640 1 -- 172.21.15.28:0/4068048275 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2538054b00 con 0x5636f278e950 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.122+0000 7f240ffff640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f240803d000 0x7f240803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.122+0000 7f240ffff640 1 -- 172.21.15.28:0/4068048275 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f2538004830 con 0x5636f278e950 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.122+0000 7f253e24b640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f240803d000 0x7f240803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.122+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.122+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f27a1800 0x5636f27a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.122+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a4410 con 0x5636f27a1800 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f253da4a640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f27a1800 0x5636f27a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f253e24b640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f240803d000 0x7f240803f4c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2530000c00 tx=0x7f253008e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f253da4a640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f27a1800 0x5636f27a3cc0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2534009050 con 0x5636f27a1800 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 20 realm 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a0ba0 con 0x5636f27a1800 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2534009050 con 0x5636f27a1800 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.123+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a4ef0 con 0x5636f27a1800 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2534009050 con 0x5636f27a1800 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f2542528800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a57c0 con 0x5636f27a1800 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2534009050 con 0x5636f27a1800 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f2542528800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f27a6100 0x5636f27a84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a8c30 con 0x5636f27a6100 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f253ea4c640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f27a6100 0x5636f27a84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.124+0000 7f253ea4c640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f27a6100 0x5636f27a84f0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f2542528800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f2542528800 20 searching for the correct realm 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f253ea4c640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f253803c3f0 0x7f25380451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f253e24b640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f253803c3f0 0x7f25380451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f253e24b640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f253803c3f0 0x7f25380451d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.125+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.126+0000 7f253e24b640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f253005c070 0x7f25300980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.126+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253005c070 2024-09-19T11:23:44.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.126+0000 7f253da4a640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f253005c070 0x7f25300980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.126+0000 7f253da4a640 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f253005c070 0x7f25300980b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.126+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534009050 con 0x7f253005c070 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.126+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534009050 con 0x5636f27a1800 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253005c070 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253005c070 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.127+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253005c070 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.128+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.129+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.130+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253005c070 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.131+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.131+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.131+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.131+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.131+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.131+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253005c070 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253005c070 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.132+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.133+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.133+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a6100 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.133+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.133+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.133+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.133+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x5636f27a1800 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa5e0 con 0x7f253803c3f0 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:44.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a96b0 con 0x5636f27a1800 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a92f0 con 0x5636f27a1800 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.134+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aa010 con 0x5636f27a6100 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 zone default found 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 4 Realm: () 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 zonegroup default 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27a9b80 con 0x7f253005c070 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ace50 con 0x5636f27a6100 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.135+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ad1f0 con 0x5636f27a6100 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.136+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.136+0000 7f2542528800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.136+0000 7f2542528800 20 started sync module instance, tier type = 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.136+0000 7f2542528800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:44.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.136+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ac290 con 0x7f253803c3f0 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.137+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.137+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94793996352992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ad620 con 0x7f253803c3f0 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.137+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ade70 con 0x5636f27a1800 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.138+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94793996352992] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.138+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.138+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94793996360128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ae290 con 0x5636f27a1800 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.138+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aec50 con 0x5636f27a6100 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.139+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94793996360128] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.139+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.139+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94793996361664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aec50 con 0x5636f27a6100 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.139+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27af610 con 0x7f253803c3f0 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.140+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94793996361664] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.140+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.140+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94793996363680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27aebc0 con 0x7f253803c3f0 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.141+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b04d0 con 0x5636f27a1800 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.141+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.141+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94793996370464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b0d70 con 0x5636f27a1800 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.141+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b1840 con 0x7f253005c070 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.142+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94793996363680] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.142+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.142+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94793996370464] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.143+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94793996374832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b1800 con 0x7f253005c070 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.143+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b08c0 con 0x7f253005c070 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.143+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94793996374832] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.144+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.144+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94793996372160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b1dd0 con 0x7f253005c070 2024-09-19T11:23:44.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.144+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b2790 con 0x7f253005c070 2024-09-19T11:23:44.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94793996372160] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94793996368384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b2790 con 0x7f253005c070 2024-09-19T11:23:44.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=0 2024-09-19T11:23:44.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=1 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=2 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=3 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=4 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=5 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=6 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94793996368384] v33'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 20 add_watcher() i=7 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.145+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b5ae0 con 0x5636f27a1800 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.146+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.146+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94793996379216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b2d20 con 0x5636f27a1800 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.147+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94793996379216] v33'168 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.147+0000 7f23f4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.147+0000 7f2542528800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.147+0000 7f2542528800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.147+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27be040 con 0x5636f27a1800 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.148+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.148+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27be630 con 0x7f253005c070 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.149+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.149+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bea50 con 0x5636f27a6100 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.150+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.150+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27be8a0 con 0x5636f27a1800 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.151+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'169 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.151+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bee30 con 0x7f253803c3f0 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.152+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'56 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.152+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bf330 con 0x5636f27a6100 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.153+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'37 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.153+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bf330 con 0x5636f27a1800 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.154+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.154+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bf330 con 0x7f253005c070 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.155+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'37 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.155+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x7f253803c3f0 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.156+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'23 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.156+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x5636f27a6100 2024-09-19T11:23:44.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.157+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.157+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x7f253803c3f0 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.158+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'57 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.158+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x7f253005c070 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.160+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.160+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x7f253803c3f0 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.160+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.160+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x7f253005c070 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.162+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'44 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.162+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x5636f27a1800 2024-09-19T11:23:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.163+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'170 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.163+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bc210 con 0x5636f27a6100 2024-09-19T11:23:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.164+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'50 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.164+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x7f253803c3f0 2024-09-19T11:23:44.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.165+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'58 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.165+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.166+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.166+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a1800 2024-09-19T11:23:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.167+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'30 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.167+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a1800 2024-09-19T11:23:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.168+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'171 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.168+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.169+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'51 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.169+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x7f253803c3f0 2024-09-19T11:23:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.170+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'32 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.170+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.171+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.171+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.172+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.172+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.173+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'18 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.173+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.174+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'38 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.174+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x7f253005c070 2024-09-19T11:23:44.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.175+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'26 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.175+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.176+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.176+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x7f253005c070 2024-09-19T11:23:44.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.177+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'38 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.177+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x5636f27a6100 2024-09-19T11:23:44.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.178+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'52 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.178+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x7f253005c070 2024-09-19T11:23:44.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.179+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'45 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.180+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27bfe40 con 0x7f253803c3f0 2024-09-19T11:23:44.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f236f7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f2542528800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f2542528800 20 Filter name: none 2024-09-19T11:23:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f236d7fa640 20 reqs_thread_entry: start 2024-09-19T11:23:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.181+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a1800 2024-09-19T11:23:44.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.182+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.182+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.182+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.182+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.182+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.182+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253005c070 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.183+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv166 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a1800 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv166 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv166 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.184+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv55 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253803c3f0 2024-09-19T11:23:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv55 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv55 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.185+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv36 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a1800 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv23 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.186+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.187+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.187+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.187+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.187+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253005c070 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv36 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253803c3f0 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv22 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.188+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.189+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.189+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.189+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.189+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.189+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.189+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253803c3f0 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv51 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.190+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253005c070 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.191+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253803c3f0 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.192+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.192+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.192+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.192+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.192+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.192+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253005c070 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv41 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv162 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a1800 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv162 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.193+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.194+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv162 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.194+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.194+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.194+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.194+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 38 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv45 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.194+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.194+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 39 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253803c3f0 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv53 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 40 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.195+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 41 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 42 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a1800 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.196+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv164 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a1800 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv164 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a1800 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv164 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.197+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 43 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 44 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv47 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 45 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.198+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253803c3f0 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 36 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv31 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 37 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 46 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 47 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.199+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 48 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 49 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 50 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 51 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.200+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 52 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 53 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv17 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 54 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 55 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.201+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.202+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 56 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.202+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.202+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 57 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.202+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.202+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv25 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.202+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.203+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv25 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.203+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.203+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv25 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.203+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 58 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 59 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 60 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.204+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253005c070 2024-09-19T11:23:44.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 61 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x5636f27a6100 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 62 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv49 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.205+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x5636f27a6100 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.206+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 63 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.206+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.206+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv43 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.206+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253005c070 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.206+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv43 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.206+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253005c070 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.207+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv43 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.207+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.207+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 38 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.207+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27e9e80 con 0x7f253803c3f0 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.207+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 39 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.207+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5636f27eae90 con 0x7f253803c3f0 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.207+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 40 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.208+0000 7f23f4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.208+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94793996379216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b41f0 con 0x5636f27a1800 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.209+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 94793996379216] v33'172 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.209+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94793996352992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b2450 con 0x7f253803c3f0 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.210+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 41 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 94793996352992] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.210+0000 7f2542528800 20 remove_watcher() i=0 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.210+0000 7f2542528800 2 removed watcher, disabling cache 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.210+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94793996360128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b2450 con 0x5636f27a1800 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.212+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 94793996360128] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.212+0000 7f2542528800 20 remove_watcher() i=1 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.212+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94793996361664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ad9c0 con 0x5636f27a6100 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.213+0000 7f253ea4c640 1 -- 172.21.15.28:0/4068048275 <== osd.3 v2:172.21.15.28:6808/3027134945 64 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 94793996361664] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253800a4d0 con 0x5636f27a6100 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.213+0000 7f2542528800 20 remove_watcher() i=2 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.213+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94793996363680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b2450 con 0x7f253803c3f0 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.214+0000 7f253e24b640 1 -- 172.21.15.28:0/4068048275 <== osd.2 v2:172.21.15.28:6800/2071226075 42 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 94793996363680] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25300a1030 con 0x7f253803c3f0 2024-09-19T11:23:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.214+0000 7f2542528800 20 remove_watcher() i=3 2024-09-19T11:23:44.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.214+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94793996370464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27adfc0 con 0x5636f27a1800 2024-09-19T11:23:44.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.215+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.1 v2:172.21.15.28:6816/656119574 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 94793996370464] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x5636f27a1800 2024-09-19T11:23:44.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.215+0000 7f2542528800 20 remove_watcher() i=4 2024-09-19T11:23:44.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.215+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94793996374832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ad9c0 con 0x7f253005c070 2024-09-19T11:23:44.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.216+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 94793996374832] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.216+0000 7f2542528800 20 remove_watcher() i=5 2024-09-19T11:23:44.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.216+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94793996372160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27ae7a0 con 0x7f253005c070 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.217+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 94793996372160] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.217+0000 7f2542528800 20 remove_watcher() i=6 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.217+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94793996368384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5636f27b2450 con 0x7f253005c070 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.218+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 94793996368384] v33'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534029040 con 0x7f253005c070 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.218+0000 7f2542528800 20 remove_watcher() i=7 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f253005c070 msgr2=0x7f25300980b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f253005c070 0x7f25300980b0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f27a1800 msgr2=0x5636f27a3cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f27a1800 0x5636f27a3cc0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f253803c3f0 msgr2=0x7f25380451d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f253803c3f0 0x7f25380451d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f27a6100 msgr2=0x5636f27a84f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f27a6100 0x5636f27a84f0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f240803d000 msgr2=0x7f240803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f240803d000 0x7f240803f4c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2530000c00 tx=0x7f253008e000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 msgr2=0x5636f2795f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/4068048275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f278e950 0x5636f2795f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2538020f40 tx=0x7f25380547b0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 reap_dead start 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f253da4a640 1 -- 172.21.15.28:0/4068048275 reap_dead start 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 shutdown_connections 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 >> 172.21.15.28:0/4068048275 conn(0x5636f278ad50 msgr2=0x5636f2788f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 shutdown_connections 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/4068048275 wait complete. 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f241803d000 msgr2=0x7f241803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f241803d000 0x7f241803f4c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5636f2769050 tx=0x7f2530068910 comp rx=0 tx=0).stop 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f2781ed0 msgr2=0x5636f27822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f2781ed0 0x5636f27822f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x5636f2768310 tx=0x7f25380167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 shutdown_connections 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f241803d000 0x7f241803f4c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 --2- 172.21.15.28:0/646160882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f2781ed0 0x5636f27822f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 >> 172.21.15.28:0/646160882 conn(0x5636f2776280 msgr2=0x5636f2776660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 shutdown_connections 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.219+0000 7f2542528800 1 -- 172.21.15.28:0/646160882 wait complete. 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f266e990 msgr2=0x5636f276e5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f266e990 0x5636f276e5a0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f2770940 msgr2=0x5636f2772de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f2770940 0x5636f2772de0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f242403d000 msgr2=0x7f242403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f242403d000 0x7f242403f4c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2534001d70 tx=0x7f25340012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 msgr2=0x5636f2766f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f2766f50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2530045f50 tx=0x7f2530002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 shutdown_connections 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5636f2770940 0x5636f2772de0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f242403d000 0x7f242403f4c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5636f266e990 0x5636f276e5a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 --2- 172.21.15.28:0/1733005441 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5636f266d890 0x5636f2766f50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 >> 172.21.15.28:0/1733005441 conn(0x5636f2668c60 msgr2=0x5636f27587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 shutdown_connections 2024-09-19T11:23:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.220+0000 7f2542528800 1 -- 172.21.15.28:0/1733005441 wait complete. 2024-09-19T11:23:44.230 INFO:teuthology.orchestra.run.smithi028.stdout:[] 2024-09-19T11:23:44.230 DEBUG:tasks.util.rgw: json result: [] 2024-09-19T11:23:44.230 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-09-19T11:23:44.230 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-09-19T11:23:44.231 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-09-19T11:23:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.276+0000 7fb5b5f28800 1 Processor -- start 2024-09-19T11:23:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.276+0000 7fb5b5f28800 1 -- start start 2024-09-19T11:23:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.276+0000 7fb5b5f28800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562d1cf00ea0 0x562d1cf03290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.276+0000 7fb5b5f28800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562d1cf075e0 con 0x562d1cf038c0 2024-09-19T11:23:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.276+0000 7fb5b5f28800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x562d1cd8b7d0 con 0x562d1cf00ea0 2024-09-19T11:23:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.277+0000 7fb5b1c6f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562d1cf00ea0 0x562d1cf03290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.277+0000 7fb5b146e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x562d1cf038c0 0x562d1cf05d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55574/0 (socket says 172.21.15.28:55574) 2024-09-19T11:23:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.277+0000 7fb5b146e640 1 -- 172.21.15.28:0/3227650013 learned_addr learned my addr 172.21.15.28:0/3227650013 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.277+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3227650013 >> v1:172.21.15.22:6789/0 conn(0x562d1cf038c0 legacy=0x562d1cf05d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.277+0000 7fb5b0c6d640 1 -- 172.21.15.28:0/3227650013 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1922115373 0 0) 0x562d1cf075e0 con 0x562d1cf038c0 2024-09-19T11:23:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.277+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3227650013 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1cf08810 con 0x562d1cf00ea0 2024-09-19T11:23:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.279+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/3227650013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cf03290 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400a470 tx=0x7fb5a4045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3e6c07115495af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.280+0000 7fb5b0c6d640 1 -- 172.21.15.28:0/3227650013 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a4053570 con 0x562d1cf00ea0 2024-09-19T11:23:44.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.280+0000 7fb5b0c6d640 1 -- 172.21.15.28:0/3227650013 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a4058070 con 0x562d1cf00ea0 2024-09-19T11:23:44.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.280+0000 7fb5b0c6d640 1 -- 172.21.15.28:0/3227650013 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a4053880 con 0x562d1cf00ea0 2024-09-19T11:23:44.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.281+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3227650013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 msgr2=0x562d1cf03290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.281+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3227650013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cf03290 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400a470 tx=0x7fb5a4045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:44.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.281+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3227650013 shutdown_connections 2024-09-19T11:23:44.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.281+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3227650013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cf03290 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.281+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3227650013 >> 172.21.15.28:0/3227650013 conn(0x562d1cefaa60 msgr2=0x562d1cefcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.281+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3227650013 shutdown_connections 2024-09-19T11:23:44.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.281+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3227650013 wait complete. 2024-09-19T11:23:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.296+0000 7fb5b5f28800 1 Processor -- start 2024-09-19T11:23:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.296+0000 7fb5b5f28800 1 -- start start 2024-09-19T11:23:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.296+0000 7fb5b5f28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.296+0000 7fb5b5f28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d1cd98dd0 con 0x562d1cf00ea0 2024-09-19T11:23:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.296+0000 7fb5b1c6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.296+0000 7fb5b1c6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53266/0 (socket says 172.21.15.28:53266) 2024-09-19T11:23:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.296+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/1442225008 learned_addr learned my addr 172.21.15.28:0/1442225008 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.297+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/1442225008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1cff0770 con 0x562d1cf00ea0 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.297+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/1442225008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff0180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4005870 tx=0x7fb5a4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf9bf9b774169a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.297+0000 7fb49fe07640 1 -- 172.21.15.28:0/1442225008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a405c580 con 0x562d1cf00ea0 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.297+0000 7fb49fe07640 1 -- 172.21.15.28:0/1442225008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a4063030 con 0x562d1cf00ea0 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.297+0000 7fb49fe07640 1 -- 172.21.15.28:0/1442225008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a405c850 con 0x562d1cf00ea0 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 -- 172.21.15.28:0/1442225008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 msgr2=0x562d1cff0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/1442225008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff0180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4005870 tx=0x7fb5a4051000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 -- 172.21.15.28:0/1442225008 shutdown_connections 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/1442225008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff0180 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 -- 172.21.15.28:0/1442225008 >> 172.21.15.28:0/1442225008 conn(0x562d1cefaa60 msgr2=0x562d1cfebb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 -- 172.21.15.28:0/1442225008 shutdown_connections 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 -- 172.21.15.28:0/1442225008 wait complete. 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 Processor -- start 2024-09-19T11:23:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 -- start start 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b5f28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d1cf07ab0 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b1c6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b1c6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53280/0 (socket says 172.21.15.28:53280) 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.298+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/90839064 learned_addr learned my addr 172.21.15.28:0/90839064 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.299+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1cff8110 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.299+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff7bb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4045f50 tx=0x7fb5a4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.299+0000 7fb49e604640 1 -- 172.21.15.28:0/90839064 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a4078070 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.299+0000 7fb49e604640 1 -- 172.21.15.28:0/90839064 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a4054a70 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.299+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d1cff83a0 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.299+0000 7fb49e604640 1 -- 172.21.15.28:0/90839064 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a4054d50 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.299+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1cff8fb0 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.300+0000 7fb49e604640 1 -- 172.21.15.28:0/90839064 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fb5a407b070 con 0x562d1cf00ea0 2024-09-19T11:23:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.300+0000 7fb49e604640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb49803d000 0x7fb49803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.300+0000 7fb49e604640 1 -- 172.21.15.28:0/90839064 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fb5a4070070 con 0x562d1cf00ea0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.300+0000 7fb5b146e640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb49803d000 0x7fb49803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.300+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1cf038c0 0x562d1cfff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.300+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1cfff740 con 0x562d1cf038c0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.300+0000 7fb5b2470640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1cf038c0 0x562d1cfff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.301+0000 7fb5b146e640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb49803d000 0x7fb49803f4c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a80013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.301+0000 7fb5b2470640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1cf038c0 0x562d1cfff200 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.301+0000 7fb5b2470640 1 -- 172.21.15.28:0/90839064 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1cf038c0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.301+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1cfffae0 con 0x562d1cf038c0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.301+0000 7fb5b2470640 1 -- 172.21.15.28:0/90839064 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1cf038c0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.301+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d0015e0 0x562d1d003a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.301+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0041f0 con 0x562d1d0015e0 2024-09-19T11:23:44.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.302+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d0015e0 0x562d1d003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.302+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d0015e0 0x562d1d003a60 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.302+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/90839064 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fb5a407a030 con 0x562d1d0015e0 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.302+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d000330 con 0x562d1cf038c0 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.302+0000 7fb5b2470640 1 -- 172.21.15.28:0/90839064 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1cf038c0 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.302+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d000780 con 0x562d1d0015e0 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.303+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/90839064 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fb5a407a030 con 0x562d1d0015e0 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.303+0000 7fb5b5f28800 1 Processor -- start 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.303+0000 7fb5b5f28800 1 -- start start 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.303+0000 7fb5b5f28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d012b80 0x562d1d012fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.303+0000 7fb5b5f28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d1cdf74a0 con 0x562d1d012b80 2024-09-19T11:23:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.303+0000 7fb5b2470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d012b80 0x562d1d012fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb5b2470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d012b80 0x562d1d012fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53290/0 (socket says 172.21.15.28:53290) 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb5b2470640 1 -- 172.21.15.28:0/525142466 learned_addr learned my addr 172.21.15.28:0/525142466 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb5b2470640 1 -- 172.21.15.28:0/525142466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1d0135e0 con 0x562d1d012b80 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb5b2470640 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d012b80 0x562d1d012fa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x562d1cff91d0 tx=0x7fb5ac0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb4977fe640 1 -- 172.21.15.28:0/525142466 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac013640 con 0x562d1d012b80 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb4977fe640 1 -- 172.21.15.28:0/525142466 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ac0137e0 con 0x562d1d012b80 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb4977fe640 1 -- 172.21.15.28:0/525142466 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac013ae0 con 0x562d1d012b80 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d1d013e10 con 0x562d1d012b80 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.304+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1d0143b0 con 0x562d1d012b80 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.305+0000 7fb4977fe640 1 -- 172.21.15.28:0/525142466 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fb5ac011040 con 0x562d1d012b80 2024-09-19T11:23:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.305+0000 7fb4977fe640 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb48c03d000 0x7fb48c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.305+0000 7fb4977fe640 1 -- 172.21.15.28:0/525142466 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fb5ac00e040 con 0x562d1d012b80 2024-09-19T11:23:44.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.305+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb48c03d000 0x7fb48c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b5f28800 1 Processor -- start 2024-09-19T11:23:44.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb48c03d000 0x7fb48c03f4c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4096760 tx=0x7fb5a4054ef0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b5f28800 1 -- start start 2024-09-19T11:23:44.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b5f28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d01fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b5f28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d1cff0b40 con 0x562d1d01f620 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b2470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d01fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b2470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d01fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53292/0 (socket says 172.21.15.28:53292) 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b2470640 1 -- 172.21.15.28:0/3670078854 learned_addr learned my addr 172.21.15.28:0/3670078854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.306+0000 7fb5b2470640 1 -- 172.21.15.28:0/3670078854 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1d01fff0 con 0x562d1d01f620 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b2470640 1 --2- 172.21.15.28:0/3670078854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d01fa00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0547f0 tx=0x7fb5ac01fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41c029dc5731625 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb4957fa640 1 -- 172.21.15.28:0/3670078854 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac051070 con 0x562d1d01f620 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb4957fa640 1 -- 172.21.15.28:0/3670078854 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ac053440 con 0x562d1d01f620 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb4957fa640 1 -- 172.21.15.28:0/3670078854 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac021ad0 con 0x562d1d01f620 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3670078854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 msgr2=0x562d1d01fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3670078854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d01fa00 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0547f0 tx=0x7fb5ac01fe70 comp rx=0 tx=0).stop 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3670078854 shutdown_connections 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3670078854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d01fa00 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3670078854 >> 172.21.15.28:0/3670078854 conn(0x562d1d01ba20 msgr2=0x562d1d019c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3670078854 shutdown_connections 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3670078854 wait complete. 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.307+0000 7fb5b5f28800 1 Processor -- start 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b5f28800 1 -- start start 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b5f28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d026c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b5f28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562d1d020630 con 0x562d1d01f620 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b2470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d026c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b2470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d026c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53298/0 (socket says 172.21.15.28:53298) 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 learned_addr learned my addr 172.21.15.28:0/3487016268 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d1d0271a0 con 0x562d1d01f620 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb5b2470640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d026c40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac022ff0 tx=0x7fb5ac01fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.308+0000 7fb4837fe640 1 -- 172.21.15.28:0/3487016268 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac021440 con 0x562d1d01f620 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.309+0000 7fb4837fe640 1 -- 172.21.15.28:0/3487016268 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ac021620 con 0x562d1d01f620 2024-09-19T11:23:44.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.309+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d1d027430 con 0x562d1d01f620 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.309+0000 7fb4837fe640 1 -- 172.21.15.28:0/3487016268 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac045b30 con 0x562d1d01f620 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.309+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1d027430 con 0x562d1d01f620 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.309+0000 7fb4837fe640 1 -- 172.21.15.28:0/3487016268 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fb5ac045cd0 con 0x562d1d01f620 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb4837fe640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb47c03d000 0x7fb47c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb4837fe640 1 -- 172.21.15.28:0/3487016268 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fb5ac034d50 con 0x562d1d01f620 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb47c03d000 0x7fb47c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb47c03d000 0x7fb47c03f4c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4098390 tx=0x7fb5a4091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1d032340 0x562d1d034800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d034f50 con 0x562d1d032340 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.310+0000 7fb5b146e640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1d032340 0x562d1d034800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b146e640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1d032340 0x562d1d034800 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5a800f5f0 con 0x562d1d032340 2024-09-19T11:23:44.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 20 realm 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d031690 con 0x562d1d032340 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5a800f5f0 con 0x562d1d032340 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d035a40 con 0x562d1d032340 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5a800f5f0 con 0x562d1d032340 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.311+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d036310 con 0x562d1d032340 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb5a800f5f0 con 0x562d1d032340 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b5f28800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d036d30 0x562d1d039120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d039890 con 0x562d1d036d30 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b2470640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d036d30 0x562d1d039120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b2470640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d036d30 0x562d1d039120 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b5f28800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b5f28800 20 searching for the correct realm 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.312+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b2470640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fb5ac045130 0x7fb5ac047960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fb5ac045130 0x7fb5ac047960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fb5ac045130 0x7fb5ac047960 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b1c6f640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fb5a40a6040 0x7fb5a4009410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5a40a6040 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.313+0000 7fb5b146e640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fb5a40a6040 0x7fb5a4009410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.314+0000 7fb5b146e640 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fb5a40a6040 0x7fb5a4009410 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.314+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a800f5f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.314+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.314+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a800f5f0 con 0x562d1d032340 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.314+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.314+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.314+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5a40a6040 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5a40a6040 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.315+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.316+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.316+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5a40a6040 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.316+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.316+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.316+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.316+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.316+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.317+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5a40a6040 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.318+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5a40a6040 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.319+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.320+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.320+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5a40a6040 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.320+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a802b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.320+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.320+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.320+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d036d30 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x562d1d032340 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b1c0 con 0x7fb5ac045130 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.321+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03a250 con 0x562d1d032340 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03aaf0 con 0x562d1d032340 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb5a802b040 con 0x562d1d032340 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b640 con 0x562d1d036d30 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 zone default found 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 4 Realm: () 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 zonegroup default 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.322+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03b9e0 con 0x7fb5a40a6040 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fb5a802b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03dca0 con 0x562d1d036d30 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03e040 con 0x562d1d036d30 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 20 started sync module instance, tier type = 2024-09-19T11:23:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.323+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03d130 con 0x7fb5ac045130 2024-09-19T11:23:44.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.324+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.324+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94751760307536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03e900 con 0x7fb5ac045130 2024-09-19T11:23:44.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.325+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03f150 con 0x562d1d032340 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.326+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94751760307536] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.326+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.326+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94751760311376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03f150 con 0x562d1d032340 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.326+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03fc70 con 0x562d1d036d30 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.327+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94751760311376] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.327+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.327+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94751760316224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03f150 con 0x562d1d036d30 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.327+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0406c0 con 0x7fb5ac045130 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.328+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94751760316224] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.328+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.328+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94751760313504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03f0c0 con 0x7fb5ac045130 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.328+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d041580 con 0x562d1d032340 2024-09-19T11:23:44.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.329+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.329+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94751760322768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0419a0 con 0x562d1d032340 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.329+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0424a0 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.329+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94751760313504] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.330+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94751760322768] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.330+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.330+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94751760326544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0414f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.330+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d042d70 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.331+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94751760326544] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.331+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.331+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94751760321200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0414f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.331+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d043360 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94751760321200] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94751760330288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d043320 con 0x7fb5a40a6040 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 20 add_watcher() i=0 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 20 add_watcher() i=1 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 20 add_watcher() i=2 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 20 add_watcher() i=3 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 20 add_watcher() i=4 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 20 add_watcher() i=5 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.332+0000 7fb5b5f28800 20 add_watcher() i=6 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.333+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94751760330288] v33'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.333+0000 7fb5b5f28800 20 add_watcher() i=7 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.333+0000 7fb5b5f28800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.333+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d046780 con 0x562d1d032340 2024-09-19T11:23:44.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.333+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.334+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94751760328624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d043cd0 con 0x562d1d032340 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.334+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94751760328624] v33'173 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.335+0000 7fb4637fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.335+0000 7fb5b5f28800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.335+0000 7fb5b5f28800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.335+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d04efe0 con 0x562d1d032340 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.336+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.336+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d04f670 con 0x7fb5a40a6040 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.337+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.337+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d04fab0 con 0x562d1d036d30 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.338+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.338+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d04fa40 con 0x562d1d032340 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.339+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'174 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.339+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d04ffb0 con 0x7fb5ac045130 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.340+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'59 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.340+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d050520 con 0x562d1d036d30 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.341+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'39 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.341+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d050920 con 0x562d1d032340 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.342+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.342+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d050700 con 0x7fb5a40a6040 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.343+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'39 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.343+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d050700 con 0x7fb5ac045130 2024-09-19T11:23:44.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.344+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'24 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.344+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d050180 con 0x562d1d036d30 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.345+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.345+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5ac045130 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.346+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'60 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.346+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.347+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.347+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5ac045130 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.348+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.348+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.350+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'46 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.350+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d032340 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.351+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'175 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.351+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.352+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'53 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.352+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5ac045130 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.353+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'61 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.353+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.354+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.354+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d032340 2024-09-19T11:23:44.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.355+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'31 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.355+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d032340 2024-09-19T11:23:44.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.356+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'176 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.356+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.357+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'54 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.357+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5ac045130 2024-09-19T11:23:44.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.358+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'33 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.358+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.359+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.359+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.360+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.360+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.361+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'19 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.361+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.362+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'40 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.362+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.363+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'27 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.363+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x562d1d036d30 2024-09-19T11:23:44.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.364+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.364+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.366+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'40 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.366+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d050a30 con 0x562d1d036d30 2024-09-19T11:23:44.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.367+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'55 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.367+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0509f0 con 0x7fb5a40a6040 2024-09-19T11:23:44.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.368+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'47 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.368+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d050a30 con 0x7fb5ac045130 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb4627fc640 20 reqs_thread_entry: start 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb5b5f28800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb5b5f28800 20 Filter name: none 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb3d3fff640 20 reqs_thread_entry: start 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb5b5f28800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb5b5f28800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.369+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d07b600 con 0x562d1d032340 2024-09-19T11:23:44.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.370+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.370+0000 7fb5b5f28800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.370+0000 7fb5b5f28800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.370+0000 7fb5b5f28800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.370+0000 7fb5b5f28800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.370+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562d1d07df30 con 0x7fb5ac045130 2024-09-19T11:23:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.370+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-09-19T11:23:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.371+0000 7fb4637fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.371+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94751760328624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d045880 con 0x562d1d032340 2024-09-19T11:23:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.372+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94751760328624] v33'177 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.372+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94751760307536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d042780 con 0x7fb5ac045130 2024-09-19T11:23:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.373+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94751760307536] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.373+0000 7fb5b5f28800 20 remove_watcher() i=0 2024-09-19T11:23:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.373+0000 7fb5b5f28800 2 removed watcher, disabling cache 2024-09-19T11:23:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.373+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94751760311376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03d550 con 0x562d1d032340 2024-09-19T11:23:44.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.374+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94751760311376] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.374+0000 7fb5b5f28800 20 remove_watcher() i=1 2024-09-19T11:23:44.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.374+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94751760316224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03d550 con 0x562d1d036d30 2024-09-19T11:23:44.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.375+0000 7fb5b2470640 1 -- 172.21.15.28:0/3487016268 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94751760316224] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac00a4d0 con 0x562d1d036d30 2024-09-19T11:23:44.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.375+0000 7fb5b5f28800 20 remove_watcher() i=2 2024-09-19T11:23:44.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.375+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94751760313504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03d550 con 0x7fb5ac045130 2024-09-19T11:23:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.376+0000 7fb5b1c6f640 1 -- 172.21.15.28:0/3487016268 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94751760313504] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a407a030 con 0x7fb5ac045130 2024-09-19T11:23:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.376+0000 7fb5b5f28800 20 remove_watcher() i=3 2024-09-19T11:23:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.376+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94751760322768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03eca0 con 0x562d1d032340 2024-09-19T11:23:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.377+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94751760322768] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x562d1d032340 2024-09-19T11:23:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.377+0000 7fb5b5f28800 20 remove_watcher() i=4 2024-09-19T11:23:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.377+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94751760326544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03d550 con 0x7fb5a40a6040 2024-09-19T11:23:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.379+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94751760326544] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.379+0000 7fb5b5f28800 20 remove_watcher() i=5 2024-09-19T11:23:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.379+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94751760321200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d0410d0 con 0x7fb5a40a6040 2024-09-19T11:23:44.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.380+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94751760321200] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.380+0000 7fb5b5f28800 20 remove_watcher() i=6 2024-09-19T11:23:44.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.380+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94751760330288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562d1d03eca0 con 0x7fb5a40a6040 2024-09-19T11:23:44.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94751760330288] v33'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a803b040 con 0x7fb5a40a6040 2024-09-19T11:23:44.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 20 remove_watcher() i=7 2024-09-19T11:23:44.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fb5a40a6040 msgr2=0x7fb5a4009410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fb5a40a6040 0x7fb5a4009410 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1d032340 msgr2=0x562d1d034800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1d032340 0x562d1d034800 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fb5ac045130 msgr2=0x7fb5ac047960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fb5ac045130 0x7fb5ac047960 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d036d30 msgr2=0x562d1d039120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d036d30 0x562d1d039120 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb47c03d000 msgr2=0x7fb47c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb47c03d000 0x7fb47c03f4c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4098390 tx=0x7fb5a4091000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 msgr2=0x562d1d026c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/3487016268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d01f620 0x562d1d026c40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac022ff0 tx=0x7fb5ac01fee0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 reap_dead start 2024-09-19T11:23:44.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b146e640 1 -- 172.21.15.28:0/3487016268 reap_dead start 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 shutdown_connections 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 >> 172.21.15.28:0/3487016268 conn(0x562d1d01ba20 msgr2=0x562d1d019c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 shutdown_connections 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.381+0000 7fb5b5f28800 1 -- 172.21.15.28:0/3487016268 wait complete. 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb48c03d000 msgr2=0x7fb48c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb48c03d000 0x7fb48c03f4c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4096760 tx=0x7fb5a4054ef0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d012b80 msgr2=0x562d1d012fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d012b80 0x562d1d012fa0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x562d1cff91d0 tx=0x7fb5ac0167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 shutdown_connections 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb48c03d000 0x7fb48c03f4c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/525142466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1d012b80 0x562d1d012fa0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 >> 172.21.15.28:0/525142466 conn(0x562d1d006f30 msgr2=0x562d1d007310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 shutdown_connections 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/525142466 wait complete. 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1cf038c0 msgr2=0x562d1cfff200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1cf038c0 0x562d1cfff200 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d0015e0 msgr2=0x562d1d003a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d0015e0 0x562d1d003a60 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb49803d000 msgr2=0x7fb49803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb49803d000 0x7fb49803f4c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a80013b0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 msgr2=0x562d1cff7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff7bb0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4045f50 tx=0x7fb5a4002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 shutdown_connections 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562d1d0015e0 0x562d1d003a60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb49803d000 0x7fb49803f4c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562d1cf038c0 0x562d1cfff200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 --2- 172.21.15.28:0/90839064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562d1cf00ea0 0x562d1cff7bb0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 >> 172.21.15.28:0/90839064 conn(0x562d1cefaa60 msgr2=0x562d1cfea2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 shutdown_connections 2024-09-19T11:23:44.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.382+0000 7fb5b5f28800 1 -- 172.21.15.28:0/90839064 wait complete. 2024-09-19T11:23:44.393 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-19T11:23:44.401 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-09-19T11:23:44.409 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:44.436 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:44.446 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:44.493 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-09-19T11:23:44.498 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:44.498 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-19T11:23:44.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-19T11:23:44.499 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-09-19T11:23:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:44.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3cb2a800 1 Processor -- start 2024-09-19T11:23:44.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3cb2a800 1 -- start start 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3cb2a800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x56512300f890 0x5651230183b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3cb2a800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5651230188f0 con 0x56512300dd70 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3cb2a800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x565122e9a7d0 con 0x56512300f890 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3804e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x56512300f890 0x5651230183b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3884f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x56512300dd70 0x56512300e170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55578/0 (socket says 172.21.15.28:55578) 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3884f640 1 -- 172.21.15.28:0/3535529603 learned_addr learned my addr 172.21.15.28:0/3535529603 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3804e640 1 -- 172.21.15.28:0/3535529603 >> v1:172.21.15.22:6789/0 conn(0x56512300dd70 legacy=0x56512300e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.545+0000 7f0a3804e640 1 -- 172.21.15.28:0/3535529603 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x565123019ad0 con 0x56512300f890 2024-09-19T11:23:44.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.547+0000 7f0a3804e640 1 --2- 172.21.15.28:0/3535529603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300f890 0x5651230183b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00a470 tx=0x7f0a2c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aced71627daaec8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.547+0000 7f0a3784d640 1 -- 172.21.15.28:0/3535529603 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a2c053570 con 0x56512300f890 2024-09-19T11:23:44.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.547+0000 7f0a3784d640 1 -- 172.21.15.28:0/3535529603 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2c058070 con 0x56512300f890 2024-09-19T11:23:44.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.547+0000 7f0a3784d640 1 -- 172.21.15.28:0/3535529603 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a2c053880 con 0x56512300f890 2024-09-19T11:23:44.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.548+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3535529603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300f890 msgr2=0x5651230183b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.548+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/3535529603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300f890 0x5651230183b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00a470 tx=0x7f0a2c045a20 comp rx=0 tx=0).stop 2024-09-19T11:23:44.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.548+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3535529603 shutdown_connections 2024-09-19T11:23:44.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.548+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/3535529603 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300f890 0x5651230183b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.548+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3535529603 >> 172.21.15.28:0/3535529603 conn(0x565123009a60 msgr2=0x56512300bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.548+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3535529603 shutdown_connections 2024-09-19T11:23:44.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.548+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3535529603 wait complete. 2024-09-19T11:23:44.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.563+0000 7f0a3cb2a800 1 Processor -- start 2024-09-19T11:23:44.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.563+0000 7f0a3cb2a800 1 -- start start 2024-09-19T11:23:44.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.563+0000 7f0a3cb2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123101520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.563+0000 7f0a3cb2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565122ea7dd0 con 0x56512300dd70 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.563+0000 7f0a3884f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.564+0000 7f0a3884f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123101520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53320/0 (socket says 172.21.15.28:53320) 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.564+0000 7f0a3884f640 1 -- 172.21.15.28:0/2633713473 learned_addr learned my addr 172.21.15.28:0/2633713473 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.564+0000 7f0a3884f640 1 -- 172.21.15.28:0/2633713473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565123101b10 con 0x56512300dd70 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.564+0000 7f0a3884f640 1 --2- 172.21.15.28:0/2633713473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123101520 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a28001df0 tx=0x7f0a2800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216ba8c1c07a7203 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.564+0000 7f0926e0d640 1 -- 172.21.15.28:0/2633713473 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a28012020 con 0x56512300dd70 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.564+0000 7f0926e0d640 1 -- 172.21.15.28:0/2633713473 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2800e470 con 0x56512300dd70 2024-09-19T11:23:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.564+0000 7f0926e0d640 1 -- 172.21.15.28:0/2633713473 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a2800e7f0 con 0x56512300dd70 2024-09-19T11:23:44.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/2633713473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 msgr2=0x565123101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/2633713473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123101520 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0a28001df0 tx=0x7f0a2800b6c0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/2633713473 shutdown_connections 2024-09-19T11:23:44.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/2633713473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123101520 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/2633713473 >> 172.21.15.28:0/2633713473 conn(0x565123009a60 msgr2=0x5651230fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/2633713473 shutdown_connections 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/2633713473 wait complete. 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 Processor -- start 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.565+0000 7f0a3cb2a800 1 -- start start 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3cb2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3cb2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565123018dc0 con 0x56512300dd70 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3884f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3884f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53334/0 (socket says 172.21.15.28:53334) 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3884f640 1 -- 172.21.15.28:0/99487960 learned_addr learned my addr 172.21.15.28:0/99487960 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3884f640 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651231095d0 con 0x56512300dd70 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3884f640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123109070 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5651231024b0 tx=0x7f0a28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f092560a640 1 -- 172.21.15.28:0/99487960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a28012030 con 0x56512300dd70 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f092560a640 1 -- 172.21.15.28:0/99487960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a28001450 con 0x56512300dd70 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565123109860 con 0x56512300dd70 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f092560a640 1 -- 172.21.15.28:0/99487960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a2801a7a0 con 0x56512300dd70 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.566+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56512310a260 con 0x56512300dd70 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.567+0000 7f092560a640 1 -- 172.21.15.28:0/99487960 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0a28015020 con 0x56512300dd70 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.567+0000 7f092560a640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092003cfb0 0x7f092003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.567+0000 7f092560a640 1 -- 172.21.15.28:0/99487960 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f0a28020030 con 0x56512300dd70 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a3804e640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092003cfb0 0x7f092003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56512300f890 0x5651231106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123110c00 con 0x56512300f890 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a39050640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56512300f890 0x5651231106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a3804e640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092003cfb0 0x7f092003f470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c057000 tx=0x7f0a2c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a39050640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56512300f890 0x5651231106c0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a39050640 1 -- 172.21.15.28:0/99487960 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a3000a4d0 con 0x56512300f890 2024-09-19T11:23:44.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.568+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123110fa0 con 0x56512300f890 2024-09-19T11:23:44.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.569+0000 7f0a39050640 1 -- 172.21.15.28:0/99487960 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0a3000a4d0 con 0x56512300f890 2024-09-19T11:23:44.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.569+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123112ae0 0x565123114f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.569+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651231156e0 con 0x565123112ae0 2024-09-19T11:23:44.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.569+0000 7f0a3884f640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123112ae0 0x565123114f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.569+0000 7f0a3884f640 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123112ae0 0x565123114f50 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.569+0000 7f0a3884f640 1 -- 172.21.15.28:0/99487960 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0a2801ab20 con 0x565123112ae0 2024-09-19T11:23:44.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.570+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651231117f0 con 0x56512300f890 2024-09-19T11:23:44.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.570+0000 7f0a39050640 1 -- 172.21.15.28:0/99487960 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0a3000a4d0 con 0x56512300f890 2024-09-19T11:23:44.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.570+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123111c10 con 0x565123112ae0 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.570+0000 7f0a3884f640 1 -- 172.21.15.28:0/99487960 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0a2801ab20 con 0x565123112ae0 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.570+0000 7f0a3cb2a800 1 Processor -- start 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.570+0000 7f0a3cb2a800 1 -- start start 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a3cb2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123124040 0x565123124460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a3cb2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565122f064a0 con 0x565123124040 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a39050640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123124040 0x565123124460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a39050640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123124040 0x565123124460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53344/0 (socket says 172.21.15.28:53344) 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a39050640 1 -- 172.21.15.28:0/3300026636 learned_addr learned my addr 172.21.15.28:0/3300026636 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a39050640 1 -- 172.21.15.28:0/3300026636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565123124aa0 con 0x565123124040 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a39050640 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123124040 0x565123124460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x56512310a430 tx=0x7f0a300167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f091e7fc640 1 -- 172.21.15.28:0/3300026636 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30013640 con 0x565123124040 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f091e7fc640 1 -- 172.21.15.28:0/3300026636 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a300137e0 con 0x565123124040 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651231252d0 con 0x565123124040 2024-09-19T11:23:44.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f091e7fc640 1 -- 172.21.15.28:0/3300026636 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30013ae0 con 0x565123124040 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.571+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565123125870 con 0x565123124040 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.572+0000 7f091e7fc640 1 -- 172.21.15.28:0/3300026636 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0a30011040 con 0x565123124040 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.572+0000 7f091e7fc640 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f091003d000 0x7f091003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.572+0000 7f091e7fc640 1 -- 172.21.15.28:0/3300026636 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f0a3000e040 con 0x565123124040 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.572+0000 7f0a3884f640 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f091003d000 0x7f091003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a3cb2a800 1 Processor -- start 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a3884f640 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f091003d000 0x7f091003f4c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x565123115a80 tx=0x7f0a28018f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a3cb2a800 1 -- start start 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a3cb2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x565123130f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a3cb2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565123101f40 con 0x565123130b00 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a39050640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x565123130f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a39050640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x565123130f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53350/0 (socket says 172.21.15.28:53350) 2024-09-19T11:23:44.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a39050640 1 -- 172.21.15.28:0/982015243 learned_addr learned my addr 172.21.15.28:0/982015243 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.573+0000 7f0a39050640 1 -- 172.21.15.28:0/982015243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651231314f0 con 0x565123130b00 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a39050640 1 --2- 172.21.15.28:0/982015243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x565123130f00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0a30054840 tx=0x7f0a30020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d255c4ceb086465 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f090ffff640 1 -- 172.21.15.28:0/982015243 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a3001fbd0 con 0x565123130b00 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f090ffff640 1 -- 172.21.15.28:0/982015243 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a30052070 con 0x565123130b00 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f090ffff640 1 -- 172.21.15.28:0/982015243 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30021440 con 0x565123130b00 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/982015243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 msgr2=0x565123130f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/982015243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x565123130f00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0a30054840 tx=0x7f0a30020460 comp rx=0 tx=0).stop 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/982015243 shutdown_connections 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/982015243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x565123130f00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/982015243 >> 172.21.15.28:0/982015243 conn(0x56512312cf00 msgr2=0x56512312b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/982015243 shutdown_connections 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/982015243 wait complete. 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.574+0000 7f0a3cb2a800 1 Processor -- start 2024-09-19T11:23:44.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a3cb2a800 1 -- start start 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a3cb2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x5651231380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a3cb2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565123131b10 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a39050640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x5651231380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a39050640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x5651231380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53360/0 (socket says 172.21.15.28:53360) 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 learned_addr learned my addr 172.21.15.28:0/1237462129 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565123138620 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f0a39050640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x5651231380c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0a3001ff70 tx=0x7f0a300547d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.575+0000 7f090e7fc640 1 -- 172.21.15.28:0/1237462129 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a300562e0 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.576+0000 7f090e7fc640 1 -- 172.21.15.28:0/1237462129 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a30045a10 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.576+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651231388b0 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.576+0000 7f090e7fc640 1 -- 172.21.15.28:0/1237462129 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a30045d30 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.576+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565123139230 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.576+0000 7f090e7fc640 1 -- 172.21.15.28:0/1237462129 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f0a30054980 con 0x565123130b00 2024-09-19T11:23:44.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f090e7fc640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f090403d000 0x7f090403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f090e7fc640 1 -- 172.21.15.28:0/1237462129 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f0a3004c030 con 0x565123130b00 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f0a3884f640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f090403d000 0x7f090403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f0a3884f640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f090403d000 0x7f090403f4c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x56512310b190 tx=0x7f0a2803ff90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565123143a40 0x565123145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123146650 con 0x565123143a40 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.577+0000 7f0a3804e640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565123143a40 0x565123145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3804e640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565123143a40 0x565123145f00 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 20 realm 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123146a80 con 0x565123143a40 2024-09-19T11:23:44.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.578+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123147350 con 0x565123143a40 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3cb2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651231476f0 con 0x565123143a40 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3cb2a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123148450 0x56512314a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512314ae50 con 0x565123148450 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a39050640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123148450 0x56512314a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.579+0000 7f0a39050640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123148450 0x56512314a840 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a3cb2a800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a3cb2a800 20 searching for the correct realm 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a39050640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a30032720 0x7f0a3003b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a3884f640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a30032720 0x7f0a3003b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.580+0000 7f0a3884f640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a30032720 0x7f0a3003b750 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.581+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.581+0000 7f0a3884f640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a28017050 0x7f0a28022750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.581+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a28017050 2024-09-19T11:23:44.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.581+0000 7f0a3804e640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a28017050 0x7f0a28022750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.581+0000 7f0a3804e640 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a28017050 0x7f0a28022750 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a28017050 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a28017050 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.582+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a28017050 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.583+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.584+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.585+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a28017050 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.586+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.587+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.587+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a28017050 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.587+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.587+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.587+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.587+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a28017050 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.588+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.588+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.588+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.588+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123148450 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.588+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.588+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x565123143a40 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x56512314c780 con 0x7f0a30032720 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512314b840 con 0x565123143a40 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.589+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512314c090 con 0x565123143a40 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512314cc00 con 0x565123148450 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 zone default found 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 4 Realm: () 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 zonegroup default 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512314d000 con 0x7f0a28017050 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.590+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512314ee30 con 0x565123148450 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a3cb2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512314f2a0 con 0x565123148450 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a3cb2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a3cb2a800 20 started sync module instance, tier type = 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a3cb2a800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.591+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x56512314e2d0 con 0x7f0a30032720 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.592+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.592+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94906480912064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x56512314fbb0 con 0x7f0a30032720 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.592+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123150400 con 0x565123143a40 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.593+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94906480912064] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.593+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.593+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94906480916224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123150460 con 0x565123143a40 2024-09-19T11:23:44.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.594+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123150f70 con 0x565123148450 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.594+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.594+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94906480921120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123150f70 con 0x565123148450 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.595+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123151f70 con 0x7f0a30032720 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.595+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94906480916224] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.595+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94906480921120] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.596+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.596+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94906480925280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123151360 con 0x7f0a30032720 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.596+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123152980 con 0x565123143a40 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.597+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94906480925280] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.597+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.597+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94906480918352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123152980 con 0x565123143a40 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.597+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123153340 con 0x7f0a28017050 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.598+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94906480918352] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.598+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.598+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94906480928432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123153340 con 0x7f0a28017050 2024-09-19T11:23:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.598+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123153d00 con 0x7f0a28017050 2024-09-19T11:23:44.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.599+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94906480928432] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.599+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.599+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94906480930448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123153ee0 con 0x7f0a28017050 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.599+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123154730 con 0x7f0a28017050 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94906480934896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123154ae0 con 0x7f0a28017050 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 20 add_watcher() i=0 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 20 add_watcher() i=1 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 20 add_watcher() i=2 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 20 add_watcher() i=3 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 20 add_watcher() i=4 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 20 add_watcher() i=5 2024-09-19T11:23:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94906480930448] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.600+0000 7f0a3cb2a800 20 add_watcher() i=6 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.601+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94906480934896] v33'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.601+0000 7f0a3cb2a800 20 add_watcher() i=7 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.601+0000 7f0a3cb2a800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.601+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123153a90 con 0x565123143a40 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.601+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.601+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94906480936304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123153a90 con 0x565123143a40 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.602+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94906480936304] v33'178 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.602+0000 7f08eefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 5 note: GC not initialized 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f08edfe3640 20 reqs_thread_entry: start 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 20 Filter name: none 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f085f7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.603+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512318b0b0 con 0x7f0a30032720 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 : miss 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:63 8.14 8:2e2e7a97:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56512318cb90 con 0x565123148450 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(63 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 info.flags=0x17 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 to cache LRU end 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 20 get_obj_state: octx=0x7f0a2803ff10 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x56512318ed58 s->prefetch_data=0 2024-09-19T11:23:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 3.2as0 3:5576cde3:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x565123143a40 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.604+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(64 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432 [call out=24b,getxattrs out=848b,stat out=16b] v0'0 uv2 ondisk = 0) ==== 304+0+888 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.605+0000 7f0a3cb2a800 10 manifest: total_size = 5 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.605+0000 7f0a3cb2a800 20 get_obj_state: setting s->obj_tag to _6WfZPhjE1AFc77HK6Yhjn_FDUrMLXXB 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.605+0000 7f08eefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.605+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94906480936304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x565123143a40 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.606+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94906480936304] v33'179 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.606+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94906480912064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x7f0a30032720 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.607+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94906480912064] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.607+0000 7f0a3cb2a800 20 remove_watcher() i=0 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.607+0000 7f0a3cb2a800 2 removed watcher, disabling cache 2024-09-19T11:23:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.607+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94906480916224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x565123143a40 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.608+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94906480916224] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.608+0000 7f0a3cb2a800 20 remove_watcher() i=1 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.608+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94906480921120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x565123148450 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.609+0000 7f0a39050640 1 -- 172.21.15.28:0/1237462129 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94906480921120] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a3003a070 con 0x565123148450 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.609+0000 7f0a3cb2a800 20 remove_watcher() i=2 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.609+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94906480925280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x7f0a30032720 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.610+0000 7f0a3884f640 1 -- 172.21.15.28:0/1237462129 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94906480925280] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2801ab20 con 0x7f0a30032720 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.610+0000 7f0a3cb2a800 20 remove_watcher() i=3 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.610+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94906480918352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x565123143a40 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.611+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94906480918352] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x565123143a40 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.611+0000 7f0a3cb2a800 20 remove_watcher() i=4 2024-09-19T11:23:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.611+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94906480928432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x7f0a28017050 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.612+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94906480928432] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.612+0000 7f0a3cb2a800 20 remove_watcher() i=5 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.612+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94906480930448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x7f0a28017050 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.613+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94906480930448] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.613+0000 7f0a3cb2a800 20 remove_watcher() i=6 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.613+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94906480934896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x565123190f00 con 0x7f0a28017050 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94906480934896] v33'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0a2c051070 con 0x7f0a28017050 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 20 remove_watcher() i=7 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a28017050 msgr2=0x7f0a28022750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f0a28017050 0x7f0a28022750 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565123143a40 msgr2=0x565123145f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x565123143a40 0x565123145f00 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a30032720 msgr2=0x7f0a3003b750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f0a30032720 0x7f0a3003b750 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123148450 msgr2=0x56512314a840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.614+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123148450 0x56512314a840 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f090403d000 msgr2=0x7f090403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f090403d000 0x7f090403f4c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x56512310b190 tx=0x7f0a2803ff90 comp rx=0 tx=0).stop 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 msgr2=0x5651231380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3804e640 1 -- 172.21.15.28:0/1237462129 reap_dead start 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x5651231380c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0a3001ff70 tx=0x7f0a300547d0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 shutdown_connections 2024-09-19T11:23:44.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/1237462129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123130b00 0x5651231380c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 >> 172.21.15.28:0/1237462129 conn(0x56512312cf00 msgr2=0x56512312b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 shutdown_connections 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/1237462129 wait complete. 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f091003d000 msgr2=0x7f091003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f091003d000 0x7f091003f4c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x565123115a80 tx=0x7f0a28018f00 comp rx=0 tx=0).stop 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123124040 msgr2=0x565123124460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123124040 0x565123124460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x56512310a430 tx=0x7f0a300167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 shutdown_connections 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f091003d000 0x7f091003f4c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/3300026636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565123124040 0x565123124460 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 >> 172.21.15.28:0/3300026636 conn(0x5651231183f0 msgr2=0x5651231187d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 shutdown_connections 2024-09-19T11:23:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/3300026636 wait complete. 2024-09-19T11:23:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56512300f890 msgr2=0x5651231106c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56512300f890 0x5651231106c0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123112ae0 msgr2=0x565123114f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.615+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123112ae0 0x565123114f50 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092003cfb0 msgr2=0x7f092003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092003cfb0 0x7f092003f470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c057000 tx=0x7f0a2c052000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 msgr2=0x565123109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123109070 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x5651231024b0 tx=0x7f0a28000bc0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 shutdown_connections 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x565123112ae0 0x565123114f50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f092003cfb0 0x7f092003f470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56512300f890 0x5651231106c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 --2- 172.21.15.28:0/99487960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56512300dd70 0x565123109070 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 >> 172.21.15.28:0/99487960 conn(0x565123009a60 msgr2=0x565123102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 shutdown_connections 2024-09-19T11:23:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.616+0000 7f0a3cb2a800 1 -- 172.21.15.28:0/99487960 wait complete. 2024-09-19T11:23:44.632 INFO:teuthology.orchestra.run.smithi028.stdout:fooFoofooFooFULL_CONTROL 2024-09-19T11:23:44.633 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-09-19T11:23:44.635 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-09-19T11:23:44.640 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:44.640 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-19T11:23:44.640 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-19T11:23:44.641 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-09-19T11:23:44.687 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:44.687 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:44.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63fb31f800 1 Processor -- start 2024-09-19T11:23:44.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63fb31f800 1 -- start start 2024-09-19T11:23:44.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63fb31f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f17b71b520 0x55f17b7af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63fb31f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f17b7af900 con 0x55f17b71aa10 2024-09-19T11:23:44.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63fb31f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55f17b6317d0 con 0x55f17b71b520 2024-09-19T11:23:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63f678a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f17b71b520 0x55f17b7af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63f678a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f17b71b520 0x55f17b7af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53376/0 (socket says 172.21.15.28:53376) 2024-09-19T11:23:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63f678a640 1 -- 172.21.15.28:0/2966400216 learned_addr learned my addr 172.21.15.28:0/2966400216 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.687+0000 7f63f6f8b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55f17b71aa10 0x55f17b71ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55584/0 (socket says 172.21.15.28:55584) 2024-09-19T11:23:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.688+0000 7f63f678a640 1 -- 172.21.15.28:0/2966400216 >> v1:172.21.15.22:6789/0 conn(0x55f17b71aa10 legacy=0x55f17b71ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.688+0000 7f63f678a640 1 -- 172.21.15.28:0/2966400216 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17b7b0ae0 con 0x55f17b71b520 2024-09-19T11:23:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.689+0000 7f63f678a640 1 --2- 172.21.15.28:0/2966400216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71b520 0x55f17b7af3c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00a520 tx=0x7f63ec045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8093509b8cd72ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63f5f89640 1 -- 172.21.15.28:0/2966400216 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63ec055070 con 0x55f17b71b520 2024-09-19T11:23:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63f5f89640 1 -- 172.21.15.28:0/2966400216 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ec0515b0 con 0x55f17b71b520 2024-09-19T11:23:44.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63f5f89640 1 -- 172.21.15.28:0/2966400216 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63ec0518d0 con 0x55f17b71b520 2024-09-19T11:23:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63fb31f800 1 -- 172.21.15.28:0/2966400216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71b520 msgr2=0x55f17b7af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2966400216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71b520 0x55f17b7af3c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00a520 tx=0x7f63ec045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63fb31f800 1 -- 172.21.15.28:0/2966400216 shutdown_connections 2024-09-19T11:23:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2966400216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71b520 0x55f17b7af3c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63fb31f800 1 -- 172.21.15.28:0/2966400216 >> 172.21.15.28:0/2966400216 conn(0x55f17b7a1c70 msgr2=0x55f17b7a40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63fb31f800 1 -- 172.21.15.28:0/2966400216 shutdown_connections 2024-09-19T11:23:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.690+0000 7f63fb31f800 1 -- 172.21.15.28:0/2966400216 wait complete. 2024-09-19T11:23:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.705+0000 7f63fb31f800 1 Processor -- start 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.705+0000 7f63fb31f800 1 -- start start 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.705+0000 7f63fb31f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b898560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.705+0000 7f63fb31f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f17b63edd0 con 0x55f17b71aa10 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f63f6f8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b898560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f63f6f8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b898560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53392/0 (socket says 172.21.15.28:53392) 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f63f6f8b640 1 -- 172.21.15.28:0/3609617012 learned_addr learned my addr 172.21.15.28:0/3609617012 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f63f6f8b640 1 -- 172.21.15.28:0/3609617012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17b898b50 con 0x55f17b71aa10 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/3609617012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b898560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b260 tx=0x7f63e800b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaedbd51910342 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f62e4e09640 1 -- 172.21.15.28:0/3609617012 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e800be60 con 0x55f17b71aa10 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f62e4e09640 1 -- 172.21.15.28:0/3609617012 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e8006070 con 0x55f17b71aa10 2024-09-19T11:23:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.706+0000 7f62e4e09640 1 -- 172.21.15.28:0/3609617012 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e800e710 con 0x55f17b71aa10 2024-09-19T11:23:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 -- 172.21.15.28:0/3609617012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 msgr2=0x55f17b898560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 --2- 172.21.15.28:0/3609617012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b898560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f63e800b260 tx=0x7f63e800b660 comp rx=0 tx=0).stop 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 -- 172.21.15.28:0/3609617012 shutdown_connections 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 --2- 172.21.15.28:0/3609617012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b898560 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 -- 172.21.15.28:0/3609617012 >> 172.21.15.28:0/3609617012 conn(0x55f17b7a1c70 msgr2=0x55f17b893ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 -- 172.21.15.28:0/3609617012 shutdown_connections 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 -- 172.21.15.28:0/3609617012 wait complete. 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.707+0000 7f63fb31f800 1 Processor -- start 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63fb31f800 1 -- start start 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63fb31f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b8a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63fb31f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f17b7afdd0 con 0x55f17b71aa10 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63f6f8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b8a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63f6f8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b8a0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53400/0 (socket says 172.21.15.28:53400) 2024-09-19T11:23:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63f6f8b640 1 -- 172.21.15.28:0/2332418838 learned_addr learned my addr 172.21.15.28:0/2332418838 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63f6f8b640 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17b8a05c0 con 0x55f17b71aa10 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b8a0060 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55f17b8994f0 tx=0x7f63e8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f62e3606640 1 -- 172.21.15.28:0/2332418838 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e800f810 con 0x55f17b71aa10 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f62e3606640 1 -- 172.21.15.28:0/2332418838 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e800f9b0 con 0x55f17b71aa10 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f62e3606640 1 -- 172.21.15.28:0/2332418838 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e800fcd0 con 0x55f17b71aa10 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.708+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f17b8a0850 con 0x55f17b71aa10 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.709+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17b8a1250 con 0x55f17b71aa10 2024-09-19T11:23:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.709+0000 7f62e3606640 1 -- 172.21.15.28:0/2332418838 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f63e8017070 con 0x55f17b71aa10 2024-09-19T11:23:44.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f62e3606640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62dc03cfb0 0x7f62dc03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f62e3606640 1 -- 172.21.15.28:0/2332418838 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f63e8015020 con 0x55f17b71aa10 2024-09-19T11:23:44.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f63f678a640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62dc03cfb0 0x7f62dc03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b71b520 0x55f17b8a76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8a7bf0 con 0x55f17b71b520 2024-09-19T11:23:44.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f63f778c640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b71b520 0x55f17b8a76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f63f678a640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62dc03cfb0 0x7f62dc03f470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63ec057000 tx=0x7f63ec052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.710+0000 7f63f778c640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b71b520 0x55f17b8a76b0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.711+0000 7f63f778c640 1 -- 172.21.15.28:0/2332418838 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b71b520 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.711+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8a7f90 con 0x55f17b71b520 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.711+0000 7f63f778c640 1 -- 172.21.15.28:0/2332418838 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b71b520 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.711+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8a9a90 0x55f17b8abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.711+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8ac6a0 con 0x55f17b8a9a90 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.711+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8a9a90 0x55f17b8abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.711+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8a9a90 0x55f17b8abf10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.712+0000 7f63f6f8b640 1 -- 172.21.15.28:0/2332418838 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f63e802e070 con 0x55f17b8a9a90 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.712+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8a87e0 con 0x55f17b71b520 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.712+0000 7f63f778c640 1 -- 172.21.15.28:0/2332418838 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b71b520 2024-09-19T11:23:44.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.712+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8a8bd0 con 0x55f17b8a9a90 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.712+0000 7f63f6f8b640 1 -- 172.21.15.28:0/2332418838 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f63e802e070 con 0x55f17b8a9a90 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63fb31f800 1 Processor -- start 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63fb31f800 1 -- start start 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63fb31f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8baff0 0x55f17b8bb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63fb31f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f17b69d4a0 con 0x55f17b8baff0 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63f778c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8baff0 0x55f17b8bb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63f778c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8baff0 0x55f17b8bb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53408/0 (socket says 172.21.15.28:53408) 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63f778c640 1 -- 172.21.15.28:0/3903810237 learned_addr learned my addr 172.21.15.28:0/3903810237 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63f778c640 1 -- 172.21.15.28:0/3903810237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17b8bba50 con 0x55f17b8baff0 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.713+0000 7f63f778c640 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8baff0 0x55f17b8bb410 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55f17b8a1060 tx=0x7f63f00167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.714+0000 7f62e0e01640 1 -- 172.21.15.28:0/3903810237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63f0013640 con 0x55f17b8baff0 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.714+0000 7f62e0e01640 1 -- 172.21.15.28:0/3903810237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f00137e0 con 0x55f17b8baff0 2024-09-19T11:23:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.714+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f17b8bc280 con 0x55f17b8baff0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.714+0000 7f62e0e01640 1 -- 172.21.15.28:0/3903810237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63f0013ae0 con 0x55f17b8baff0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.714+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17b8bc820 con 0x55f17b8baff0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.714+0000 7f62e0e01640 1 -- 172.21.15.28:0/3903810237 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f63f0011040 con 0x55f17b8baff0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f62e0e01640 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62d003cfb0 0x7f62d003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f62e0e01640 1 -- 172.21.15.28:0/3903810237 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f63f000e040 con 0x55f17b8baff0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62d003cfb0 0x7f62d003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f63fb31f800 1 Processor -- start 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62d003cfb0 0x7f62d003f470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55f17b8a2150 tx=0x7f63e8017c10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f63fb31f800 1 -- start start 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f63fb31f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f63fb31f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f17b898f80 con 0x55f17b8c7ab0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.715+0000 7f63f778c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f63f778c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53424/0 (socket says 172.21.15.28:53424) 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f63f778c640 1 -- 172.21.15.28:0/465502825 learned_addr learned my addr 172.21.15.28:0/465502825 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f63f778c640 1 -- 172.21.15.28:0/465502825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17b8c84a0 con 0x55f17b8c7ab0 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f63f778c640 1 --2- 172.21.15.28:0/465502825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8c7eb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f63f0054840 tx=0x7f63f00204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1452fd465798a061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f62da7fc640 1 -- 172.21.15.28:0/465502825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63f0021a40 con 0x55f17b8c7ab0 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f62da7fc640 1 -- 172.21.15.28:0/465502825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f0052070 con 0x55f17b8c7ab0 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f62da7fc640 1 -- 172.21.15.28:0/465502825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63f00215b0 con 0x55f17b8c7ab0 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f63fb31f800 1 -- 172.21.15.28:0/465502825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 msgr2=0x55f17b8c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f63fb31f800 1 --2- 172.21.15.28:0/465502825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8c7eb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f63f0054840 tx=0x7f63f00204d0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.716+0000 7f63fb31f800 1 -- 172.21.15.28:0/465502825 shutdown_connections 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 --2- 172.21.15.28:0/465502825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8c7eb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 -- 172.21.15.28:0/465502825 >> 172.21.15.28:0/465502825 conn(0x55f17b8c3eb0 msgr2=0x55f17b8c20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 -- 172.21.15.28:0/465502825 shutdown_connections 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 -- 172.21.15.28:0/465502825 wait complete. 2024-09-19T11:23:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 Processor -- start 2024-09-19T11:23:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 -- start start 2024-09-19T11:23:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63fb31f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f17b8c8a90 con 0x55f17b8c7ab0 2024-09-19T11:23:44.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63f778c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63f778c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53428/0 (socket says 172.21.15.28:53428) 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.717+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 learned_addr learned my addr 172.21.15.28:0/1903115127 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.718+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17b8cf5a0 con 0x55f17b8c7ab0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.718+0000 7f63f778c640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8cf040 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f63f001fcf0 tx=0x7f63f00547a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.718+0000 7f62d8ff9640 1 -- 172.21.15.28:0/1903115127 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63f00561a0 con 0x55f17b8c7ab0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.718+0000 7f62d8ff9640 1 -- 172.21.15.28:0/1903115127 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f0056340 con 0x55f17b8c7ab0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.718+0000 7f62d8ff9640 1 -- 172.21.15.28:0/1903115127 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63f001f830 con 0x55f17b8c7ab0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.718+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f17b8cf830 con 0x55f17b8c7ab0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.718+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17b8d03c0 con 0x55f17b8c7ab0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.719+0000 7f62d8ff9640 1 -- 172.21.15.28:0/1903115127 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f63f00459e0 con 0x55f17b8c7ab0 2024-09-19T11:23:44.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.719+0000 7f62d8ff9640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62c003cfb0 0x7f62c003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.719+0000 7f62d8ff9640 1 -- 172.21.15.28:0/1903115127 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f63f0053ac0 con 0x55f17b8c7ab0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.719+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62c003cfb0 0x7f62c003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.719+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62c003cfb0 0x7f62c003f470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63e80535e0 tx=0x7f63e8028000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b8da9c0 0x55f17b8dce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8dd5d0 con 0x55f17b8da9c0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63f678a640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b8da9c0 0x55f17b8dce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63f678a640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b8da9c0 0x55f17b8dce80 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63fb31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63fb31f800 20 realm 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.720+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8dda00 con 0x55f17b8da9c0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8de2d0 con 0x55f17b8da9c0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8de670 con 0x55f17b8da9c0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8df3d0 0x55f17b8e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e1dd0 con 0x55f17b8df3d0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.721+0000 7f63f778c640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8df3d0 0x55f17b8e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63f778c640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8df3d0 0x55f17b8e17c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63fb31f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63fb31f800 20 searching for the correct realm 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63f778c640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f63f0033440 0x7f63f00451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63f0033440 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.722+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f63f0033440 0x7f63f00451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.723+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f63f0033440 0x7f63f00451d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.723+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.723+0000 7f63f6f8b640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f63e800f070 0x7f63e80536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.723+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63e800f070 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.723+0000 7f63f678a640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f63e800f070 0x7f63e80536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.723+0000 7f63f678a640 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f63e800f070 0x7f63e80536d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8da9c0 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8da9c0 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63e800f070 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.724+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63e800f070 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63f0033440 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8da9c0 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.725+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63e800f070 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8da9c0 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63f0033440 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.726+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63f0033440 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8da9c0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.727+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8da9c0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63e800f070 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63f0033440 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.728+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8da9c0 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63e800f070 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63e800f070 2024-09-19T11:23:44.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.729+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63f0033440 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x55f17b8df3d0 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3720 con 0x7f63f0033440 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f63e8066040 con 0x55f17b8da9c0 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.730+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f63e8066040 con 0x7f63f0033440 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:44.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e27c0 con 0x55f17b8da9c0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3010 con 0x55f17b8da9c0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3ba0 con 0x55f17b8df3d0 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.731+0000 7f63fb31f800 20 zone default found 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 4 Realm: () 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 20 zonegroup default 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e3f60 con 0x7f63e800f070 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e6240 con 0x55f17b8df3d0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.732+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e65e0 con 0x55f17b8df3d0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.733+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.733+0000 7f63fb31f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.733+0000 7f63fb31f800 20 started sync module instance, tier type = 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.733+0000 7f63fb31f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.733+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e5680 con 0x7f63f0033440 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.734+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.734+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94495648403920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e6a10 con 0x7f63f0033440 2024-09-19T11:23:44.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.734+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e7260 con 0x55f17b8da9c0 2024-09-19T11:23:44.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.735+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.735+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94495648411328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e78f0 con 0x55f17b8da9c0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.735+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e8410 con 0x55f17b8df3d0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.735+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94495648403920] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.736+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.736+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94495648415456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e83d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.736+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94495648411328] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.736+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e73a0 con 0x7f63f0033440 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.737+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94495648415456] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.737+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.737+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94495648406176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e83d0 con 0x7f63f0033440 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.737+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e94a0 con 0x55f17b8da9c0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.738+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.738+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94495648419824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e98c0 con 0x55f17b8da9c0 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.738+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8ea3c0 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.738+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94495648406176] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.739+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.739+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94495648419824] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.739+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94495648423600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8ea380 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.739+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e9410 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.740+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94495648423600] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.740+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.740+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94495648420880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8ea920 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.740+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8eb2e0 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94495648420880] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94495648427568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8eb700 con 0x7f63e800f070 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 20 add_watcher() i=0 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 20 add_watcher() i=1 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 20 add_watcher() i=2 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 20 add_watcher() i=3 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 20 add_watcher() i=4 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 20 add_watcher() i=5 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.741+0000 7f63fb31f800 20 add_watcher() i=6 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.742+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94495648427568] v33'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.742+0000 7f63fb31f800 20 add_watcher() i=7 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.742+0000 7f63fb31f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.742+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8eb700 con 0x55f17b8da9c0 2024-09-19T11:23:44.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.743+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.743+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94495648428624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8eb6b0 con 0x55f17b8da9c0 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94495648428624] v33'180 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f62ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f63fb31f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f63fb31f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f63fb31f800 5 note: GC not initialized 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f6227fff640 20 reqs_thread_entry: start 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f63fb31f800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f63fb31f800 20 Filter name: none 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.744+0000 7f6225ffb640 20 reqs_thread_entry: start 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b921bb0 con 0x7f63f0033440 2024-09-19T11:23:44.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 : miss 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:63 8.14 8:2e2e7a97:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b923690 con 0x55f17b8df3d0 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(63 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 info.flags=0x17 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 to cache LRU end 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 2024-09-19T11:23:44.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 20 get_obj_state: octx=0x7f63e8034040 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55f17b925858 s->prefetch_data=0 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.745+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 3.2as0 3:5576cde3:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f17b927b30 con 0x55f17b8da9c0 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.746+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(64 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432 [call out=24b,getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 304+0+950 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.746+0000 7f63fb31f800 10 manifest: total_size = 5 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.746+0000 7f63fb31f800 20 get_obj_state: setting s->obj_tag to _m2gLGlux3KHWMQDgk1QE0nUgPvBea1Q 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.747+0000 7f62ad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.747+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94495648428624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b91f440 con 0x55f17b8da9c0 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.748+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94495648428624] v33'181 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.748+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94495648403920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b91f440 con 0x7f63f0033440 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.749+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94495648403920] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.749+0000 7f63fb31f800 20 remove_watcher() i=0 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.749+0000 7f63fb31f800 2 removed watcher, disabling cache 2024-09-19T11:23:44.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.749+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94495648411328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e51d0 con 0x55f17b8da9c0 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.750+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94495648411328] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.750+0000 7f63fb31f800 20 remove_watcher() i=1 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.750+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94495648415456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b91f440 con 0x55f17b8df3d0 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.751+0000 7f63f778c640 1 -- 172.21.15.28:0/1903115127 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94495648415456] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63f000a4d0 con 0x55f17b8df3d0 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.751+0000 7f63fb31f800 20 remove_watcher() i=2 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.751+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94495648406176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e6ec0 con 0x7f63f0033440 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.752+0000 7f63f6f8b640 1 -- 172.21.15.28:0/1903115127 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94495648406176] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63e802e070 con 0x7f63f0033440 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.752+0000 7f63fb31f800 20 remove_watcher() i=3 2024-09-19T11:23:44.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.752+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94495648419824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e51d0 con 0x55f17b8da9c0 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.754+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94495648419824] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x55f17b8da9c0 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.754+0000 7f63fb31f800 20 remove_watcher() i=4 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.754+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94495648423600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e7ee0 con 0x7f63e800f070 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.755+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94495648423600] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.755+0000 7f63fb31f800 20 remove_watcher() i=5 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.755+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94495648420880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b91f440 con 0x7f63e800f070 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.756+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94495648420880] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.756+0000 7f63fb31f800 20 remove_watcher() i=6 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.756+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94495648427568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f17b8e6ec0 con 0x7f63e800f070 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94495648427568] v33'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f63ec057030 con 0x7f63e800f070 2024-09-19T11:23:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 20 remove_watcher() i=7 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f63e800f070 msgr2=0x7f63e80536d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f63e800f070 0x7f63e80536d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b8da9c0 msgr2=0x55f17b8dce80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b8da9c0 0x55f17b8dce80 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f63f0033440 msgr2=0x7f63f00451d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f63f0033440 0x7f63f00451d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8df3d0 msgr2=0x55f17b8e17c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8df3d0 0x55f17b8e17c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62c003cfb0 msgr2=0x7f62c003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62c003cfb0 0x7f62c003f470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f63e80535e0 tx=0x7f63e8028000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 msgr2=0x55f17b8cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8cf040 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f63f001fcf0 tx=0x7f63f00547a0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63f678a640 1 -- 172.21.15.28:0/1903115127 reap_dead start 2024-09-19T11:23:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 shutdown_connections 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/1903115127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8c7ab0 0x55f17b8cf040 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 >> 172.21.15.28:0/1903115127 conn(0x55f17b8c3eb0 msgr2=0x55f17b8c20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 shutdown_connections 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/1903115127 wait complete. 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62d003cfb0 msgr2=0x7f62d003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62d003cfb0 0x7f62d003f470 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55f17b8a2150 tx=0x7f63e8017c10 comp rx=0 tx=0).stop 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8baff0 msgr2=0x55f17b8bb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8baff0 0x55f17b8bb410 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55f17b8a1060 tx=0x7f63f00167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 shutdown_connections 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62d003cfb0 0x7f62d003f470 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.757+0000 7f63fb31f800 1 --2- 172.21.15.28:0/3903810237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b8baff0 0x55f17b8bb410 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 >> 172.21.15.28:0/3903810237 conn(0x55f17b8af3a0 msgr2=0x55f17b8af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 shutdown_connections 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/3903810237 wait complete. 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b71b520 msgr2=0x55f17b8a76b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b71b520 0x55f17b8a76b0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8a9a90 msgr2=0x55f17b8abf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8a9a90 0x55f17b8abf10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62dc03cfb0 msgr2=0x7f62dc03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62dc03cfb0 0x7f62dc03f470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63ec057000 tx=0x7f63ec052000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 msgr2=0x55f17b8a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b8a0060 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55f17b8994f0 tx=0x7f63e8000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 shutdown_connections 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f17b8a9a90 0x55f17b8abf10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f62dc03cfb0 0x7f62dc03f470 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f17b71b520 0x55f17b8a76b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 --2- 172.21.15.28:0/2332418838 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f17b71aa10 0x55f17b8a0060 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 >> 172.21.15.28:0/2332418838 conn(0x55f17b7a1c70 msgr2=0x55f17b899a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 shutdown_connections 2024-09-19T11:23:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.758+0000 7f63fb31f800 1 -- 172.21.15.28:0/2332418838 wait complete. 2024-09-19T11:23:44.770 INFO:teuthology.orchestra.run.smithi028.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-09-19T11:23:44.770 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-09-19T11:23:44.772 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-09-19T11:23:44.785 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:44.841 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-09-19T11:23:44.842 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-09-19T11:23:44.842 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-19T11:23:44.842 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-09-19T11:23:44.842 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-09-19T11:23:44.842 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-09-19T11:23:44.842 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-09-19T11:23:44.889 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:44.889 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.888+0000 7f3239b2c800 1 Processor -- start 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f3239b2c800 1 -- start start 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f3239b2c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560d7eafaf20 0x560d7eafd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f3239b2c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560d7eb015d0 con 0x560d7eafd8b0 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f3239b2c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x560d7e9857d0 con 0x560d7eafaf20 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f323573e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560d7eafaf20 0x560d7eafd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f323573e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560d7eafaf20 0x560d7eafd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53432/0 (socket says 172.21.15.28:53432) 2024-09-19T11:23:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f3234f3d640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x560d7eafd8b0 0x560d7eaffd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55600/0 (socket says 172.21.15.28:55600) 2024-09-19T11:23:44.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f323573e640 1 -- 172.21.15.28:0/3956742343 learned_addr learned my addr 172.21.15.28:0/3956742343 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f323573e640 1 -- 172.21.15.28:0/3956742343 >> v1:172.21.15.22:6789/0 conn(0x560d7eafd8b0 legacy=0x560d7eaffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.889+0000 7f323573e640 1 -- 172.21.15.28:0/3956742343 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7eb02800 con 0x560d7eafaf20 2024-09-19T11:23:44.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.891+0000 7f323573e640 1 --2- 172.21.15.28:0/3956742343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafd310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f322800a520 tx=0x7f3228045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c0253d9e9b8df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.891+0000 7f3227fff640 1 -- 172.21.15.28:0/3956742343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3228055070 con 0x560d7eafaf20 2024-09-19T11:23:44.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.891+0000 7f3227fff640 1 -- 172.21.15.28:0/3956742343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32280515b0 con 0x560d7eafaf20 2024-09-19T11:23:44.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.891+0000 7f3227fff640 1 -- 172.21.15.28:0/3956742343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32280518d0 con 0x560d7eafaf20 2024-09-19T11:23:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.892+0000 7f3239b2c800 1 -- 172.21.15.28:0/3956742343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 msgr2=0x560d7eafd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.892+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3956742343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafd310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f322800a520 tx=0x7f3228045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.892+0000 7f3239b2c800 1 -- 172.21.15.28:0/3956742343 shutdown_connections 2024-09-19T11:23:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.892+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3956742343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafd310 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.892+0000 7f3239b2c800 1 -- 172.21.15.28:0/3956742343 >> 172.21.15.28:0/3956742343 conn(0x560d7eaf4a60 msgr2=0x560d7eaf6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.892+0000 7f3239b2c800 1 -- 172.21.15.28:0/3956742343 shutdown_connections 2024-09-19T11:23:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.892+0000 7f3239b2c800 1 -- 172.21.15.28:0/3956742343 wait complete. 2024-09-19T11:23:44.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.907+0000 7f3239b2c800 1 Processor -- start 2024-09-19T11:23:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.907+0000 7f3239b2c800 1 -- start start 2024-09-19T11:23:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.907+0000 7f3239b2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.907+0000 7f3239b2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560d7e992dd0 con 0x560d7eafaf20 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f323573e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f323573e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53446/0 (socket says 172.21.15.28:53446) 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f323573e640 1 -- 172.21.15.28:0/1938800615 learned_addr learned my addr 172.21.15.28:0/1938800615 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f323573e640 1 -- 172.21.15.28:0/1938800615 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7eafac80 con 0x560d7eafaf20 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f323573e640 1 --2- 172.21.15.28:0/1938800615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafa690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3228025da0 tx=0x7f3228051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfd7d64e2746264 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f3123606640 1 -- 172.21.15.28:0/1938800615 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3228051950 con 0x560d7eafaf20 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f3123606640 1 -- 172.21.15.28:0/1938800615 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3228051af0 con 0x560d7eafaf20 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.908+0000 7f3123606640 1 -- 172.21.15.28:0/1938800615 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3228051dd0 con 0x560d7eafaf20 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 -- 172.21.15.28:0/1938800615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 msgr2=0x560d7eafa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 --2- 172.21.15.28:0/1938800615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafa690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3228025da0 tx=0x7f3228051000 comp rx=0 tx=0).stop 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 -- 172.21.15.28:0/1938800615 shutdown_connections 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 --2- 172.21.15.28:0/1938800615 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7eafa690 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 -- 172.21.15.28:0/1938800615 >> 172.21.15.28:0/1938800615 conn(0x560d7eaf4a60 msgr2=0x560d7ebe5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 -- 172.21.15.28:0/1938800615 shutdown_connections 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 -- 172.21.15.28:0/1938800615 wait complete. 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 Processor -- start 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.909+0000 7f3239b2c800 1 -- start start 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f3239b2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7ebf3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f3239b2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560d7eb01aa0 con 0x560d7eafaf20 2024-09-19T11:23:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f323573e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7ebf3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f323573e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7ebf3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53450/0 (socket says 172.21.15.28:53450) 2024-09-19T11:23:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f323573e640 1 -- 172.21.15.28:0/2270631165 learned_addr learned my addr 172.21.15.28:0/2270631165 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f323573e640 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7ebf43f0 con 0x560d7eafaf20 2024-09-19T11:23:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f323573e640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7ebf3e90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3228067040 tx=0x7f3228002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.910+0000 7f3121e03640 1 -- 172.21.15.28:0/2270631165 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3228073070 con 0x560d7eafaf20 2024-09-19T11:23:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.911+0000 7f3121e03640 1 -- 172.21.15.28:0/2270631165 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f322805a450 con 0x560d7eafaf20 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.911+0000 7f3121e03640 1 -- 172.21.15.28:0/2270631165 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f322805a730 con 0x560d7eafaf20 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.911+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560d7ebf4680 con 0x560d7eafaf20 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.911+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7ebf5290 con 0x560d7eafaf20 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.911+0000 7f3121e03640 1 -- 172.21.15.28:0/2270631165 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3228076070 con 0x560d7eafaf20 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3121e03640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311c03cfb0 0x7f311c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3121e03640 1 -- 172.21.15.28:0/2270631165 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3228062070 con 0x560d7eafaf20 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3234f3d640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311c03cfb0 0x7f311c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7eafd8b0 0x560d7ebfb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ebfba20 con 0x560d7eafd8b0 2024-09-19T11:23:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3235f3f640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7eafd8b0 0x560d7ebfb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3234f3d640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311c03cfb0 0x7f311c03f470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f322c001e30 tx=0x7f322c001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.912+0000 7f3235f3f640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7eafd8b0 0x560d7ebfb4e0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.913+0000 7f3235f3f640 1 -- 172.21.15.28:0/2270631165 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7eafd8b0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.913+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ebfbdc0 con 0x560d7eafd8b0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.913+0000 7f3235f3f640 1 -- 172.21.15.28:0/2270631165 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7eafd8b0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.913+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ebfd880 0x560d7ebffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.913+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec004b0 con 0x560d7ebfd880 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.913+0000 7f323573e640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ebfd880 0x560d7ebffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.913+0000 7f323573e640 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ebfd880 0x560d7ebffd20 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.914+0000 7f323573e640 1 -- 172.21.15.28:0/2270631165 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f322807f070 con 0x560d7ebfd880 2024-09-19T11:23:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.914+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ebfc610 con 0x560d7eafd8b0 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.914+0000 7f3235f3f640 1 -- 172.21.15.28:0/2270631165 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7eafd8b0 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.914+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ebfca30 con 0x560d7ebfd880 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.914+0000 7f323573e640 1 -- 172.21.15.28:0/2270631165 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f322807f070 con 0x560d7ebfd880 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3239b2c800 1 Processor -- start 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3239b2c800 1 -- start start 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3239b2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec0ee40 0x560d7ec0f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3239b2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560d7e9f14a0 con 0x560d7ec0ee40 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3235f3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec0ee40 0x560d7ec0f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3235f3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec0ee40 0x560d7ec0f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53462/0 (socket says 172.21.15.28:53462) 2024-09-19T11:23:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3235f3f640 1 -- 172.21.15.28:0/3659849256 learned_addr learned my addr 172.21.15.28:0/3659849256 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.915+0000 7f3235f3f640 1 -- 172.21.15.28:0/3659849256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7ec0f8a0 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.916+0000 7f3235f3f640 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec0ee40 0x560d7ec0f260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x560d7ebf4d20 tx=0x7f32300167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.916+0000 7f311affd640 1 -- 172.21.15.28:0/3659849256 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3230013640 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.916+0000 7f311affd640 1 -- 172.21.15.28:0/3659849256 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32300137e0 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.916+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560d7ec100d0 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.916+0000 7f311affd640 1 -- 172.21.15.28:0/3659849256 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3230013ae0 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.916+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7ec10670 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f311affd640 1 -- 172.21.15.28:0/3659849256 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3230011040 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f311affd640 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311003cfb0 0x7f311003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f311affd640 1 -- 172.21.15.28:0/3659849256 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f323000e040 con 0x560d7ec0ee40 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f323573e640 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311003cfb0 0x7f311003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f3239b2c800 1 Processor -- start 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f3239b2c800 1 -- start start 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f323573e640 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311003cfb0 0x7f311003f470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x560d7ec00880 tx=0x7f322806f8a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f3239b2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.917+0000 7f3239b2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560d7eaf9010 con 0x560d7ec1b900 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3235f3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3235f3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec1bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53466/0 (socket says 172.21.15.28:53466) 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3235f3f640 1 -- 172.21.15.28:0/3162372244 learned_addr learned my addr 172.21.15.28:0/3162372244 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3235f3f640 1 -- 172.21.15.28:0/3162372244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7ec1c2f0 con 0x560d7ec1b900 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3235f3f640 1 --2- 172.21.15.28:0/3162372244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec1bd00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3230054840 tx=0x7f32300204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfc2d670c218637 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3118ff9640 1 -- 172.21.15.28:0/3162372244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3230051020 con 0x560d7ec1b900 2024-09-19T11:23:44.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3118ff9640 1 -- 172.21.15.28:0/3162372244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3230021440 con 0x560d7ec1b900 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.918+0000 7f3118ff9640 1 -- 172.21.15.28:0/3162372244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f323001f340 con 0x560d7ec1b900 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 -- 172.21.15.28:0/3162372244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 msgr2=0x560d7ec1bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3162372244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec1bd00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3230054840 tx=0x7f32300204d0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 -- 172.21.15.28:0/3162372244 shutdown_connections 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3162372244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec1bd00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 -- 172.21.15.28:0/3162372244 >> 172.21.15.28:0/3162372244 conn(0x560d7ec17d00 msgr2=0x560d7ec15f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 -- 172.21.15.28:0/3162372244 shutdown_connections 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 -- 172.21.15.28:0/3162372244 wait complete. 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 Processor -- start 2024-09-19T11:23:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 -- start start 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec22ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3239b2c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560d7ec1c910 con 0x560d7ec1b900 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3235f3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec22ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3235f3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec22ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53480/0 (socket says 172.21.15.28:53480) 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.919+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 learned_addr learned my addr 172.21.15.28:0/3805764106 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.920+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7ec23440 con 0x560d7ec1b900 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.920+0000 7f3235f3f640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec22ee0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f323002bfd0 tx=0x7f3230045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.920+0000 7f3106ffd640 1 -- 172.21.15.28:0/3805764106 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32300459e0 con 0x560d7ec1b900 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.920+0000 7f3106ffd640 1 -- 172.21.15.28:0/3805764106 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3230045b80 con 0x560d7ec1b900 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.920+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560d7ec236d0 con 0x560d7ec1b900 2024-09-19T11:23:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.920+0000 7f3106ffd640 1 -- 172.21.15.28:0/3805764106 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3230054950 con 0x560d7ec1b900 2024-09-19T11:23:44.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.920+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7ec236d0 con 0x560d7ec1b900 2024-09-19T11:23:44.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.921+0000 7f3106ffd640 1 -- 172.21.15.28:0/3805764106 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3230054af0 con 0x560d7ec1b900 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.921+0000 7f3106ffd640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f310003cfb0 0x7f310003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.921+0000 7f3106ffd640 1 -- 172.21.15.28:0/3805764106 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3230034d50 con 0x560d7ec1b900 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.921+0000 7f323573e640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f310003cfb0 0x7f310003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.922+0000 7f323573e640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f310003cfb0 0x7f310003f470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3228050830 tx=0x7f322808b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.922+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.922+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7ec2e600 0x560d7ec30ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.922+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec31210 con 0x560d7ec2e600 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.922+0000 7f3234f3d640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7ec2e600 0x560d7ec30ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.922+0000 7f3234f3d640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7ec2e600 0x560d7ec30ac0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f322c00f610 con 0x560d7ec2e600 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 realm 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec2d950 con 0x560d7ec2e600 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f322c00f610 con 0x560d7ec2e600 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec31d00 con 0x560d7ec2e600 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f322c00f610 con 0x560d7ec2e600 2024-09-19T11:23:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec325d0 con 0x560d7ec2e600 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f322c00f610 con 0x560d7ec2e600 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ec32ff0 0x560d7ec353e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec35b20 con 0x560d7ec32ff0 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.923+0000 7f3235f3f640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ec32ff0 0x560d7ec353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3235f3f640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ec32ff0 0x560d7ec353e0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3239b2c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3239b2c800 20 searching for the correct realm 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3235f3f640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f323003c3f0 0x7f32300230c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.924+0000 7f323573e640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f323003c3f0 0x7f32300230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.925+0000 7f323573e640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f323003c3f0 0x7f32300230c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.925+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.925+0000 7f323573e640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f32280a6040 0x7f322806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.925+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f32280a6040 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.925+0000 7f3234f3d640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f32280a6040 0x7f322806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.926+0000 7f3234f3d640 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f32280a6040 0x7f322806e3c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.926+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c00f610 con 0x7f32280a6040 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.926+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.926+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c00f610 con 0x560d7ec2e600 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.926+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.926+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f32280a6040 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x7f32280a6040 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f32280a6040 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x7f32280a6040 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.927+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f32280a6040 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x7f32280a6040 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.928+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.929+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f32280a6040 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x7f32280a6040 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.930+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.931+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.931+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.931+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.931+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.931+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.931+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f32280a6040 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x7f32280a6040 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f32280a6040 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322c02b040 con 0x7f32280a6040 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.932+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec32ff0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x560d7ec2e600 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37450 con 0x7f323003c3f0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.933+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec36510 con 0x560d7ec2e600 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec36d60 con 0x560d7ec2e600 2024-09-19T11:23:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f322c02b040 con 0x560d7ec2e600 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec378d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 zone default found 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 4 Realm: () 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 zonegroup default 2024-09-19T11:23:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.934+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec37cb0 con 0x7f32280a6040 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f322c02b040 con 0x7f32280a6040 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec39f90 con 0x560d7ec32ff0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3a330 con 0x560d7ec32ff0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 20 started sync module instance, tier type = 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.935+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec39420 con 0x7f323003c3f0 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.936+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.936+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94615961311296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec39080 con 0x7f323003c3f0 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.936+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3b230 con 0x560d7ec2e600 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.937+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94615961311296] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.938+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.938+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94615961316736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3b650 con 0x560d7ec2e600 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.938+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3c010 con 0x560d7ec32ff0 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.939+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94615961316736] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.939+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.939+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94615961318272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3c010 con 0x560d7ec32ff0 2024-09-19T11:23:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.939+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3ca10 con 0x7f323003c3f0 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.940+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94615961318272] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.940+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.940+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94615961320288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3bf80 con 0x7f323003c3f0 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.940+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3d8d0 con 0x560d7ec2e600 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.941+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94615961320288] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.941+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.941+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94615961326624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3d840 con 0x560d7ec2e600 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.941+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3e790 con 0x7f32280a6040 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.942+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94615961326624] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.942+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.942+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94615961328752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3e790 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.942+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3f1a0 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.943+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94615961328752] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.943+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.943+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94615961330880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3f1a0 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.943+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3fb60 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.944+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94615961330880] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.944+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.944+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94615961333248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3fb60 con 0x7f32280a6040 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=0 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=1 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=2 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=3 2024-09-19T11:23:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=4 2024-09-19T11:23:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=5 2024-09-19T11:23:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=6 2024-09-19T11:23:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94615961333248] v33'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 20 add_watcher() i=7 2024-09-19T11:23:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.945+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec42f60 con 0x560d7ec2e600 2024-09-19T11:23:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.946+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.946+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94615961335888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec40480 con 0x560d7ec2e600 2024-09-19T11:23:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.947+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94615961335888] v33'182 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.947+0000 7f30eb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.947+0000 7f3239b2c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.947+0000 7f3239b2c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.947+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4b7a0 con 0x560d7ec2e600 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.948+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.948+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4be30 con 0x7f32280a6040 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.949+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.949+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4c270 con 0x560d7ec32ff0 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.950+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.950+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4c200 con 0x560d7ec2e600 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.951+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'183 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.951+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4c220 con 0x7f323003c3f0 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.952+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'62 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.952+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4c870 con 0x560d7ec32ff0 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.953+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'41 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.953+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4c810 con 0x560d7ec2e600 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.954+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.954+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4c7b0 con 0x7f32280a6040 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.955+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'41 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.955+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4c7b0 con 0x7f323003c3f0 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.956+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'25 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.956+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd60 con 0x560d7ec32ff0 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.957+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.957+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cc60 con 0x7f323003c3f0 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.958+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'63 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.958+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cc60 con 0x7f32280a6040 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.960+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.960+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cc60 con 0x7f323003c3f0 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.961+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.961+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cc60 con 0x7f32280a6040 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.962+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'48 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.962+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cc60 con 0x560d7ec2e600 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.963+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'184 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.963+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cc60 con 0x560d7ec32ff0 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.964+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'56 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.964+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4ccf0 con 0x7f323003c3f0 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.965+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'64 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.965+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4ccf0 con 0x560d7ec32ff0 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.966+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.966+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4ccf0 con 0x560d7ec2e600 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.967+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'32 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.967+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4ccf0 con 0x560d7ec2e600 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.968+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'185 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.968+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd10 con 0x560d7ec32ff0 2024-09-19T11:23:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.969+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'57 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.969+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd10 con 0x7f323003c3f0 2024-09-19T11:23:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.970+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'34 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.970+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd50 con 0x560d7ec32ff0 2024-09-19T11:23:44.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.971+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.971+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4ccf0 con 0x560d7ec32ff0 2024-09-19T11:23:44.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.972+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.972+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd30 con 0x560d7ec32ff0 2024-09-19T11:23:44.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.973+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'20 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.973+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd30 con 0x560d7ec32ff0 2024-09-19T11:23:44.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.974+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'42 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.974+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cdb0 con 0x7f32280a6040 2024-09-19T11:23:44.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.976+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'28 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.976+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd70 con 0x560d7ec32ff0 2024-09-19T11:23:44.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.977+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.977+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd70 con 0x7f32280a6040 2024-09-19T11:23:44.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.978+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'42 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.978+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cdb0 con 0x560d7ec32ff0 2024-09-19T11:23:44.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.979+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'58 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.979+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd70 con 0x7f32280a6040 2024-09-19T11:23:44.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.980+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'49 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.980+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec4cd70 con 0x7f323003c3f0 2024-09-19T11:23:44.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f30ea7e4640 20 reqs_thread_entry: start 2024-09-19T11:23:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f3239b2c800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f3239b2c800 20 Filter name: none 2024-09-19T11:23:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f305bfff640 20 reqs_thread_entry: start 2024-09-19T11:23:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f3239b2c800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.981+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec77e10 con 0x7f323003c3f0 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 : miss 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:95 8.14 8:2e2e7a97:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec796f0 con 0x560d7ec32ff0 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(95 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:23:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 info.flags=0x17 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 to cache LRU end 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.982+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 4.2 4:43d0b650:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf6be0 con 0x560d7ec2e600 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.983+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(96 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.983+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:97 4.1 4:80c4f73b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf6be0 con 0x7f323003c3f0 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.983+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(97 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.983+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 4.a 4:50db715b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf6be0 con 0x7f323003c3f0 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.983+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(98 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2 [call out=430b] v0'0 uv3 ondisk = 0) ==== 194+0+430 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.983+0000 7f3239b2c800 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-09-19T11:23:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.983+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:99 4.3f 4:fec15e8a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf6c20 con 0x7f323003c3f0 2024-09-19T11:23:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.984+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(99 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.984+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:100 4.c 4:314cf4ce:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf71c0 con 0x560d7ec2e600 2024-09-19T11:23:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.984+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(100 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.984+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:101 4.1a 4:5bf4a3f6:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf71c0 con 0x7f323003c3f0 2024-09-19T11:23:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.984+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(101 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.984+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:102 4.15 4:a869d901:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf71c0 con 0x560d7ec32ff0 2024-09-19T11:23:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.985+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(102 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6 [call out=429b] v0'0 uv7 ondisk = 0) ==== 194+0+429 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.985+0000 7f3239b2c800 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-09-19T11:23:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.985+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:103 4.39 4:9da88f2d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf71c0 con 0x560d7ec32ff0 2024-09-19T11:23:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.985+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(103 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.985+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:104 4.24 4:25e6d5fd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf71c0 con 0x7f32280a6040 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(104 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8 [call out=429b] v0'0 uv5 ondisk = 0) ==== 194+0+429 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3239b2c800 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:105 4.18 4:1b58c098:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf7180 con 0x7f323003c3f0 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(105 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:106 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf7180 con 0x7f32280a6040 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(106 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [call out=702b] v0'0 uv9 ondisk = 0) ==== 195+0+702 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3239b2c800 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3239b2c800 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.986+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:107 3.14s0 3:29d050a2:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_9c8454ddf7aa50116496bac348d7550d:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ecf47a0 con 0x560d7ec32ff0 2024-09-19T11:23:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.987+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(107 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_9c8454ddf7aa50116496bac348d7550d [call out=24b,getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 304+0+919 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.987+0000 7f3239b2c800 10 manifest: total_size = 6 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.987+0000 7f3239b2c800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.14445594459200602719 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.987+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.987+0000 7f3239b2c800 20 prepare_atomic_modification: state is not atomic. state=0x560d7ec79f78 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.987+0000 7f3239b2c800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.987+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:108 4.a 4:50db715b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7ee80 con 0x7f323003c3f0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.988+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(108 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2 [stat,call,call] v33'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.988+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:109 3.14s0 3:29d050a2:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7ee80 con 0x560d7ec32ff0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.989+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(109 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_9c8454ddf7aa50116496bac348d7550d [call] v33'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:110 4.a 4:50db715b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7f910 con 0x7f323003c3f0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:111 3.2es0 3:762dc260:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_b1b9a972ccd8c962a473909b97007eb4:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f323007a000 con 0x7f323003c3f0 2024-09-19T11:23:44.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(111 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_b1b9a972ccd8c962a473909b97007eb4 [call out=24b,getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 304+0+917 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 10 manifest: total_size = 3 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.1168607578603562857 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 20 prepare_atomic_modification: state is not atomic. state=0x560d7ec79f78 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.990+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:112 4.15 4:a869d901:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec845c0 con 0x560d7ec32ff0 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.991+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(110 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2 [stat,call,call] v33'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.991+0000 7f3118ff9640 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.991+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(112 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6 [stat,call,call] v33'8 uv8 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.991+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:113 3.2es0 3:762dc260:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7c910 con 0x7f323003c3f0 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.992+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(113 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_b1b9a972ccd8c962a473909b97007eb4 [call] v33'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:44.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.992+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:114 4.15 4:a869d901:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec80580 con 0x560d7ec32ff0 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.992+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.992+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:115 3.1fs0 3:fb98ff6d:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f32280b6000 con 0x560d7ec2e600 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(115 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_c785e1ed2950e3e36b1e2ca01f299a54 [call out=24b,getxattrs out=877b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 304+0+917 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3239b2c800 10 manifest: total_size = 4 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3239b2c800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.6366666987280743110 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3239b2c800 20 prepare_atomic_modification: state is not atomic. state=0x560d7ec79f78 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3239b2c800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:116 4.24 4:25e6d5fd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec841b0 con 0x7f32280a6040 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.993+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(114 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6 [stat,call,call] v33'9 uv9 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.994+0000 7f3107fff640 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-09-19T11:23:44.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.994+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(116 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8 [stat,call,call] v33'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.994+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:117 3.1fs0 3:fb98ff6d:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec84770 con 0x560d7ec2e600 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.995+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(117 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v33'4 uv4 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.995+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:118 4.24 4:25e6d5fd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec83d00 con 0x7f32280a6040 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.995+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.995+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:119 3.1es0 3:78c6277a:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_24d27c169c2c881eb09a065116f2aa5c:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f322c042000 con 0x7f32280a6040 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.996+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(119 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_24d27c169c2c881eb09a065116f2aa5c [call out=24b,getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 304+0+919 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.996+0000 7f3239b2c800 10 manifest: total_size = 5 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.996+0000 7f3239b2c800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.14901568037757912131 2024-09-19T11:23:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.996+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.996+0000 7f3239b2c800 20 prepare_atomic_modification: state is not atomic. state=0x560d7ec79f78 2024-09-19T11:23:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.996+0000 7f3239b2c800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 2024-09-19T11:23:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.996+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:120 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7e0f0 con 0x7f32280a6040 2024-09-19T11:23:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.997+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(118 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8 [stat,call,call] v33'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.997+0000 7f3118ff9640 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-09-19T11:23:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.997+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(120 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [stat,call,call] v33'10 uv10 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.997+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:121 3.1es0 3:78c6277a:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7e570 con 0x7f32280a6040 2024-09-19T11:23:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.998+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(121 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_24d27c169c2c881eb09a065116f2aa5c [call] v33'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.998+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:122 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7e570 con 0x7f32280a6040 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.998+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.998+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:123 3.2as0 3:5576cde3:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f322c043000 con 0x560d7ec2e600 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(123 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432 [call out=24b,getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 304+0+950 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3239b2c800 10 manifest: total_size = 5 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3239b2c800 20 get_obj_state: setting s->obj_tag to _m2gLGlux3KHWMQDgk1QE0nUgPvBea1Q 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3239b2c800 20 get_obj_state: octx=0x7f3228065e80 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x560d7ec79f78 s->prefetch_data=0 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3239b2c800 20 prepare_atomic_modification: state is not atomic. state=0x560d7ec79f78 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3239b2c800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:124 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec86bc0 con 0x7f32280a6040 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(122 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [stat,call,call] v33'11 uv11 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:44.999+0000 7f3107fff640 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-09-19T11:23:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.000+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(124 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [stat,call,call] v33'12 uv12 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.000+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:125 3.2as0 3:5576cde3:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec85bb0 con 0x560d7ec2e600 2024-09-19T11:23:45.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(125 293899f1-2738-4b64-a8e1-a18ff039d626.4229.5_bb3aec0fdcdbc2974890f805c585d432 [call] v33'4 uv4 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:126 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7b900 con 0x7f32280a6040 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 20 list_objects_ordered: starting attempt 1 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:127 4.2 4:43d0b650:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7b900 con 0x560d7ec2e600 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:128 4.1 4:80c4f73b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec79d90 con 0x7f323003c3f0 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:129 4.a 4:50db715b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7bff0 con 0x7f323003c3f0 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:130 4.3f 4:fec15e8a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7c8e0 con 0x7f323003c3f0 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:131 4.c 4:314cf4ce:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7bff0 con 0x560d7ec2e600 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:132 4.1a 4:5bf4a3f6:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7c8e0 con 0x7f323003c3f0 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:133 4.15 4:a869d901:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec93090 con 0x560d7ec32ff0 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:134 4.39 4:9da88f2d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec95be0 con 0x560d7ec32ff0 2024-09-19T11:23:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.001+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:135 4.24 4:25e6d5fd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec98a60 con 0x7f32280a6040 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:136 4.18 4:1b58c098:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b470 con 0x7f323003c3f0 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:137 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x7f32280a6040 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(135 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8 [call out=128b] v0'0 uv7 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(133 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6 [call out=128b] v0'0 uv9 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== osd_op_reply(128 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(127 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(129 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2 [call out=128b] v0'0 uv5 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(131 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(132 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== osd_op_reply(130 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.002+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(136 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(126 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [stat,call,call] v33'13 uv13 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3107fff640 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(134 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(137 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [call out=128b] v0'0 uv13 ondisk = 0) ==== 195+0+128 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.5])"; uploads.size()=0; is_truncated=0 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:138 4.2 4:43d0b650:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b4d0 con 0x560d7ec2e600 2024-09-19T11:23:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:139 4.1 4:80c4f73b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec95be0 con 0x7f323003c3f0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:140 4.a 4:50db715b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x7f323003c3f0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:141 4.3f 4:fec15e8a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b4d0 con 0x7f323003c3f0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:142 4.c 4:314cf4ce:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec95be0 con 0x560d7ec2e600 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:143 4.1a 4:5bf4a3f6:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b4d0 con 0x7f323003c3f0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:144 4.15 4:a869d901:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec95be0 con 0x560d7ec32ff0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:145 4.39 4:9da88f2d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x560d7ec32ff0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:146 4.24 4:25e6d5fd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec985d0 con 0x7f32280a6040 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:147 4.18 4:1b58c098:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec95be0 con 0x7f323003c3f0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(138 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:148 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x7f32280a6040 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(139 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(142 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 36 ==== osd_op_reply(140 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2 [call out=128b] v0'0 uv5 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.003+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 37 ==== osd_op_reply(141 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.004+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 38 ==== osd_op_reply(143 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.004+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(144 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6 [call out=128b] v0'0 uv9 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.004+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 38 ==== osd_op_reply(145 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.004+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(148 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [call out=128b] v0'0 uv13 ondisk = 0) ==== 195+0+128 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.004+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(146 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8 [call out=128b] v0'0 uv7 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.004+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 39 ==== osd_op_reply(147 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.004+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:149 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x7f323003c3f0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.005+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 40 ==== osd_op_reply(149 foo.buckets [call] v33'37 uv37 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.005+0000 7f3239b2c800 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x11) 2024-09-19T11:23:45.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.005+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:150 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b4d0 con 0x7f323003c3f0 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 41 ==== osd_op_reply(150 myfoo [call,call,delete] v33'38 uv38 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3239b2c800 10 removing default.rgw.meta+root+myfoo from cache 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3239b2c800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:151 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94615961711696 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x7f32280a6040 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== watch-notify(notify (1) cookie 94615961330880 notify 141733920791 ret 0) ==== 191+0+0 (crc 0 0 0) 0x560d7ec4be30 con 0x7f32280a6040 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3118ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920791 cookie 94615961330880 notifier 4649 bl.length()=149 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(151 notify.6 [notify cookie 94615961711696 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3118ff9640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:152 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f30f8003d60 con 0x7f32280a6040 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== osd_op_reply(152 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.006+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== watch-notify(notify_complete (2) cookie 94615961711696 notify 141733920791 ret 0) ==== 42+0+48 (crc 0 0 0) 0x560d7ec4b760 con 0x7f32280a6040 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.007+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:153 8.14 8:2e2e7a97:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b450 con 0x560d7ec32ff0 2024-09-19T11:23:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 39 ==== osd_op_reply(153 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 [call,call,delete] v33'4 uv4 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3239b2c800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 from cache 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3239b2c800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.5, ofs0, ns] 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:154 7.3 7:c609908c:::notify.5:head [notify cookie 94615961722464 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b450 con 0x7f32280a6040 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== watch-notify(notify (1) cookie 94615961328752 notify 141733920792 ret 0) ==== 248+0+0 (crc 0 0 0) 0x7f322c055040 con 0x7f32280a6040 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3118ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920792 cookie 94615961328752 notifier 4649 bl.length()=206 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(154 notify.5 [notify cookie 94615961722464 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3118ff9640 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:155 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f30f8003d60 con 0x7f32280a6040 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== watch-notify(notify_complete (2) cookie 94615961722464 notify 141733920792 ret 0) ==== 42+0+48 (crc 0 0 0) 0x560d7ec4b760 con 0x7f32280a6040 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(155 notify.5 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:156 4.2 4:43d0b650:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b450 con 0x560d7ec2e600 2024-09-19T11:23:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:157 4.1 4:80c4f73b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x7f323003c3f0 2024-09-19T11:23:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:158 4.a 4:50db715b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec79e20 con 0x7f323003c3f0 2024-09-19T11:23:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.008+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:159 4.3f 4:fec15e8a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7d150 con 0x7f323003c3f0 2024-09-19T11:23:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.009+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:160 4.c 4:314cf4ce:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7da20 con 0x560d7ec2e600 2024-09-19T11:23:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.009+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:161 4.1a 4:5bf4a3f6:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec76e00 con 0x7f323003c3f0 2024-09-19T11:23:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.009+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:162 4.15 4:a869d901:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x560d7ec32ff0 2024-09-19T11:23:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.009+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:163 4.39 4:9da88f2d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec79e20 con 0x560d7ec32ff0 2024-09-19T11:23:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.009+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:164 4.24 4:25e6d5fd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7d150 con 0x7f32280a6040 2024-09-19T11:23:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.009+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:165 4.18 4:1b58c098:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9b450 con 0x7f323003c3f0 2024-09-19T11:23:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.009+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:166 4.26 4:67f24c2b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec76e00 con 0x7f32280a6040 2024-09-19T11:23:45.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.010+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== osd_op_reply(164 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.8 [delete] v33'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.010+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== osd_op_reply(166 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.10 [delete] v33'14 uv14 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 42 ==== osd_op_reply(161 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.5 [delete] v33'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 40 ==== osd_op_reply(162 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.6 [delete] v33'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 43 ==== osd_op_reply(165 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.9 [delete] v33'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 41 ==== osd_op_reply(163 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.7 [delete] v33'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(156 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.0 [delete] v33'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 44 ==== osd_op_reply(157 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.1 [delete] v33'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(160 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.4 [delete] v33'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 45 ==== osd_op_reply(158 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.2 [delete] v33'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 46 ==== osd_op_reply(159 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.5.3 [delete] v33'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3239b2c800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 : miss 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3239b2c800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:167 6.15 6:af91dda8:::pubsub..bucket.myfoo%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec79e20 con 0x7f323003c3f0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 47 ==== osd_op_reply(167 pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3239b2c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3239b2c800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 info.flags=0x0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3239b2c800 10 adding default.rgw.log++pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 to cache LRU end 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.011+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:168 6.15 6:af91dda8:::pubsub..bucket.myfoo%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec9de60 con 0x7f323003c3f0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.012+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 48 ==== osd_op_reply(168 pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.5 [delete] v33'25 uv21 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.012+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:169 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7d150 con 0x7f323003c3f0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.013+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 49 ==== osd_op_reply(169 foo.buckets [call] v33'39 uv39 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.014+0000 7f30eb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.014+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:170 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94615961335888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec7da20 con 0x560d7ec2e600 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.015+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(170 data_loggenerations_metadata [watch unwatch cookie 94615961335888] v33'186 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.015+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:171 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94615961311296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3f820 con 0x7f323003c3f0 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.016+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 50 ==== osd_op_reply(171 notify.0 [watch unwatch cookie 94615961311296] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.016+0000 7f3239b2c800 20 remove_watcher() i=0 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.016+0000 7f3239b2c800 2 removed watcher, disabling cache 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.016+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:172 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94615961316736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec39840 con 0x560d7ec2e600 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.017+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(172 notify.1 [watch unwatch cookie 94615961316736] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.017+0000 7f3239b2c800 20 remove_watcher() i=1 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.017+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:173 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94615961318272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3ad80 con 0x560d7ec32ff0 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.018+0000 7f3235f3f640 1 -- 172.21.15.28:0/3805764106 <== osd.3 v2:172.21.15.28:6808/3027134945 42 ==== osd_op_reply(173 notify.2 [watch unwatch cookie 94615961318272] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f323000a4d0 con 0x560d7ec32ff0 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.018+0000 7f3239b2c800 20 remove_watcher() i=2 2024-09-19T11:23:45.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.018+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:174 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94615961320288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3ad80 con 0x7f323003c3f0 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.019+0000 7f323573e640 1 -- 172.21.15.28:0/3805764106 <== osd.2 v2:172.21.15.28:6800/2071226075 51 ==== osd_op_reply(174 notify.3 [watch unwatch cookie 94615961320288] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322807f070 con 0x7f323003c3f0 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.019+0000 7f3239b2c800 20 remove_watcher() i=3 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.019+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:175 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94615961326624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3ad80 con 0x560d7ec2e600 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.020+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.1 v2:172.21.15.28:6816/656119574 41 ==== osd_op_reply(175 notify.4 [watch unwatch cookie 94615961326624] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x560d7ec2e600 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.020+0000 7f3239b2c800 20 remove_watcher() i=4 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.020+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:176 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94615961328752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3d420 con 0x7f32280a6040 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.021+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 46 ==== osd_op_reply(176 notify.5 [watch unwatch cookie 94615961328752] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.021+0000 7f3239b2c800 20 remove_watcher() i=5 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.021+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:177 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94615961330880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3ad80 con 0x7f32280a6040 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.023+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 47 ==== osd_op_reply(177 notify.6 [watch unwatch cookie 94615961330880] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.023+0000 7f3239b2c800 20 remove_watcher() i=6 2024-09-19T11:23:45.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.023+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:178 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94615961333248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560d7ec3d420 con 0x7f32280a6040 2024-09-19T11:23:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 <== osd.0 v2:172.21.15.22:6800/3144434255 48 ==== osd_op_reply(178 notify.7 [watch unwatch cookie 94615961333248] v33'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f322c03b040 con 0x7f32280a6040 2024-09-19T11:23:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 20 remove_watcher() i=7 2024-09-19T11:23:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f32280a6040 msgr2=0x7f322806e3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f32280a6040 0x7f322806e3c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7ec2e600 msgr2=0x560d7ec30ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7ec2e600 0x560d7ec30ac0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f323003c3f0 msgr2=0x7f32300230c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f323003c3f0 0x7f32300230c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ec32ff0 msgr2=0x560d7ec353e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ec32ff0 0x560d7ec353e0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f310003cfb0 msgr2=0x7f310003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f310003cfb0 0x7f310003f470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3228050830 tx=0x7f322808b000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 msgr2=0x560d7ec22ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3234f3d640 1 -- 172.21.15.28:0/3805764106 reap_dead start 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec22ee0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f323002bfd0 tx=0x7f3230045000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 shutdown_connections 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3805764106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec1b900 0x560d7ec22ee0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 >> 172.21.15.28:0/3805764106 conn(0x560d7ec17d00 msgr2=0x560d7ec15f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 shutdown_connections 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.024+0000 7f3239b2c800 1 -- 172.21.15.28:0/3805764106 wait complete. 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311003cfb0 msgr2=0x7f311003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311003cfb0 0x7f311003f470 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x560d7ec00880 tx=0x7f322806f8a0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec0ee40 msgr2=0x560d7ec0f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec0ee40 0x560d7ec0f260 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x560d7ebf4d20 tx=0x7f32300167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 shutdown_connections 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311003cfb0 0x7f311003f470 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/3659849256 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7ec0ee40 0x560d7ec0f260 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 >> 172.21.15.28:0/3659849256 conn(0x560d7ec031f0 msgr2=0x560d7ec035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 shutdown_connections 2024-09-19T11:23:45.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/3659849256 wait complete. 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7eafd8b0 msgr2=0x560d7ebfb4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7eafd8b0 0x560d7ebfb4e0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ebfd880 msgr2=0x560d7ebffd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ebfd880 0x560d7ebffd20 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311c03cfb0 msgr2=0x7f311c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311c03cfb0 0x7f311c03f470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f322c001e30 tx=0x7f322c001390 comp rx=0 tx=0).stop 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 msgr2=0x560d7ebf3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7ebf3e90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f3228067040 tx=0x7f3228002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 shutdown_connections 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560d7ebfd880 0x560d7ebffd20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f311c03cfb0 0x7f311c03f470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560d7eafd8b0 0x560d7ebfb4e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 --2- 172.21.15.28:0/2270631165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560d7eafaf20 0x560d7ebf3e90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 >> 172.21.15.28:0/2270631165 conn(0x560d7eaf4a60 msgr2=0x560d7ebe5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 shutdown_connections 2024-09-19T11:23:45.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.025+0000 7f3239b2c800 1 -- 172.21.15.28:0/2270631165 wait complete. 2024-09-19T11:23:45.036 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-09-19T11:23:45.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-09-19T11:23:45.036 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-09-19T11:23:45.084 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:45.084 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc22939800 1 Processor -- start 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc22939800 1 -- start start 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc22939800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55dbdbbfcab0 0x55dbdbbfef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc22939800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55dbdbbff5f0 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc22939800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55dbdba837d0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc1ddc6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55dbdbbfcab0 0x55dbdbbfef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc1ddc6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55dbdbbfcab0 0x55dbdbbfef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53486/0 (socket says 172.21.15.28:53486) 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc1e5c7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55dbdbbf8eb0 0x55dbdbbfb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:55612/0 (socket says 172.21.15.28:55612) 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/482230850 learned_addr learned my addr 172.21.15.28:0/482230850 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/482230850 >> v1:172.21.15.22:6789/0 conn(0x55dbdbbf8eb0 legacy=0x55dbdbbfb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.084+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/482230850 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdbc00820 con 0x55dbdbbfcab0 2024-09-19T11:23:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.086+0000 7fbc1ddc6640 1 --2- 172.21.15.28:0/482230850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbfcab0 0x55dbdbbfef70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400a520 tx=0x7fbc14045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe077dc5c38a83f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.086+0000 7fbc1d5c5640 1 -- 172.21.15.28:0/482230850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc14055070 con 0x55dbdbbfcab0 2024-09-19T11:23:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.086+0000 7fbc1d5c5640 1 -- 172.21.15.28:0/482230850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc140515b0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.086+0000 7fbc1d5c5640 1 -- 172.21.15.28:0/482230850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc140518d0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.087+0000 7fbc22939800 1 -- 172.21.15.28:0/482230850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbfcab0 msgr2=0x55dbdbbfef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.087+0000 7fbc22939800 1 --2- 172.21.15.28:0/482230850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbfcab0 0x55dbdbbfef70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400a520 tx=0x7fbc14045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.087+0000 7fbc22939800 1 -- 172.21.15.28:0/482230850 shutdown_connections 2024-09-19T11:23:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.087+0000 7fbc22939800 1 --2- 172.21.15.28:0/482230850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbfcab0 0x55dbdbbfef70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.087+0000 7fbc22939800 1 -- 172.21.15.28:0/482230850 >> 172.21.15.28:0/482230850 conn(0x55dbdbbf2a70 msgr2=0x55dbdbbf4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.087+0000 7fbc22939800 1 -- 172.21.15.28:0/482230850 shutdown_connections 2024-09-19T11:23:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.087+0000 7fbc22939800 1 -- 172.21.15.28:0/482230850 wait complete. 2024-09-19T11:23:45.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.102+0000 7fbc22939800 1 Processor -- start 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.102+0000 7fbc22939800 1 -- start start 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.102+0000 7fbc22939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbbf8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.102+0000 7fbc22939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbdba90dd0 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.102+0000 7fbc1e5c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbbf8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.102+0000 7fbc1e5c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbbf8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53498/0 (socket says 172.21.15.28:53498) 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.102+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1468637634 learned_addr learned my addr 172.21.15.28:0/1468637634 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.103+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1468637634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdbbf8bf0 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.103+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/1468637634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbbf8600 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b260 tx=0x7fbc1000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a6bad291cd6bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.103+0000 7fbb0c608640 1 -- 172.21.15.28:0/1468637634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc1000be60 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.103+0000 7fbb0c608640 1 -- 172.21.15.28:0/1468637634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc10006070 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.103+0000 7fbb0c608640 1 -- 172.21.15.28:0/1468637634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc1000e710 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 -- 172.21.15.28:0/1468637634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 msgr2=0x55dbdbbf8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 --2- 172.21.15.28:0/1468637634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbbf8600 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000b260 tx=0x7fbc1000b660 comp rx=0 tx=0).stop 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 -- 172.21.15.28:0/1468637634 shutdown_connections 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 --2- 172.21.15.28:0/1468637634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbbf8600 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 -- 172.21.15.28:0/1468637634 >> 172.21.15.28:0/1468637634 conn(0x55dbdbbf2a70 msgr2=0x55dbdbce3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 -- 172.21.15.28:0/1468637634 shutdown_connections 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 -- 172.21.15.28:0/1468637634 wait complete. 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 Processor -- start 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 -- start start 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbcf2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc22939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbdbbffac0 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc1e5c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbcf2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.104+0000 7fbc1e5c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbcf2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53512/0 (socket says 172.21.15.28:53512) 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/3365994320 learned_addr learned my addr 172.21.15.28:0/3365994320 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdbcf25b0 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbcf2050 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55dbdbbf8130 tx=0x7fbc10000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbb0ae05640 1 -- 172.21.15.28:0/3365994320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc1000f810 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbb0ae05640 1 -- 172.21.15.28:0/3365994320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1000f9b0 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dbdbcf2840 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbb0ae05640 1 -- 172.21.15.28:0/3365994320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc1000fcd0 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.105+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdbcf3240 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.106+0000 7fbb0ae05640 1 -- 172.21.15.28:0/3365994320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbc10017070 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.106+0000 7fbb0ae05640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbb0403cfb0 0x7fbb0403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.106+0000 7fbb0ae05640 1 -- 172.21.15.28:0/3365994320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbc10015020 con 0x55dbdbbf8eb0 2024-09-19T11:23:45.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.106+0000 7fbc1ddc6640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbb0403cfb0 0x7fbb0403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.106+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbbfcab0 0x55dbdbcf96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.106+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbcf9be0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.107+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbbfcab0 0x55dbdbcf96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.107+0000 7fbc1ddc6640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbb0403cfb0 0x7fbb0403f470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc14057000 tx=0x7fbc14052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.107+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbbfcab0 0x55dbdbcf96a0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.107+0000 7fbc1edc8640 1 -- 172.21.15.28:0/3365994320 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.107+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbcf9f80 con 0x55dbdbbfcab0 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.107+0000 7fbc1edc8640 1 -- 172.21.15.28:0/3365994320 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.108+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbcfba80 0x55dbdbcfdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.108+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbcfe690 con 0x55dbdbcfba80 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.108+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbcfba80 0x55dbdbcfdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.108+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbcfba80 0x55dbdbcfdf00 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.108+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/3365994320 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbc1002e070 con 0x55dbdbcfba80 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.108+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbcfa7d0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.109+0000 7fbc1edc8640 1 -- 172.21.15.28:0/3365994320 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbbfcab0 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.109+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbcfabc0 con 0x55dbdbcfba80 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.109+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/3365994320 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbc1002e070 con 0x55dbdbcfba80 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.109+0000 7fbc22939800 1 Processor -- start 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.109+0000 7fbc22939800 1 -- start start 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.109+0000 7fbc22939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd0cfe0 0x55dbdbd0d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.109+0000 7fbc22939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbdbaef4a0 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbc1edc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd0cfe0 0x55dbdbd0d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbc1edc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd0cfe0 0x55dbdbd0d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53526/0 (socket says 172.21.15.28:53526) 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbc1edc8640 1 -- 172.21.15.28:0/3442390331 learned_addr learned my addr 172.21.15.28:0/3442390331 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbc1edc8640 1 -- 172.21.15.28:0/3442390331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdbd0da40 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd0cfe0 0x55dbdbd0d400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55dbdbcf3050 tx=0x7fbc180167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbb03fff640 1 -- 172.21.15.28:0/3442390331 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc18013640 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbb03fff640 1 -- 172.21.15.28:0/3442390331 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc180137e0 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbb03fff640 1 -- 172.21.15.28:0/3442390331 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc18013ae0 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dbdbd0e270 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.110+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdbd0e810 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.111+0000 7fbb03fff640 1 -- 172.21.15.28:0/3442390331 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbc18011040 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.111+0000 7fbb03fff640 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbaf803cfb0 0x7fbaf803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.111+0000 7fbb03fff640 1 -- 172.21.15.28:0/3442390331 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbc1800e040 con 0x55dbdbd0cfe0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.111+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbaf803cfb0 0x7fbaf803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc22939800 1 Processor -- start 2024-09-19T11:23:45.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc22939800 1 -- start start 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbaf803cfb0 0x7fbaf803f470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55dbdbcf4140 tx=0x7fbc10017c10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc22939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd19ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc22939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbdbbf7010 con 0x55dbdbd19aa0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc1edc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd19ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc1edc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd19ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53532/0 (socket says 172.21.15.28:53532) 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc1edc8640 1 -- 172.21.15.28:0/4004334449 learned_addr learned my addr 172.21.15.28:0/4004334449 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.112+0000 7fbc1edc8640 1 -- 172.21.15.28:0/4004334449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdbd1a490 con 0x55dbdbd19aa0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/4004334449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd19ea0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbc18054840 tx=0x7fbc180204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4728e320810b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbb01ffb640 1 -- 172.21.15.28:0/4004334449 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc18021a40 con 0x55dbdbd19aa0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbb01ffb640 1 -- 172.21.15.28:0/4004334449 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc18052070 con 0x55dbdbd19aa0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbb01ffb640 1 -- 172.21.15.28:0/4004334449 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc180215b0 con 0x55dbdbd19aa0 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc22939800 1 -- 172.21.15.28:0/4004334449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 msgr2=0x55dbdbd19ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc22939800 1 --2- 172.21.15.28:0/4004334449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd19ea0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbc18054840 tx=0x7fbc180204d0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc22939800 1 -- 172.21.15.28:0/4004334449 shutdown_connections 2024-09-19T11:23:45.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc22939800 1 --2- 172.21.15.28:0/4004334449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd19ea0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc22939800 1 -- 172.21.15.28:0/4004334449 >> 172.21.15.28:0/4004334449 conn(0x55dbdbd15ea0 msgr2=0x55dbdbd140e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc22939800 1 -- 172.21.15.28:0/4004334449 shutdown_connections 2024-09-19T11:23:45.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.113+0000 7fbc22939800 1 -- 172.21.15.28:0/4004334449 wait complete. 2024-09-19T11:23:45.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc22939800 1 Processor -- start 2024-09-19T11:23:45.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc22939800 1 -- start start 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc22939800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd21030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc22939800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbdbd1aa80 con 0x55dbdbd19aa0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc1edc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd21030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc1edc8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd21030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:53534/0 (socket says 172.21.15.28:53534) 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 learned_addr learned my addr 172.21.15.28:0/1054188050 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbdbd21590 con 0x55dbdbd19aa0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd21030 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbc18054740 tx=0x7fbc180547a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbaeffff640 1 -- 172.21.15.28:0/1054188050 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc18056070 con 0x55dbdbd19aa0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbaeffff640 1 -- 172.21.15.28:0/1054188050 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc18056210 con 0x55dbdbd19aa0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dbdbd21820 con 0x55dbdbd19aa0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.114+0000 7fbaeffff640 1 -- 172.21.15.28:0/1054188050 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc180459e0 con 0x55dbdbd19aa0 2024-09-19T11:23:45.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.115+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbdbd223b0 con 0x55dbdbd19aa0 2024-09-19T11:23:45.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.115+0000 7fbaeffff640 1 -- 172.21.15.28:0/1054188050 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbc18045b80 con 0x55dbdbd19aa0 2024-09-19T11:23:45.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.116+0000 7fbaeffff640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbae803cfb0 0x7fbae803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.116+0000 7fbaeffff640 1 -- 172.21.15.28:0/1054188050 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbc18034d50 con 0x55dbdbd19aa0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.116+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbae803cfb0 0x7fbae803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.116+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbae803cfb0 0x7fbae803f470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc100535e0 tx=0x7fbc10028000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.116+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.116+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbd2c9b0 0x55dbdbd2ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.116+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd2f5c0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc1ddc6640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbd2c9b0 0x55dbdbd2ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc1ddc6640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbd2c9b0 0x55dbdbd2ee70 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 20 realm 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd2f9f0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.117+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd302c0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc22939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd30660 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc22939800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbd313c0 0x55dbdbd337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd33dc0 con 0x55dbdbd313c0 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbd313c0 0x55dbdbd337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbd313c0 0x55dbdbd337b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.118+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc22939800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc22939800 20 searching for the correct realm 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1edc8640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbc18054c20 0x7fbc1803c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc18054c20 2024-09-19T11:23:45.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbc18054c20 0x7fbc1803c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbc18054c20 0x7fbc1803c870 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1e5c7640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbc1000f070 0x7fbc100536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.119+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc1000f070 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.120+0000 7fbc1ddc6640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbc1000f070 0x7fbc100536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.120+0000 7fbc1ddc6640 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbc1000f070 0x7fbc100536d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.120+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.120+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc1000f070 2024-09-19T11:23:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc1000f070 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.121+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc18054c20 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc1000f070 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.122+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc18054c20 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc18054c20 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.123+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.124+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc1000f070 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc18054c20 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.125+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc1000f070 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc1000f070 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc18054c20 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.126+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x55dbdbd313c0 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35710 con 0x7fbc18054c20 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7fbc10066040 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7fbc10066040 con 0x7fbc18054c20 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.127+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd34780 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd34fd0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35b90 con 0x55dbdbd313c0 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 zone default found 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 4 Realm: () 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:45.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 zonegroup default 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.128+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd35f30 con 0x7fbc1000f070 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd381f0 con 0x55dbdbd313c0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd38590 con 0x55dbdbd313c0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 20 started sync module instance, tier type = 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.129+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd37680 con 0x7fbc18054c20 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.130+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.130+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94402774268528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd372e0 con 0x7fbc18054c20 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.130+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd39380 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.131+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94402774268528] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.131+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.131+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94402774272448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd39770 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.131+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3a1d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.132+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94402774272448] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.132+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.133+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94402774277248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3a190 con 0x55dbdbd313c0 2024-09-19T11:23:45.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.133+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3ac60 con 0x7fbc18054c20 2024-09-19T11:23:45.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.133+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94402774277248] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.134+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.134+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94402774279984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3ac20 con 0x7fbc18054c20 2024-09-19T11:23:45.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.134+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3b710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.135+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.135+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94402774282784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3b710 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.135+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3c710 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.135+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94402774279984] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.136+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.136+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94402774286848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3c6d0 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.136+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3d120 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.136+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94402774282784] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.137+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94402774286848] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.137+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.137+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94402774273504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3cc70 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.137+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3d630 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94402774273504] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94402774285056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3d630 con 0x7fbc1000f070 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 20 add_watcher() i=0 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 20 add_watcher() i=1 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 20 add_watcher() i=2 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 20 add_watcher() i=3 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 20 add_watcher() i=4 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 20 add_watcher() i=5 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.138+0000 7fbc22939800 20 add_watcher() i=6 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.139+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94402774285056] v33'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.139+0000 7fbc22939800 20 add_watcher() i=7 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.139+0000 7fbc22939800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.139+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd40990 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.139+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.139+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94402774303632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd3d7a0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.140+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94402774303632] v33'187 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.140+0000 7fbad4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.140+0000 7fbc22939800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.140+0000 7fbc22939800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.140+0000 7fbc22939800 5 note: GC not initialized 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fba4f7fe640 20 reqs_thread_entry: start 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 20 Filter name: none 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fba4d7fa640 20 reqs_thread_entry: start 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd73830 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.141+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~403 in=403b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd79850 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~403] v33'16 uv16 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc22939800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc22939800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc22939800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94402774536752 in=591b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd75150 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94402774282784 notify 141733920786 ret 0) ==== 621+0+0 (crc 0 0 0) 0x55dbdbc00820 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbb01ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920786 cookie 94402774282784 notifier 4656 bl.length()=579 2024-09-19T11:23:45.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbb01ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:45.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbb01ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:45.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbb01ffb640 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fbae4003ea0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(64 notify.4 [notify cookie 94402774536752 out=8b] v0'0 uv180 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.143+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94402774536752 notify 141733920786 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fbc1405bc90 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.157 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[{"type":"user","perm":"read"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.144+0000 7fbad4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.144+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94402774303632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.145+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94402774303632] v33'188 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.145+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94402774268528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x7fbc18054c20 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.146+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94402774268528] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.146+0000 7fbc22939800 20 remove_watcher() i=0 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.146+0000 7fbc22939800 2 removed watcher, disabling cache 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.146+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94402774272448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.147+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94402774272448] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.147+0000 7fbc22939800 20 remove_watcher() i=1 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.147+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94402774277248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x55dbdbd313c0 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.148+0000 7fbc1edc8640 1 -- 172.21.15.28:0/1054188050 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94402774277248] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1800a4d0 con 0x55dbdbd313c0 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.148+0000 7fbc22939800 20 remove_watcher() i=2 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.148+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94402774279984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x7fbc18054c20 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.149+0000 7fbc1e5c7640 1 -- 172.21.15.28:0/1054188050 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94402774279984] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc1002e070 con 0x7fbc18054c20 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.149+0000 7fbc22939800 20 remove_watcher() i=3 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.149+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94402774282784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.150+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94402774282784] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x55dbdbd2c9b0 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.150+0000 7fbc22939800 20 remove_watcher() i=4 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.150+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94402774286848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x7fbc1000f070 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.151+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94402774286848] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.151+0000 7fbc22939800 20 remove_watcher() i=5 2024-09-19T11:23:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.151+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94402774273504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x7fbc1000f070 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.152+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94402774273504] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.152+0000 7fbc22939800 20 remove_watcher() i=6 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.152+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94402774285056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55dbdbd716f0 con 0x7fbc1000f070 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94402774285056] v33'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbc14057030 con 0x7fbc1000f070 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 20 remove_watcher() i=7 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbc1000f070 msgr2=0x7fbc100536d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbc1000f070 0x7fbc100536d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbd2c9b0 msgr2=0x55dbdbd2ee70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbd2c9b0 0x55dbdbd2ee70 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbc18054c20 msgr2=0x7fbc1803c870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbc18054c20 0x7fbc1803c870 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbd313c0 msgr2=0x55dbdbd337b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbd313c0 0x55dbdbd337b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbae803cfb0 msgr2=0x7fbae803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.153+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbae803cfb0 0x7fbae803f470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbc100535e0 tx=0x7fbc10028000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 msgr2=0x55dbdbd21030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/1054188050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd19aa0 0x55dbdbd21030 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbc18054740 tx=0x7fbc180547a0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc1ddc6640 1 -- 172.21.15.28:0/1054188050 reap_dead start 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 shutdown_connections 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 >> 172.21.15.28:0/1054188050 conn(0x55dbdbd15ea0 msgr2=0x55dbdbd140b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 shutdown_connections 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/1054188050 wait complete. 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbaf803cfb0 msgr2=0x7fbaf803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbaf803cfb0 0x7fbaf803f470 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55dbdbcf4140 tx=0x7fbc10017c10 comp rx=0 tx=0).stop 2024-09-19T11:23:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd0cfe0 msgr2=0x55dbdbd0d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd0cfe0 0x55dbdbd0d400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x55dbdbcf3050 tx=0x7fbc180167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 shutdown_connections 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbaf803cfb0 0x7fbaf803f470 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3442390331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbd0cfe0 0x55dbdbd0d400 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 >> 172.21.15.28:0/3442390331 conn(0x55dbdbd01390 msgr2=0x55dbdbd01770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 shutdown_connections 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3442390331 wait complete. 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbbfcab0 msgr2=0x55dbdbcf96a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbbfcab0 0x55dbdbcf96a0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbcfba80 msgr2=0x55dbdbcfdf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbcfba80 0x55dbdbcfdf00 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbb0403cfb0 msgr2=0x7fbb0403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbb0403cfb0 0x7fbb0403f470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc14057000 tx=0x7fbc14052000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 msgr2=0x55dbdbcf2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.154+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbcf2050 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x55dbdbbf8130 tx=0x7fbc10000ed0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 shutdown_connections 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55dbdbcfba80 0x55dbdbcfdf00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbb0403cfb0 0x7fbb0403f470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55dbdbbfcab0 0x55dbdbcf96a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 --2- 172.21.15.28:0/3365994320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbdbbf8eb0 0x55dbdbcf2050 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 >> 172.21.15.28:0/3365994320 conn(0x55dbdbbf2a70 msgr2=0x55dbdbce3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 shutdown_connections 2024-09-19T11:23:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.155+0000 7fbc22939800 1 -- 172.21.15.28:0/3365994320 wait complete. 2024-09-19T11:23:45.168 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:45.168 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-09-19T11:23:45.168 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-09-19T11:23:45.169 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-09-19T11:23:45.216 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:45.216 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:45.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.215+0000 7f3629214800 1 Processor -- start 2024-09-19T11:23:45.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f3629214800 1 -- start start 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f3629214800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55aa1e928cf0 0x55aa1e92b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f3629214800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55aa1e92c0f0 con 0x55aa1e92bc40 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f3629214800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55aa1e7b57d0 con 0x55aa1e928cf0 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f36249e0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55aa1e928cf0 0x55aa1e92b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f36241df640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55aa1e92bc40 0x55aa1e930750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:58120/0 (socket says 172.21.15.28:58120) 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f36249e0640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55aa1e928cf0 0x55aa1e92b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56164/0 (socket says 172.21.15.28:56164) 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f36241df640 1 -- 172.21.15.28:0/2803299152 learned_addr learned my addr 172.21.15.28:0/2803299152 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f36249e0640 1 -- 172.21.15.28:0/2803299152 >> v1:172.21.15.22:6789/0 conn(0x55aa1e92bc40 legacy=0x55aa1e930750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.216+0000 7f36249e0640 1 -- 172.21.15.28:0/2803299152 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1e92c2c0 con 0x55aa1e928cf0 2024-09-19T11:23:45.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.218+0000 7f36249e0640 1 --2- 172.21.15.28:0/2803299152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1e92b700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3614005fd0 tx=0x7f3614045c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bd6dfc600d7053 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.218+0000 7f36239de640 1 -- 172.21.15.28:0/2803299152 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614058530 con 0x55aa1e928cf0 2024-09-19T11:23:45.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.218+0000 7f36239de640 1 -- 172.21.15.28:0/2803299152 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36140586d0 con 0x55aa1e928cf0 2024-09-19T11:23:45.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.218+0000 7f36239de640 1 -- 172.21.15.28:0/2803299152 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36140589d0 con 0x55aa1e928cf0 2024-09-19T11:23:45.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.219+0000 7f3629214800 1 -- 172.21.15.28:0/2803299152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 msgr2=0x55aa1e92b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.219+0000 7f3629214800 1 --2- 172.21.15.28:0/2803299152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1e92b700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3614005fd0 tx=0x7f3614045c30 comp rx=0 tx=0).stop 2024-09-19T11:23:45.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.219+0000 7f3629214800 1 -- 172.21.15.28:0/2803299152 shutdown_connections 2024-09-19T11:23:45.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.219+0000 7f3629214800 1 --2- 172.21.15.28:0/2803299152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1e92b700 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.219+0000 7f3629214800 1 -- 172.21.15.28:0/2803299152 >> 172.21.15.28:0/2803299152 conn(0x55aa1e9241d0 msgr2=0x55aa1e926650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.219+0000 7f3629214800 1 -- 172.21.15.28:0/2803299152 shutdown_connections 2024-09-19T11:23:45.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.219+0000 7f3629214800 1 -- 172.21.15.28:0/2803299152 wait complete. 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.234+0000 7f3629214800 1 Processor -- start 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.234+0000 7f3629214800 1 -- start start 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.234+0000 7f3629214800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea1a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.234+0000 7f3629214800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa1e7c2dd0 con 0x55aa1e928cf0 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.234+0000 7f36249e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea1a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.234+0000 7f36249e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea1a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56166/0 (socket says 172.21.15.28:56166) 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.234+0000 7f36249e0640 1 -- 172.21.15.28:0/3483097346 learned_addr learned my addr 172.21.15.28:0/3483097346 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.235+0000 7f36249e0640 1 -- 172.21.15.28:0/3483097346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1ea1abc0 con 0x55aa1e928cf0 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.235+0000 7f36249e0640 1 --2- 172.21.15.28:0/3483097346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea1a5d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f361400a3f0 tx=0x7f3614051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99615143a2b9b288 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.235+0000 7f3512e0d640 1 -- 172.21.15.28:0/3483097346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36140589b0 con 0x55aa1e928cf0 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.235+0000 7f3512e0d640 1 -- 172.21.15.28:0/3483097346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3614068070 con 0x55aa1e928cf0 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.235+0000 7f3512e0d640 1 -- 172.21.15.28:0/3483097346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614058c80 con 0x55aa1e928cf0 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 -- 172.21.15.28:0/3483097346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 msgr2=0x55aa1ea1a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 --2- 172.21.15.28:0/3483097346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea1a5d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f361400a3f0 tx=0x7f3614051000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 -- 172.21.15.28:0/3483097346 shutdown_connections 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 --2- 172.21.15.28:0/3483097346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea1a5d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 -- 172.21.15.28:0/3483097346 >> 172.21.15.28:0/3483097346 conn(0x55aa1e9241d0 msgr2=0x55aa1ea15f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 -- 172.21.15.28:0/3483097346 shutdown_connections 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 -- 172.21.15.28:0/3483097346 wait complete. 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 Processor -- start 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.236+0000 7f3629214800 1 -- start start 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.237+0000 7f3629214800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea21f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.237+0000 7f3629214800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa1e932080 con 0x55aa1e928cf0 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.237+0000 7f36249e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea21f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.237+0000 7f36249e0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea21f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56172/0 (socket says 172.21.15.28:56172) 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.237+0000 7f36249e0640 1 -- 172.21.15.28:0/2301582512 learned_addr learned my addr 172.21.15.28:0/2301582512 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.237+0000 7f36249e0640 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1ea224c0 con 0x55aa1e928cf0 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.237+0000 7f36249e0640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea21f60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3614065040 tx=0x7f3614003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.238+0000 7f351160a640 1 -- 172.21.15.28:0/2301582512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614063570 con 0x55aa1e928cf0 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.238+0000 7f351160a640 1 -- 172.21.15.28:0/2301582512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3614063710 con 0x55aa1e928cf0 2024-09-19T11:23:45.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.238+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa1ea22750 con 0x55aa1e928cf0 2024-09-19T11:23:45.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.238+0000 7f351160a640 1 -- 172.21.15.28:0/2301582512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3614063a10 con 0x55aa1e928cf0 2024-09-19T11:23:45.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.238+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa1ea23150 con 0x55aa1e928cf0 2024-09-19T11:23:45.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.238+0000 7f351160a640 1 -- 172.21.15.28:0/2301582512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3614075020 con 0x55aa1e928cf0 2024-09-19T11:23:45.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f351160a640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f350c03d000 0x7f350c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f351160a640 1 -- 172.21.15.28:0/2301582512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f36140988e0 con 0x55aa1e928cf0 2024-09-19T11:23:45.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f36241df640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f350c03d000 0x7f350c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1e92bc40 0x55aa1ea295b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea23cf0 con 0x55aa1e92bc40 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f36251e1640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1e92bc40 0x55aa1ea295b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f36241df640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f350c03d000 0x7f350c03f4c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3618001d70 tx=0x7f36180012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.239+0000 7f36251e1640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1e92bc40 0x55aa1ea295b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.240+0000 7f36251e1640 1 -- 172.21.15.28:0/2301582512 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f361c0083e0 con 0x55aa1e92bc40 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.240+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea29ff0 con 0x55aa1e92bc40 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.240+0000 7f36251e1640 1 -- 172.21.15.28:0/2301582512 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f361c0083e0 con 0x55aa1e92bc40 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.240+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea2b900 0x55aa1ea2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.240+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea2e530 con 0x55aa1ea2b900 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.240+0000 7f36249e0640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea2b900 0x55aa1ea2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.240+0000 7f36249e0640 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea2b900 0x55aa1ea2dda0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.241+0000 7f36249e0640 1 -- 172.21.15.28:0/2301582512 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3614077930 con 0x55aa1ea2b900 2024-09-19T11:23:45.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.241+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea2a840 con 0x55aa1e92bc40 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.241+0000 7f36251e1640 1 -- 172.21.15.28:0/2301582512 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f361c0083e0 con 0x55aa1e92bc40 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.241+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea2abe0 con 0x55aa1ea2b900 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.241+0000 7f36249e0640 1 -- 172.21.15.28:0/2301582512 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3614077930 con 0x55aa1ea2b900 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f3629214800 1 Processor -- start 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f3629214800 1 -- start start 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f3629214800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea3cee0 0x55aa1ea3d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f3629214800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa1e8214a0 con 0x55aa1ea3cee0 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f36251e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea3cee0 0x55aa1ea3d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f36251e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea3cee0 0x55aa1ea3d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56180/0 (socket says 172.21.15.28:56180) 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f36251e1640 1 -- 172.21.15.28:0/2682156992 learned_addr learned my addr 172.21.15.28:0/2682156992 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.242+0000 7f36251e1640 1 -- 172.21.15.28:0/2682156992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1ea3d940 con 0x55aa1ea3cee0 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.243+0000 7f36251e1640 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea3cee0 0x55aa1ea3d300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55aa1ea23550 tx=0x7f361c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.243+0000 7f350a7fc640 1 -- 172.21.15.28:0/2682156992 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c01c020 con 0x55aa1ea3cee0 2024-09-19T11:23:45.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.243+0000 7f350a7fc640 1 -- 172.21.15.28:0/2682156992 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c007c90 con 0x55aa1ea3cee0 2024-09-19T11:23:45.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.243+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa1ea3e170 con 0x55aa1ea3cee0 2024-09-19T11:23:45.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.243+0000 7f350a7fc640 1 -- 172.21.15.28:0/2682156992 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c018440 con 0x55aa1ea3cee0 2024-09-19T11:23:45.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.243+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa1ea3e710 con 0x55aa1ea3cee0 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.244+0000 7f350a7fc640 1 -- 172.21.15.28:0/2682156992 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f361c01f020 con 0x55aa1ea3cee0 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.244+0000 7f350a7fc640 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34fc03d000 0x7f34fc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.244+0000 7f350a7fc640 1 -- 172.21.15.28:0/2682156992 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f361c055b00 con 0x55aa1ea3cee0 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.244+0000 7f36249e0640 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34fc03d000 0x7f34fc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.244+0000 7f36249e0640 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34fc03d000 0x7f34fc03f4c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55aa1ea2e8d0 tx=0x7f3614077cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.244+0000 7f3629214800 1 Processor -- start 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f3629214800 1 -- start start 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f3629214800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f3629214800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa1ea1af30 con 0x55aa1ea498d0 2024-09-19T11:23:45.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f36251e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f36251e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea49cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56192/0 (socket says 172.21.15.28:56192) 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f36251e1640 1 -- 172.21.15.28:0/3353871110 learned_addr learned my addr 172.21.15.28:0/3353871110 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f36251e1640 1 -- 172.21.15.28:0/3353871110 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1ea4a2c0 con 0x55aa1ea498d0 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.245+0000 7f36251e1640 1 --2- 172.21.15.28:0/3353871110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea49cd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f361c017f70 tx=0x7f361c0175a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06824ef2904b268 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f34fbfff640 1 -- 172.21.15.28:0/3353871110 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c04a070 con 0x55aa1ea498d0 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f34fbfff640 1 -- 172.21.15.28:0/3353871110 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c015e80 con 0x55aa1ea498d0 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f34fbfff640 1 -- 172.21.15.28:0/3353871110 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c015820 con 0x55aa1ea498d0 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f3629214800 1 -- 172.21.15.28:0/3353871110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 msgr2=0x55aa1ea49cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f3629214800 1 --2- 172.21.15.28:0/3353871110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea49cd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f361c017f70 tx=0x7f361c0175a0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f3629214800 1 -- 172.21.15.28:0/3353871110 shutdown_connections 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f3629214800 1 --2- 172.21.15.28:0/3353871110 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea49cd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f3629214800 1 -- 172.21.15.28:0/3353871110 >> 172.21.15.28:0/3353871110 conn(0x55aa1ea45cd0 msgr2=0x55aa1ea43f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f3629214800 1 -- 172.21.15.28:0/3353871110 shutdown_connections 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.246+0000 7f3629214800 1 -- 172.21.15.28:0/3353871110 wait complete. 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f3629214800 1 Processor -- start 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f3629214800 1 -- start start 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f3629214800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea50f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f3629214800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55aa1ea4a8e0 con 0x55aa1ea498d0 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f36251e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea50f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f36251e1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea50f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56204/0 (socket says 172.21.15.28:56204) 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 learned_addr learned my addr 172.21.15.28:0/822328344 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa1ea51480 con 0x55aa1ea498d0 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.247+0000 7f36251e1640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea50f20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f361c052ce0 tx=0x7f361c053d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.248+0000 7f34fa7fc640 1 -- 172.21.15.28:0/822328344 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c04a040 con 0x55aa1ea498d0 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.248+0000 7f34fa7fc640 1 -- 172.21.15.28:0/822328344 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361c013c90 con 0x55aa1ea498d0 2024-09-19T11:23:45.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.248+0000 7f34fa7fc640 1 -- 172.21.15.28:0/822328344 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f361c0208e0 con 0x55aa1ea498d0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.248+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa1ea51710 con 0x55aa1ea498d0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.248+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa1ea522a0 con 0x55aa1ea498d0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.248+0000 7f34fa7fc640 1 -- 172.21.15.28:0/822328344 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f361c016040 con 0x55aa1ea498d0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.249+0000 7f34fa7fc640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34f0041410 0x7f34f00438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.249+0000 7f34fa7fc640 1 -- 172.21.15.28:0/822328344 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f361c03ccf0 con 0x55aa1ea498d0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.249+0000 7f36249e0640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34f0041410 0x7f34f00438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.249+0000 7f36249e0640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34f0041410 0x7f34f00438d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f361406c710 tx=0x7f3614091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1ea5a6d0 0x55aa1ea5cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea5d2e0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f36241df640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1ea5a6d0 0x55aa1ea5cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f36241df640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1ea5a6d0 0x55aa1ea5cb90 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3618009050 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f3629214800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f3629214800 20 realm 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.250+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea59a20 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3618009050 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea5ddd0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3618009050 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea5e6a0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3618009050 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea5f0c0 0x55aa1ea614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.251+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea61bf0 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.252+0000 7f36251e1640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea5f0c0 0x55aa1ea614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.252+0000 7f36251e1640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea5f0c0 0x55aa1ea614b0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.252+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.252+0000 7f3629214800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.252+0000 7f3629214800 20 searching for the correct realm 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.252+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.253+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.253+0000 7f36251e1640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f361c032020 0x7f361c02aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.253+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.253+0000 7f36249e0640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f361c032020 0x7f361c02aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.253+0000 7f36249e0640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f361c032020 0x7f361c02aa00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.254+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.254+0000 7f36249e0640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3614096210 0x7f361404f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.254+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f3614096210 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.254+0000 7f36241df640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3614096210 0x7f361404f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.254+0000 7f36241df640 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3614096210 0x7f361404f610 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.254+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618009050 con 0x7f3614096210 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.254+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618009050 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f3614096210 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f3614096210 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.255+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f3614096210 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.256+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.257+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.258+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f3614096210 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.259+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f3614096210 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f3614096210 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.260+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63520 con 0x7f361c032020 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.261+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea625b0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea62e00 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea639a0 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 zone default found 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 4 Realm: () 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 zonegroup default 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.262+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea63d40 con 0x7f3614096210 2024-09-19T11:23:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f3629214800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea65f80 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f3629214800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.263+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea66320 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.264+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.264+0000 7f3629214800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.264+0000 7f3629214800 20 started sync module instance, tier type = 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.264+0000 7f3629214800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.264+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea653c0 con 0x7f361c032020 2024-09-19T11:23:45.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.265+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.265+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94189147016976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea66320 con 0x7f361c032020 2024-09-19T11:23:45.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.265+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea66ce0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.266+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94189147016976] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.266+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.266+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94189147023408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea67100 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.266+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea67b00 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.267+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.267+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94189147026960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea67ab0 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.267+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea68ad0 con 0x7f361c032020 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.267+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94189147023408] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.268+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94189147026960] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.269+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.269+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94189147029232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea66c50 con 0x7f361c032020 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.269+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea69370 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.270+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94189147029232] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.270+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.270+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94189147030976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea69370 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.270+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea69e60 con 0x7f3614096210 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.271+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.271+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94189147035952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea69e20 con 0x7f3614096210 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.271+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94189147030976] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.271+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea6ae40 con 0x7f3614096210 2024-09-19T11:23:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.272+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.272+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94189147040080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea6ae40 con 0x7f3614096210 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.272+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea6b880 con 0x7f3614096210 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.272+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94189147035952] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94189147040080] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94189147038016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea6b3d0 con 0x7f3614096210 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 20 add_watcher() i=0 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 20 add_watcher() i=1 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 20 add_watcher() i=2 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 20 add_watcher() i=3 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 20 add_watcher() i=4 2024-09-19T11:23:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 20 add_watcher() i=5 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.273+0000 7f3629214800 20 add_watcher() i=6 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.274+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94189147038016] v33'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.274+0000 7f3629214800 20 add_watcher() i=7 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.274+0000 7f3629214800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.274+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea6e8b0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.275+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.275+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94189147042768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea6bca0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.275+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94189147042768] v33'189 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f34db7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:45.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 5 note: GC not initialized 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f34da7e4640 20 reqs_thread_entry: start 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 20 Filter name: none 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f34d3fff640 20 reqs_thread_entry: start 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.276+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1eaa15c0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.277+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~403 out=403b,getxattrs out=75b] v0'0 uv16 ondisk = 0) ==== 231+0+526 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.277+0000 7f3629214800 20 rados_obj.operate() r=0 bl.length=403 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.277+0000 7f3629214800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.277+0000 7f3629214800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.277+0000 7f3629214800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.277+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1eaa75c0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v33'17 uv17 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f3629214800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f3629214800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f3629214800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94189147287488 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55aa1eaa2ee0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify (1) cookie 94189147030976 notify 141733920787 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55aa1e92c0f0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f34fbfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920787 cookie 94189147030976 notifier 4663 bl.length()=567 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(64 notify.4 [notify cookie 94189147287488 out=8b] v0'0 uv183 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f34fbfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f34fbfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f34fbfff640 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f34e8003ea0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.278+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.279+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== watch-notify(notify_complete (2) cookie 94189147287488 notify 141733920787 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f361802d9f0 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.287 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-09-19T11:23:21.921555Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:23:21.921540Z","tags":[],"group_ids":[]} 2024-09-19T11:23:45.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.279+0000 7f34db7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:45.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.279+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94189147042768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.280+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94189147042768] v33'190 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.280+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94189147016976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x7f361c032020 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.282+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94189147016976] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.282+0000 7f3629214800 20 remove_watcher() i=0 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.282+0000 7f3629214800 2 removed watcher, disabling cache 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.282+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94189147023408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.283+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94189147023408] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.283+0000 7f3629214800 20 remove_watcher() i=1 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.283+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94189147026960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.284+0000 7f36251e1640 1 -- 172.21.15.28:0/822328344 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94189147026960] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361c039070 con 0x55aa1ea5f0c0 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.284+0000 7f3629214800 20 remove_watcher() i=2 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.284+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94189147029232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x7f361c032020 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.285+0000 7f36249e0640 1 -- 172.21.15.28:0/822328344 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94189147029232] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f361408c040 con 0x7f361c032020 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.285+0000 7f3629214800 20 remove_watcher() i=3 2024-09-19T11:23:45.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.285+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94189147030976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.286+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94189147030976] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x55aa1ea5a6d0 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.286+0000 7f3629214800 20 remove_watcher() i=4 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.286+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94189147035952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x7f3614096210 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.287+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94189147035952] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.287+0000 7f3629214800 20 remove_watcher() i=5 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.287+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94189147040080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x7f3614096210 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.287+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94189147040080] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.287+0000 7f3629214800 20 remove_watcher() i=6 2024-09-19T11:23:45.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.288+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94189147038016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55aa1ea9f440 con 0x7f3614096210 2024-09-19T11:23:45.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.288+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94189147038016] v33'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3618029040 con 0x7f3614096210 2024-09-19T11:23:45.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.288+0000 7f3629214800 20 remove_watcher() i=7 2024-09-19T11:23:45.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3614096210 msgr2=0x7f361404f610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3614096210 0x7f361404f610 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1ea5a6d0 msgr2=0x55aa1ea5cb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1ea5a6d0 0x55aa1ea5cb90 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f361c032020 msgr2=0x7f361c02aa00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f361c032020 0x7f361c02aa00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea5f0c0 msgr2=0x55aa1ea614b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea5f0c0 0x55aa1ea614b0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34f0041410 msgr2=0x7f34f00438d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34f0041410 0x7f34f00438d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f361406c710 tx=0x7f3614091000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 msgr2=0x55aa1ea50f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea50f20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f361c052ce0 tx=0x7f361c053d50 comp rx=0 tx=0).stop 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f36241df640 1 -- 172.21.15.28:0/822328344 reap_dead start 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 shutdown_connections 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/822328344 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea498d0 0x55aa1ea50f20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 >> 172.21.15.28:0/822328344 conn(0x55aa1ea45cd0 msgr2=0x55aa1ea43ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 shutdown_connections 2024-09-19T11:23:45.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/822328344 wait complete. 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34fc03d000 msgr2=0x7f34fc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34fc03d000 0x7f34fc03f4c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55aa1ea2e8d0 tx=0x7f3614077cc0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea3cee0 msgr2=0x55aa1ea3d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea3cee0 0x55aa1ea3d300 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55aa1ea23550 tx=0x7f361c00bf30 comp rx=0 tx=0).stop 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 shutdown_connections 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f34fc03d000 0x7f34fc03f4c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 --2- 172.21.15.28:0/2682156992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1ea3cee0 0x55aa1ea3d300 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 >> 172.21.15.28:0/2682156992 conn(0x55aa1ea31290 msgr2=0x55aa1ea31670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 shutdown_connections 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.289+0000 7f3629214800 1 -- 172.21.15.28:0/2682156992 wait complete. 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1e92bc40 msgr2=0x55aa1ea295b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1e92bc40 0x55aa1ea295b0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea2b900 msgr2=0x55aa1ea2dda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea2b900 0x55aa1ea2dda0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f350c03d000 msgr2=0x7f350c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f350c03d000 0x7f350c03f4c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3618001d70 tx=0x7f36180012d0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 msgr2=0x55aa1ea21f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea21f60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3614065040 tx=0x7f3614003330 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 shutdown_connections 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55aa1ea2b900 0x55aa1ea2dda0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f350c03d000 0x7f350c03f4c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55aa1e92bc40 0x55aa1ea295b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 --2- 172.21.15.28:0/2301582512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55aa1e928cf0 0x55aa1ea21f60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 >> 172.21.15.28:0/2301582512 conn(0x55aa1e9241d0 msgr2=0x55aa1ea1b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 shutdown_connections 2024-09-19T11:23:45.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.290+0000 7f3629214800 1 -- 172.21.15.28:0/2301582512 wait complete. 2024-09-19T11:23:45.300 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-09-19T11:23:21.921555Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:23:21.921540Z', 'tags': [], 'group_ids': []} 2024-09-19T11:23:45.310 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:45.310 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-09-19T11:23:45.310 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-09-19T11:23:45.310 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-09-19T11:23:45.356 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:23:45.356 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.355+0000 7fe0a972e800 1 Processor -- start 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.355+0000 7fe0a972e800 1 -- start start 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a972e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a48a8a2d20 0x55a48a8a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a972e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a48a8a4220 con 0x55a48a8a3660 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a972e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55a48a72f7d0 con 0x55a48a8a2d20 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a54dd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a48a8a2d20 0x55a48a8a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a54dd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a48a8a2d20 0x55a48a8a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56218/0 (socket says 172.21.15.28:56218) 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a54dd640 1 -- 172.21.15.28:0/199492014 learned_addr learned my addr 172.21.15.28:0/199492014 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a4cdc640 1 --1- 172.21.15.28:0/199492014 >> v1:172.21.15.22:6789/0 conn(0x55a48a8a3660 0x55a48a8a3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:58130/0 (socket says 172.21.15.28:58130) 2024-09-19T11:23:45.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a54dd640 1 -- 172.21.15.28:0/199492014 >> v1:172.21.15.22:6789/0 conn(0x55a48a8a3660 legacy=0x55a48a8a3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.356+0000 7fe0a54dd640 1 -- 172.21.15.28:0/199492014 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48a8a43f0 con 0x55a48a8a2d20 2024-09-19T11:23:45.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.358+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/199492014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a8a3120 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe09800a520 tx=0x7fe098045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03d77e5c622d0fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.358+0000 7fe097fff640 1 -- 172.21.15.28:0/199492014 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe098055070 con 0x55a48a8a2d20 2024-09-19T11:23:45.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.358+0000 7fe097fff640 1 -- 172.21.15.28:0/199492014 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0980515b0 con 0x55a48a8a2d20 2024-09-19T11:23:45.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.358+0000 7fe097fff640 1 -- 172.21.15.28:0/199492014 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0980518d0 con 0x55a48a8a2d20 2024-09-19T11:23:45.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.359+0000 7fe0a972e800 1 -- 172.21.15.28:0/199492014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 msgr2=0x55a48a8a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.359+0000 7fe0a972e800 1 --2- 172.21.15.28:0/199492014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a8a3120 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe09800a520 tx=0x7fe098045ba0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.359+0000 7fe0a972e800 1 -- 172.21.15.28:0/199492014 shutdown_connections 2024-09-19T11:23:45.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.359+0000 7fe0a972e800 1 --2- 172.21.15.28:0/199492014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a8a3120 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.359+0000 7fe0a972e800 1 -- 172.21.15.28:0/199492014 >> 172.21.15.28:0/199492014 conn(0x55a48a89ea50 msgr2=0x55a48a8a0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.359+0000 7fe0a972e800 1 -- 172.21.15.28:0/199492014 shutdown_connections 2024-09-19T11:23:45.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.359+0000 7fe0a972e800 1 -- 172.21.15.28:0/199492014 wait complete. 2024-09-19T11:23:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.374+0000 7fe0a972e800 1 Processor -- start 2024-09-19T11:23:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.374+0000 7fe0a972e800 1 -- start start 2024-09-19T11:23:45.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.374+0000 7fe0a972e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a996530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.374+0000 7fe0a972e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a48a73cdd0 con 0x55a48a8a2d20 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fe0a54dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a996530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fe0a54dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a996530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56228/0 (socket says 172.21.15.28:56228) 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fe0a54dd640 1 -- 172.21.15.28:0/4252391613 learned_addr learned my addr 172.21.15.28:0/4252391613 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fe0a54dd640 1 -- 172.21.15.28:0/4252391613 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48a996b20 con 0x55a48a8a2d20 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/4252391613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a996530 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe098025da0 tx=0x7fe098051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbf41f4ee0d4536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fdf93606640 1 -- 172.21.15.28:0/4252391613 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe098051950 con 0x55a48a8a2d20 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fdf93606640 1 -- 172.21.15.28:0/4252391613 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe098051af0 con 0x55a48a8a2d20 2024-09-19T11:23:45.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.375+0000 7fdf93606640 1 -- 172.21.15.28:0/4252391613 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe098051dd0 con 0x55a48a8a2d20 2024-09-19T11:23:45.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 -- 172.21.15.28:0/4252391613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 msgr2=0x55a48a996530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 --2- 172.21.15.28:0/4252391613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a996530 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe098025da0 tx=0x7fe098051000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 -- 172.21.15.28:0/4252391613 shutdown_connections 2024-09-19T11:23:45.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 --2- 172.21.15.28:0/4252391613 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a996530 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 -- 172.21.15.28:0/4252391613 >> 172.21.15.28:0/4252391613 conn(0x55a48a89ea50 msgr2=0x55a48a991eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 -- 172.21.15.28:0/4252391613 shutdown_connections 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 -- 172.21.15.28:0/4252391613 wait complete. 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 Processor -- start 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.376+0000 7fe0a972e800 1 -- start start 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fe0a972e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a99dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fe0a972e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a48a8a4980 con 0x55a48a8a2d20 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fe0a54dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a99dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fe0a54dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a99dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56242/0 (socket says 172.21.15.28:56242) 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fe0a54dd640 1 -- 172.21.15.28:0/3759464109 learned_addr learned my addr 172.21.15.28:0/3759464109 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fe0a54dd640 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48a99e400 con 0x55a48a8a2d20 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a99dea0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe098067040 tx=0x7fe098002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.377+0000 7fdf91e03640 1 -- 172.21.15.28:0/3759464109 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe098073070 con 0x55a48a8a2d20 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.378+0000 7fdf91e03640 1 -- 172.21.15.28:0/3759464109 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe09805a450 con 0x55a48a8a2d20 2024-09-19T11:23:45.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.378+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a48a99e690 con 0x55a48a8a2d20 2024-09-19T11:23:45.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.378+0000 7fdf91e03640 1 -- 172.21.15.28:0/3759464109 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe09805a730 con 0x55a48a8a2d20 2024-09-19T11:23:45.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.378+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48a99f2a0 con 0x55a48a8a2d20 2024-09-19T11:23:45.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.378+0000 7fdf91e03640 1 -- 172.21.15.28:0/3759464109 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe098076070 con 0x55a48a8a2d20 2024-09-19T11:23:45.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fdf91e03640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf8c03cfb0 0x7fdf8c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fdf91e03640 1 -- 172.21.15.28:0/3759464109 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fe098062070 con 0x55a48a8a2d20 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fe0a4cdc640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf8c03cfb0 0x7fdf8c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a8a3660 0x55a48a9a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9a5a30 con 0x55a48a8a3660 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a8a3660 0x55a48a9a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fe0a4cdc640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf8c03cfb0 0x7fdf8c03f470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe09c001e30 tx=0x7fe09c001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.379+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a8a3660 0x55a48a9a54f0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.380+0000 7fe0a5cde640 1 -- 172.21.15.28:0/3759464109 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a8a3660 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.380+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9a5dd0 con 0x55a48a8a3660 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.380+0000 7fe0a5cde640 1 -- 172.21.15.28:0/3759464109 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a8a3660 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.380+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9a7890 0x55a48a9a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.380+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9aa4c0 con 0x55a48a9a7890 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.380+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9a7890 0x55a48a9a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.380+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9a7890 0x55a48a9a9d30 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.381+0000 7fe0a54dd640 1 -- 172.21.15.28:0/3759464109 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe09807f070 con 0x55a48a9a7890 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.381+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9a6620 con 0x55a48a8a3660 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.381+0000 7fe0a5cde640 1 -- 172.21.15.28:0/3759464109 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a8a3660 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.381+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9a6a40 con 0x55a48a9a7890 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a54dd640 1 -- 172.21.15.28:0/3759464109 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe09807f070 con 0x55a48a9a7890 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a972e800 1 Processor -- start 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a972e800 1 -- start start 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a972e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9b8e50 0x55a48a9b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a972e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a48a79b4a0 con 0x55a48a9b8e50 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a5cde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9b8e50 0x55a48a9b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a5cde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9b8e50 0x55a48a9b9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56254/0 (socket says 172.21.15.28:56254) 2024-09-19T11:23:45.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.382+0000 7fe0a5cde640 1 -- 172.21.15.28:0/59117410 learned_addr learned my addr 172.21.15.28:0/59117410 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.383+0000 7fe0a5cde640 1 -- 172.21.15.28:0/59117410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48a9b98b0 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.383+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9b8e50 0x55a48a9b9270 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55a48a99ed30 tx=0x7fe0a00167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.383+0000 7fdf8affd640 1 -- 172.21.15.28:0/59117410 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0a0013640 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.383+0000 7fdf8affd640 1 -- 172.21.15.28:0/59117410 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a00137e0 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.383+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a48a9ba0e0 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.383+0000 7fdf8affd640 1 -- 172.21.15.28:0/59117410 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0a0013ae0 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.383+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48a9ba680 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.384+0000 7fdf8affd640 1 -- 172.21.15.28:0/59117410 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe0a0011040 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.384+0000 7fdf8affd640 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7c03d000 0x7fdf7c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.384+0000 7fdf8affd640 1 -- 172.21.15.28:0/59117410 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fe0a000e040 con 0x55a48a9b8e50 2024-09-19T11:23:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.384+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7c03d000 0x7fdf7c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.384+0000 7fe0a972e800 1 Processor -- start 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a972e800 1 -- start start 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7c03d000 0x7fdf7c03f4c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55a48a9aa890 tx=0x7fe09806ef00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a972e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a972e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a48a996e90 con 0x55a48a9c58d0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a5cde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a5cde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56260/0 (socket says 172.21.15.28:56260) 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2258962465 learned_addr learned my addr 172.21.15.28:0/2258962465 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2258962465 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48a9c62c0 con 0x55a48a9c58d0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/2258962465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9c5cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0016cd0 tx=0x7fe0a0014480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594c2f214c252661 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fdf88ff9640 1 -- 172.21.15.28:0/2258962465 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0a0051070 con 0x55a48a9c58d0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fdf88ff9640 1 -- 172.21.15.28:0/2258962465 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a0053310 con 0x55a48a9c58d0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.385+0000 7fdf88ff9640 1 -- 172.21.15.28:0/2258962465 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0a0021910 con 0x55a48a9c58d0 2024-09-19T11:23:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 -- 172.21.15.28:0/2258962465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 msgr2=0x55a48a9c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2258962465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9c5cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0016cd0 tx=0x7fe0a0014480 comp rx=0 tx=0).stop 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 -- 172.21.15.28:0/2258962465 shutdown_connections 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2258962465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9c5cd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 -- 172.21.15.28:0/2258962465 >> 172.21.15.28:0/2258962465 conn(0x55a48a9c1cd0 msgr2=0x55a48a9bff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 -- 172.21.15.28:0/2258962465 shutdown_connections 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 -- 172.21.15.28:0/2258962465 wait complete. 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 Processor -- start 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 -- start start 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.386+0000 7fe0a972e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a48a9c68e0 con 0x55a48a9c58d0 2024-09-19T11:23:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fe0a5cde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fe0a5cde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56272/0 (socket says 172.21.15.28:56272) 2024-09-19T11:23:45.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 learned_addr learned my addr 172.21.15.28:0/2786429384 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a48a9cd480 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9ccf20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0054740 tx=0x7fe0a0054770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fdf7affd640 1 -- 172.21.15.28:0/2786429384 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0a0013c80 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fdf7affd640 1 -- 172.21.15.28:0/2786429384 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a004c070 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fdf7affd640 1 -- 172.21.15.28:0/2786429384 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0a00459b0 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a48a9cd710 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.387+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48a9ce2a0 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.388+0000 7fdf7affd640 1 -- 172.21.15.28:0/2786429384 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe0a0045b50 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.388+0000 7fdf7affd640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7003d000 0x7fdf7003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.388+0000 7fdf7affd640 1 -- 172.21.15.28:0/2786429384 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fe0a0004930 con 0x55a48a9c58d0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.388+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7003d000 0x7fdf7003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.389+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7003d000 0x7fdf7003f4c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe098051840 tx=0x7fe09808b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.389+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.389+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a9d8800 0x55a48a9dacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.389+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9db410 con 0x55a48a9d8800 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.389+0000 7fe0a4cdc640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a9d8800 0x55a48a9dacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.389+0000 7fe0a4cdc640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a9d8800 0x55a48a9dacc0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe09c00f610 con 0x55a48a9d8800 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 realm 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9d7b50 con 0x55a48a9d8800 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe09c00f610 con 0x55a48a9d8800 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:23:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9dbf00 con 0x55a48a9d8800 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe09c00f610 con 0x55a48a9d8800 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9dc7d0 con 0x55a48a9d8800 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe09c00f610 con 0x55a48a9d8800 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.390+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9dd1f0 0x55a48a9df5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9dfd20 con 0x55a48a9dd1f0 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9dd1f0 0x55a48a9df5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9dd1f0 0x55a48a9df5e0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a972e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a972e800 20 searching for the correct realm 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a5cde640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe0a0033b20 0x7fe0a0031a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.391+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.392+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe0a0033b20 0x7fe0a0031a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.392+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe0a0033b20 0x7fe0a0031a00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.392+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.392+0000 7fe0a54dd640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe098093040 0x7fe09808f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.392+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe098093040 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.392+0000 7fe0a4cdc640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe098093040 0x7fe09808f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.393+0000 7fe0a4cdc640 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe098093040 0x7fe09808f0d0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:23:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.393+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c00f610 con 0x7fe098093040 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.393+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.393+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c00f610 con 0x55a48a9d8800 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.393+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.393+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.393+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe098093040 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x7fe098093040 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe098093040 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x7fe098093040 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.394+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe098093040 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x7fe098093040 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.395+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.396+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe098093040 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x7fe098093040 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.397+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.398+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.398+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.398+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.398+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe098093040 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.398+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x7fe098093040 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.398+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe098093040 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe09c02b040 con 0x7fe098093040 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.399+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x55a48a9d8800 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1650 con 0x7fe0a0033b20 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e06e0 con 0x55a48a9d8800 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.400+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e0f30 con 0x55a48a9d8800 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe09c02b040 con 0x55a48a9d8800 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1ad0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 zone default found 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 4 Realm: () 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:23:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 zonegroup default 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e1e90 con 0x7fe098093040 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe09c02b040 con 0x7fe098093040 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e4170 con 0x55a48a9dd1f0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.401+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a972e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e4510 con 0x55a48a9dd1f0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a972e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:23:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a972e800 20 started sync module instance, tier type = 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a972e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.402+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e3600 con 0x7fe0a0033b20 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.403+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.403+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94165188622880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e4e00 con 0x7fe0a0033b20 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.403+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e5650 con 0x55a48a9d8800 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.404+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94165188622880] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.404+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.404+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94165188626768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e5650 con 0x55a48a9d8800 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.404+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e6140 con 0x55a48a9dd1f0 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.405+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94165188626768] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.405+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.405+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94165188631568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e5650 con 0x55a48a9dd1f0 2024-09-19T11:23:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.405+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e6b90 con 0x7fe0a0033b20 2024-09-19T11:23:45.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.406+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94165188631568] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.407+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.407+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94165188628896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e55c0 con 0x7fe0a0033b20 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.407+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e7a80 con 0x55a48a9d8800 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.408+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.408+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94165188638160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e7a30 con 0x55a48a9d8800 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.408+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e8a40 con 0x7fe098093040 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.408+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94165188628896] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.409+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94165188638160] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.409+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.409+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94165188642032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e8a00 con 0x7fe098093040 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.409+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e7f90 con 0x7fe098093040 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.410+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.410+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94165188644320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e7f90 con 0x7fe098093040 2024-09-19T11:23:45.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.410+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e9e40 con 0x7fe098093040 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.410+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94165188642032] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94165188644320] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94165188636592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e9e00 con 0x7fe098093040 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 20 add_watcher() i=0 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 20 add_watcher() i=1 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 20 add_watcher() i=2 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 20 add_watcher() i=3 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 20 add_watcher() i=4 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 20 add_watcher() i=5 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.411+0000 7fe0a972e800 20 add_watcher() i=6 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.412+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94165188636592] v33'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.412+0000 7fe0a972e800 20 add_watcher() i=7 2024-09-19T11:23:45.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.412+0000 7fe0a972e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.412+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9ece30 con 0x55a48a9d8800 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.412+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.412+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94165188647248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9ea250 con 0x55a48a9d8800 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.413+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94165188647248] v33'191 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.413+0000 7fdf5b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.413+0000 7fe0a972e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.413+0000 7fe0a972e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.414+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f5660 con 0x55a48a9d8800 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.415+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.415+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f5660 con 0x7fe098093040 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.416+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.416+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6080 con 0x55a48a9dd1f0 2024-09-19T11:23:45.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.417+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.417+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6080 con 0x55a48a9d8800 2024-09-19T11:23:45.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.418+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'192 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.418+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6080 con 0x7fe0a0033b20 2024-09-19T11:23:45.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.419+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'65 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.419+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6610 con 0x55a48a9dd1f0 2024-09-19T11:23:45.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.420+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'43 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.420+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6be0 con 0x55a48a9d8800 2024-09-19T11:23:45.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.421+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.421+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6be0 con 0x7fe098093040 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.422+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'43 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.422+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6be0 con 0x7fe0a0033b20 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.423+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'26 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.423+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6be0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.424+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.424+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6be0 con 0x7fe0a0033b20 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.425+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'66 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.425+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6fc0 con 0x7fe098093040 2024-09-19T11:23:45.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.426+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.426+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6db0 con 0x7fe0a0033b20 2024-09-19T11:23:45.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.427+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.427+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f6db0 con 0x7fe098093040 2024-09-19T11:23:45.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.429+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'50 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.429+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7090 con 0x55a48a9d8800 2024-09-19T11:23:45.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.430+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'193 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.430+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.431+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'59 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.431+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x7fe0a0033b20 2024-09-19T11:23:45.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.432+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'67 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.432+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.433+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.433+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9d8800 2024-09-19T11:23:45.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.434+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'33 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.434+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9d8800 2024-09-19T11:23:45.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.435+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'194 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.435+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.437+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'60 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.437+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x7fe0a0033b20 2024-09-19T11:23:45.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.438+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'35 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.438+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.438+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.439+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.439+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.439+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.440+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'21 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.440+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.442+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'44 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.442+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x7fe098093040 2024-09-19T11:23:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.443+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'29 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.443+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.444+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.444+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x7fe098093040 2024-09-19T11:23:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.445+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'44 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.445+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x55a48a9dd1f0 2024-09-19T11:23:45.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.446+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'61 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.446+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x7fe098093040 2024-09-19T11:23:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.447+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'51 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.447+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9f7070 con 0x7fe0a0033b20 2024-09-19T11:23:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.448+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fdf5a7e4640 20 reqs_thread_entry: start 2024-09-19T11:23:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fe0a972e800 20 init_complete bucket index max shards: 11 2024-09-19T11:23:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fe0a972e800 20 Filter name: none 2024-09-19T11:23:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fdecbfff640 20 reqs_thread_entry: start 2024-09-19T11:23:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fe0a972e800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:23:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fe0a972e800 20 rados->read ofs=0 len=0 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48aa218d0 con 0x55a48a9d8800 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.449+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.450+0000 7fe0a972e800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.450+0000 7fe0a972e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.450+0000 7fe0a972e800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.450+0000 7fe0a972e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.450+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a48aa214a0 con 0x7fe0a0033b20 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.450+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv41 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-09-19T11:23:45.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.451+0000 7fdf5b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:23:45.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.451+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94165188647248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9d49e0 con 0x55a48a9d8800 2024-09-19T11:23:45.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.452+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94165188647248] v33'195 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.452+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94165188622880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e9950 con 0x7fe0a0033b20 2024-09-19T11:23:45.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.453+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94165188622880] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.453+0000 7fe0a972e800 20 remove_watcher() i=0 2024-09-19T11:23:45.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.453+0000 7fe0a972e800 2 removed watcher, disabling cache 2024-09-19T11:23:45.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.453+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94165188626768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e9950 con 0x55a48a9d8800 2024-09-19T11:23:45.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.454+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94165188626768] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.454+0000 7fe0a972e800 20 remove_watcher() i=1 2024-09-19T11:23:45.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.454+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94165188631568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e4950 con 0x55a48a9dd1f0 2024-09-19T11:23:45.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.455+0000 7fe0a5cde640 1 -- 172.21.15.28:0/2786429384 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94165188631568] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0a000a4d0 con 0x55a48a9dd1f0 2024-09-19T11:23:45.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.455+0000 7fe0a972e800 20 remove_watcher() i=2 2024-09-19T11:23:45.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.455+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94165188628896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e9950 con 0x7fe0a0033b20 2024-09-19T11:23:45.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.456+0000 7fe0a54dd640 1 -- 172.21.15.28:0/2786429384 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94165188628896] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe0980a1030 con 0x7fe0a0033b20 2024-09-19T11:23:45.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.456+0000 7fe0a972e800 20 remove_watcher() i=3 2024-09-19T11:23:45.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.456+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94165188638160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e5c10 con 0x55a48a9d8800 2024-09-19T11:23:45.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.457+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94165188638160] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x55a48a9d8800 2024-09-19T11:23:45.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.457+0000 7fe0a972e800 20 remove_watcher() i=4 2024-09-19T11:23:45.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.457+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94165188642032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e4950 con 0x7fe098093040 2024-09-19T11:23:45.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.458+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94165188642032] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.458+0000 7fe0a972e800 20 remove_watcher() i=5 2024-09-19T11:23:45.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.458+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94165188644320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e51a0 con 0x7fe098093040 2024-09-19T11:23:45.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.459+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94165188644320] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.459+0000 7fe0a972e800 20 remove_watcher() i=6 2024-09-19T11:23:45.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.459+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94165188636592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a48a9e51a0 con 0x7fe098093040 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94165188636592] v33'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe09c03b040 con 0x7fe098093040 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 20 remove_watcher() i=7 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe098093040 msgr2=0x7fe09808f0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fe098093040 0x7fe09808f0d0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a9d8800 msgr2=0x55a48a9dacc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a9d8800 0x55a48a9dacc0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe0a0033b20 msgr2=0x7fe0a0031a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fe0a0033b20 0x7fe0a0031a00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9dd1f0 msgr2=0x55a48a9df5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.460+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9dd1f0 0x55a48a9df5e0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7003d000 msgr2=0x7fdf7003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7003d000 0x7fdf7003f4c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe098051840 tx=0x7fe09808b000 comp rx=0 tx=0).stop 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 msgr2=0x55a48a9ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a4cdc640 1 -- 172.21.15.28:0/2786429384 reap_dead start 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9ccf20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0054740 tx=0x7fe0a0054770 comp rx=0 tx=0).stop 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 shutdown_connections 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/2786429384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9c58d0 0x55a48a9ccf20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 >> 172.21.15.28:0/2786429384 conn(0x55a48a9c1cd0 msgr2=0x55a48a9bfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 shutdown_connections 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/2786429384 wait complete. 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7c03d000 msgr2=0x7fdf7c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7c03d000 0x7fdf7c03f4c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55a48a9aa890 tx=0x7fe09806ef00 comp rx=0 tx=0).stop 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9b8e50 msgr2=0x55a48a9b9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9b8e50 0x55a48a9b9270 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55a48a99ed30 tx=0x7fe0a00167b0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 shutdown_connections 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf7c03d000 0x7fdf7c03f4c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/59117410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a9b8e50 0x55a48a9b9270 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 >> 172.21.15.28:0/59117410 conn(0x55a48a9ad200 msgr2=0x55a48a9ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 shutdown_connections 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/59117410 wait complete. 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a8a3660 msgr2=0x55a48a9a54f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a8a3660 0x55a48a9a54f0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9a7890 msgr2=0x55a48a9a9d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.461+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9a7890 0x55a48a9a9d30 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf8c03cfb0 msgr2=0x7fdf8c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf8c03cfb0 0x7fdf8c03f470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe09c001e30 tx=0x7fe09c001390 comp rx=0 tx=0).stop 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 msgr2=0x55a48a99dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a99dea0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe098067040 tx=0x7fe098002fb0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 shutdown_connections 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a48a9a7890 0x55a48a9a9d30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdf8c03cfb0 0x7fdf8c03f470 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a48a8a3660 0x55a48a9a54f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 --2- 172.21.15.28:0/3759464109 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a48a8a2d20 0x55a48a99dea0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 >> 172.21.15.28:0/3759464109 conn(0x55a48a89ea50 msgr2=0x55a48a997860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 shutdown_connections 2024-09-19T11:23:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:23:45.462+0000 7fe0a972e800 1 -- 172.21.15.28:0/3759464109 wait complete. 2024-09-19T11:23:45.472 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-19T11:23:45.486 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:23:45.529 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-09-19T11:24:20.530 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-09-19T11:24:20.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-09-19T11:24:20.531 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-09-19T11:24:20.585 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:20.585 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:20.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.584+0000 7f2eae920800 1 Processor -- start 2024-09-19T11:24:20.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.584+0000 7f2eae920800 1 -- start start 2024-09-19T11:24:20.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.584+0000 7f2eae920800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55564de17890 0x55564de18450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.584+0000 7f2eae920800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55564de17f40 con 0x55564de18990 2024-09-19T11:24:20.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.584+0000 7f2eae920800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55564dca27d0 con 0x55564de17890 2024-09-19T11:24:20.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.585+0000 7f2eaa502640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55564de17890 0x55564de18450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.585+0000 7f2eaa502640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55564de17890 0x55564de18450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56202/0 (socket says 172.21.15.28:56202) 2024-09-19T11:24:20.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.585+0000 7f2eaa502640 1 -- 172.21.15.28:0/2522650257 learned_addr learned my addr 172.21.15.28:0/2522650257 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.585+0000 7f2ea9d01640 1 --1- 172.21.15.28:0/2522650257 >> v1:172.21.15.22:6789/0 conn(0x55564de18990 0x55564de1d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59282/0 (socket says 172.21.15.28:59282) 2024-09-19T11:24:20.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.585+0000 7f2eaa502640 1 -- 172.21.15.28:0/2522650257 >> v1:172.21.15.22:6789/0 conn(0x55564de18990 legacy=0x55564de1d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.585+0000 7f2eaa502640 1 -- 172.21.15.28:0/2522650257 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564de18e40 con 0x55564de17890 2024-09-19T11:24:20.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.587+0000 7f2eaa502640 1 --2- 172.21.15.28:0/2522650257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564de18450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00a520 tx=0x7f2e9c045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ace786b478cf23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.587+0000 7f2ea9500640 1 -- 172.21.15.28:0/2522650257 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c055070 con 0x55564de17890 2024-09-19T11:24:20.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.587+0000 7f2ea9500640 1 -- 172.21.15.28:0/2522650257 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c0515b0 con 0x55564de17890 2024-09-19T11:24:20.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.587+0000 7f2ea9500640 1 -- 172.21.15.28:0/2522650257 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c0518d0 con 0x55564de17890 2024-09-19T11:24:20.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.588+0000 7f2eae920800 1 -- 172.21.15.28:0/2522650257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 msgr2=0x55564de18450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.588+0000 7f2eae920800 1 --2- 172.21.15.28:0/2522650257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564de18450 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00a520 tx=0x7f2e9c045ba0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.588+0000 7f2eae920800 1 -- 172.21.15.28:0/2522650257 shutdown_connections 2024-09-19T11:24:20.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.588+0000 7f2eae920800 1 --2- 172.21.15.28:0/2522650257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564de18450 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.588+0000 7f2eae920800 1 -- 172.21.15.28:0/2522650257 >> 172.21.15.28:0/2522650257 conn(0x55564de12c60 msgr2=0x55564de150c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.588+0000 7f2eae920800 1 -- 172.21.15.28:0/2522650257 shutdown_connections 2024-09-19T11:24:20.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.588+0000 7f2eae920800 1 -- 172.21.15.28:0/2522650257 wait complete. 2024-09-19T11:24:20.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.605+0000 7f2eae920800 1 Processor -- start 2024-09-19T11:24:20.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.605+0000 7f2eae920800 1 -- start start 2024-09-19T11:24:20.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.606+0000 7f2eae920800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564df07330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.606+0000 7f2eae920800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55564dcafdd0 con 0x55564de17890 2024-09-19T11:24:20.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.606+0000 7f2eaa502640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564df07330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.606+0000 7f2eaa502640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564df07330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56214/0 (socket says 172.21.15.28:56214) 2024-09-19T11:24:20.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.606+0000 7f2eaa502640 1 -- 172.21.15.28:0/1438417088 learned_addr learned my addr 172.21.15.28:0/1438417088 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.606+0000 7f2eaa502640 1 -- 172.21.15.28:0/1438417088 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564df07920 con 0x55564de17890 2024-09-19T11:24:20.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.607+0000 7f2eaa502640 1 --2- 172.21.15.28:0/1438417088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564df07330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c025da0 tx=0x7f2e9c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d41c4359ca21d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.607+0000 7f2d98608640 1 -- 172.21.15.28:0/1438417088 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c051950 con 0x55564de17890 2024-09-19T11:24:20.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.607+0000 7f2d98608640 1 -- 172.21.15.28:0/1438417088 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c051af0 con 0x55564de17890 2024-09-19T11:24:20.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.607+0000 7f2d98608640 1 -- 172.21.15.28:0/1438417088 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c051dd0 con 0x55564de17890 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.607+0000 7f2eae920800 1 -- 172.21.15.28:0/1438417088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 msgr2=0x55564df07330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.607+0000 7f2eae920800 1 --2- 172.21.15.28:0/1438417088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564df07330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c025da0 tx=0x7f2e9c051000 comp rx=0 tx=0).stop 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 -- 172.21.15.28:0/1438417088 shutdown_connections 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 --2- 172.21.15.28:0/1438417088 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564df07330 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 -- 172.21.15.28:0/1438417088 >> 172.21.15.28:0/1438417088 conn(0x55564de12c60 msgr2=0x55564df02cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 -- 172.21.15.28:0/1438417088 shutdown_connections 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 -- 172.21.15.28:0/1438417088 wait complete. 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 Processor -- start 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 -- start start 2024-09-19T11:24:20.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564dd8d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eae920800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55564de1ee10 con 0x55564de17890 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eaa502640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564dd8d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eaa502640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564dd8d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56220/0 (socket says 172.21.15.28:56220) 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.608+0000 7f2eaa502640 1 -- 172.21.15.28:0/3164265923 learned_addr learned my addr 172.21.15.28:0/3164265923 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.609+0000 7f2eaa502640 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564dd8d650 con 0x55564de17890 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.609+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564dd8d0f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c067040 tx=0x7f2e9c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.609+0000 7f2d96e05640 1 -- 172.21.15.28:0/3164265923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c073070 con 0x55564de17890 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.609+0000 7f2d96e05640 1 -- 172.21.15.28:0/3164265923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c05a450 con 0x55564de17890 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.609+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55564dd8ba00 con 0x55564de17890 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.609+0000 7f2d96e05640 1 -- 172.21.15.28:0/3164265923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e9c05a730 con 0x55564de17890 2024-09-19T11:24:20.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.609+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564dd8c020 con 0x55564de17890 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.610+0000 7f2d96e05640 1 -- 172.21.15.28:0/3164265923 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2e9c076070 con 0x55564de17890 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.610+0000 7f2d96e05640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d9003cfb0 0x7f2d9003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.610+0000 7f2d96e05640 1 -- 172.21.15.28:0/3164265923 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f2e9c062070 con 0x55564de17890 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.611+0000 7f2ea9d01640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d9003cfb0 0x7f2d9003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.611+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564de18990 0x55564df180d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.611+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df18720 con 0x55564de18990 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.611+0000 7f2eaad03640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564de18990 0x55564df180d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.611+0000 7f2ea9d01640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d9003cfb0 0x7f2d9003f470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0001e30 tx=0x7f2ea0001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.611+0000 7f2eaad03640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564de18990 0x55564df180d0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.611+0000 7f2eaad03640 1 -- 172.21.15.28:0/3164265923 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564de18990 2024-09-19T11:24:20.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.612+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df19000 con 0x55564de18990 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.612+0000 7f2eaad03640 1 -- 172.21.15.28:0/3164265923 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564de18990 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.612+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df1aa30 0x55564df1cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.612+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df1d630 con 0x55564df1aa30 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.612+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df1aa30 0x55564df1cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.612+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df1aa30 0x55564df1cea0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.613+0000 7f2eaa502640 1 -- 172.21.15.28:0/3164265923 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2e9c075030 con 0x55564df1aa30 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.613+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df19850 con 0x55564de18990 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.613+0000 7f2eaad03640 1 -- 172.21.15.28:0/3164265923 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564de18990 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.613+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df19bf0 con 0x55564df1aa30 2024-09-19T11:24:20.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eaa502640 1 -- 172.21.15.28:0/3164265923 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2e9c075030 con 0x55564df1aa30 2024-09-19T11:24:20.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eae920800 1 Processor -- start 2024-09-19T11:24:20.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eae920800 1 -- start start 2024-09-19T11:24:20.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eae920800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df2bea0 0x55564df2c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eae920800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55564dd0e4a0 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eaad03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df2bea0 0x55564df2c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eaad03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df2bea0 0x55564df2c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56232/0 (socket says 172.21.15.28:56232) 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.614+0000 7f2eaad03640 1 -- 172.21.15.28:0/3231548669 learned_addr learned my addr 172.21.15.28:0/3231548669 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.615+0000 7f2eaad03640 1 -- 172.21.15.28:0/3231548669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564df2c900 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.615+0000 7f2eaad03640 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df2bea0 0x55564df2c2c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55564dd8cb20 tx=0x7f2ea40167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.615+0000 7f2d8ffff640 1 -- 172.21.15.28:0/3231548669 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4013640 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.615+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55564df2d130 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.615+0000 7f2d8ffff640 1 -- 172.21.15.28:0/3231548669 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea40137e0 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.615+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564df2d6d0 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.615+0000 7f2d8ffff640 1 -- 172.21.15.28:0/3231548669 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4013ac0 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.616+0000 7f2d8ffff640 1 -- 172.21.15.28:0/3231548669 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2ea4011040 con 0x55564df2bea0 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2d8ffff640 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d8403cfb0 0x7f2d8403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2d8ffff640 1 -- 172.21.15.28:0/3231548669 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f2ea400e040 con 0x55564df2bea0 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d8403cfb0 0x7f2d8403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eae920800 1 Processor -- start 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eae920800 1 -- start start 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d8403cfb0 0x7f2d8403f470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55564dd8c780 tx=0x7f2e9c0504e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eae920800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df39df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eae920800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55564df07c90 con 0x55564df399f0 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eaad03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df39df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eaad03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df39df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56248/0 (socket says 172.21.15.28:56248) 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.617+0000 7f2eaad03640 1 -- 172.21.15.28:0/2494113106 learned_addr learned my addr 172.21.15.28:0/2494113106 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2eaad03640 1 -- 172.21.15.28:0/2494113106 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564df3a3c0 con 0x55564df399f0 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2eaad03640 1 --2- 172.21.15.28:0/2494113106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df39df0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4016cd0 tx=0x7f2ea401f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9368dfb14b288 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2d8dffb640 1 -- 172.21.15.28:0/2494113106 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4044070 con 0x55564df399f0 2024-09-19T11:24:20.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2d8dffb640 1 -- 172.21.15.28:0/2494113106 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea40143c0 con 0x55564df399f0 2024-09-19T11:24:20.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2d8dffb640 1 -- 172.21.15.28:0/2494113106 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4021a60 con 0x55564df399f0 2024-09-19T11:24:20.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2eae920800 1 -- 172.21.15.28:0/2494113106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 msgr2=0x55564df39df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2eae920800 1 --2- 172.21.15.28:0/2494113106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df39df0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4016cd0 tx=0x7f2ea401f400 comp rx=0 tx=0).stop 2024-09-19T11:24:20.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2eae920800 1 -- 172.21.15.28:0/2494113106 shutdown_connections 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2eae920800 1 --2- 172.21.15.28:0/2494113106 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df39df0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.618+0000 7f2eae920800 1 -- 172.21.15.28:0/2494113106 >> 172.21.15.28:0/2494113106 conn(0x55564df34d80 msgr2=0x55564df32fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eae920800 1 -- 172.21.15.28:0/2494113106 shutdown_connections 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eae920800 1 -- 172.21.15.28:0/2494113106 wait complete. 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eae920800 1 Processor -- start 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eae920800 1 -- start start 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eae920800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df3ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eae920800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55564df388b0 con 0x55564df399f0 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eaad03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df3ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eaad03640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df3ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56256/0 (socket says 172.21.15.28:56256) 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.619+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 learned_addr learned my addr 172.21.15.28:0/3256343554 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.620+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564df40470 con 0x55564df399f0 2024-09-19T11:24:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.620+0000 7f2eaad03640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df3ff10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4053710 tx=0x7f2ea4054820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.620+0000 7f2d7bfff640 1 -- 172.21.15.28:0/3256343554 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4056070 con 0x55564df399f0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.620+0000 7f2d7bfff640 1 -- 172.21.15.28:0/3256343554 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea4056210 con 0x55564df399f0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.620+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55564df40700 con 0x55564df399f0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.620+0000 7f2d7bfff640 1 -- 172.21.15.28:0/3256343554 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea4045a60 con 0x55564df399f0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.620+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564df41080 con 0x55564df399f0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.621+0000 7f2d7bfff640 1 -- 172.21.15.28:0/3256343554 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2ea4045c00 con 0x55564df399f0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.621+0000 7f2d7bfff640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d7403cfb0 0x7f2d7403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.621+0000 7f2d7bfff640 1 -- 172.21.15.28:0/3256343554 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f2ea4017c80 con 0x55564df399f0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.621+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d7403cfb0 0x7f2d7403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.622+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.622+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564df4b7e0 0x55564df4dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.622+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df4e3f0 con 0x55564df4b7e0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.622+0000 7f2ea9d01640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564df4b7e0 0x55564df4dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.622+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d7403cfb0 0x7f2d7403f470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c050550 tx=0x7f2e9c091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.622+0000 7f2ea9d01640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564df4b7e0 0x55564df4dca0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 20 realm 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df4ab30 con 0x55564df4b7e0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df4eed0 con 0x55564df4b7e0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df4f7a0 con 0x55564df4b7e0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.623+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eae920800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df500e0 0x55564df524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df52c10 con 0x55564df500e0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eaad03640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df500e0 0x55564df524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eaad03640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df500e0 0x55564df524d0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eae920800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eae920800 20 searching for the correct realm 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.624+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df500e0 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaad03640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f2ea4017450 0x7f2ea403ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x7f2ea4017450 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f2ea4017450 0x7f2ea403ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f2ea4017450 0x7f2ea403ca60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaa502640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f2e9c090070 0x7f2e9c098700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.625+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x7f2e9c090070 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.626+0000 7f2ea9d01640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f2e9c090070 0x7f2e9c098700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.626+0000 7f2ea9d01640 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f2e9c090070 0x7f2e9c098700 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.626+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.626+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df4b7e0 2024-09-19T11:24:20.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.627+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.627+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df4b7e0 2024-09-19T11:24:20.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.627+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.627+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x7f2e9c090070 2024-09-19T11:24:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.627+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.627+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x7f2e9c090070 2024-09-19T11:24:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.628+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.628+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x7f2ea4017450 2024-09-19T11:24:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.628+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.628+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df500e0 2024-09-19T11:24:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.628+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.628+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df4b7e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x7f2e9c090070 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df4b7e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x7f2ea4017450 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.629+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df500e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.630+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.630+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55564df545c0 con 0x55564df500e0 2024-09-19T11:24:20.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.630+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.630+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2ea4017450 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.630+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.630+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df500e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df500e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df4b7e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df4b7e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.631+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2e9c090070 2024-09-19T11:24:20.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.632+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.632+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df500e0 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.632+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.632+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2ea4017450 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.632+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.632+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df4b7e0 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.632+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.633+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2e9c090070 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.633+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.633+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df500e0 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.633+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.633+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2e9c090070 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.633+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.633+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2ea4017450 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.634+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.634+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df500e0 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.634+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.634+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2ea4017450 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.634+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.634+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x55564df4b7e0 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f2ea40630c0 con 0x7f2ea4017450 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:20.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df53690 con 0x55564df4b7e0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df532a0 con 0x55564df4b7e0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.635+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df53f80 con 0x55564df500e0 2024-09-19T11:24:20.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 20 zone default found 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 4 Realm: () 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 20 zonegroup default 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df53b30 con 0x7f2e9c090070 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.636+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df56e60 con 0x55564df500e0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eae920800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eae920800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df57230 con 0x55564df500e0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eae920800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eae920800 20 started sync module instance, tier type = 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eae920800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.637+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df57200 con 0x7f2ea4017450 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.638+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.638+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93829163474720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df56a20 con 0x7f2ea4017450 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.638+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df57d50 con 0x55564df4b7e0 2024-09-19T11:24:20.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.639+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93829163474720] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.640+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.640+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93829163481152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df58140 con 0x55564df4b7e0 2024-09-19T11:24:20.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.640+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df58ba0 con 0x55564df500e0 2024-09-19T11:24:20.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.641+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93829163481152] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.641+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.641+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93829163476080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df58ba0 con 0x55564df500e0 2024-09-19T11:24:20.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.641+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df59610 con 0x7f2ea4017450 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.642+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93829163476080] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.642+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.642+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93829163487488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df59a30 con 0x7f2ea4017450 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.642+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5a430 con 0x55564df4b7e0 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.643+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93829163487488] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.644+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.644+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93829163484800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5a820 con 0x55564df4b7e0 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.644+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5b260 con 0x7f2e9c090070 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.645+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.645+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93829163494704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5b220 con 0x7f2e9c090070 2024-09-19T11:24:20.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.645+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93829163484800] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.645+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5c210 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.646+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 93829163494704] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.646+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.646+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93829163491264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5b650 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.646+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5c6c0 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 93829163491264] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93829163498784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5c6c0 con 0x7f2e9c090070 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 20 add_watcher() i=0 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 20 add_watcher() i=1 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 20 add_watcher() i=2 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 20 add_watcher() i=3 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 20 add_watcher() i=4 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 20 add_watcher() i=5 2024-09-19T11:24:20.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.647+0000 7f2eae920800 20 add_watcher() i=6 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.648+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93829163498784] v33'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.648+0000 7f2eae920800 20 add_watcher() i=7 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.648+0000 7f2eae920800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.648+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df5fac0 con 0x55564df4b7e0 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.649+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.649+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93829163500512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df5d010 con 0x55564df4b7e0 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93829163500512] v33'196 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2d60fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2eae920800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2eae920800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2eae920800 5 note: GC not initialized 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2cdb7fe640 20 reqs_thread_entry: start 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2eae920800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2eae920800 20 Filter name: none 2024-09-19T11:24:20.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2cd97fa640 20 reqs_thread_entry: start 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.650+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x55564df4b7e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.651+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.651+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x7f2ea4017450 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.651+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.651+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df4b7e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.651+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.651+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df500e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df500e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df500e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df4b7e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df4b7e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.652+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df500e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.653+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.653+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92fc0 con 0x55564df500e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.653+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.653+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x7f2e9c090070 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.654+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(72 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.654+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df4b7e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.654+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.654+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2ea4017450 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.654+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.654+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df500e0 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.655+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.655+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2e9c090070 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.655+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.655+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2e9c090070 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.655+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.655+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df500e0 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.656+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.656+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2e9c090070 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.656+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(79 usage.17 [call out=185b] v0'0 uv5 ondisk = 0) ==== 152+0+185 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.656+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df4b7e0 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.656+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.656+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df500e0 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.657+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.657+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2e9c090070 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.657+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.657+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2ea4017450 2024-09-19T11:24:20.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.657+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.657+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2e9c090070 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.658+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.658+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.658+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.658+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.658+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.658+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df500e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df500e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df500e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.659+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.660+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x55564df500e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.660+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.660+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55564df92a50 con 0x7f2e9c090070 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.660+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.661+0000 7f2d60fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.661+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93829163500512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x55564df4b7e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.662+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 93829163500512] v33'197 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.662+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93829163474720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x7f2ea4017450 2024-09-19T11:24:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.663+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 93829163474720] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.663+0000 7f2eae920800 20 remove_watcher() i=0 2024-09-19T11:24:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.663+0000 7f2eae920800 2 removed watcher, disabling cache 2024-09-19T11:24:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.663+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93829163481152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x55564df4b7e0 2024-09-19T11:24:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.665+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 93829163481152] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.665+0000 7f2eae920800 20 remove_watcher() i=1 2024-09-19T11:24:20.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.665+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93829163476080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x55564df500e0 2024-09-19T11:24:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.666+0000 7f2eaad03640 1 -- 172.21.15.28:0/3256343554 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 93829163476080] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea400a4d0 con 0x55564df500e0 2024-09-19T11:24:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.666+0000 7f2eae920800 20 remove_watcher() i=2 2024-09-19T11:24:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.666+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93829163487488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x7f2ea4017450 2024-09-19T11:24:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.667+0000 7f2eaa502640 1 -- 172.21.15.28:0/3256343554 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 93829163487488] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2e9c075030 con 0x7f2ea4017450 2024-09-19T11:24:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.667+0000 7f2eae920800 20 remove_watcher() i=3 2024-09-19T11:24:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.667+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93829163484800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x55564df4b7e0 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.668+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 93829163484800] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x55564df4b7e0 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.668+0000 7f2eae920800 20 remove_watcher() i=4 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.668+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93829163494704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x7f2e9c090070 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.669+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 93829163494704] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.669+0000 7f2eae920800 20 remove_watcher() i=5 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.669+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93829163491264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x7f2e9c090070 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.670+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 93829163491264] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.670+0000 7f2eae920800 20 remove_watcher() i=6 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.670+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93829163498784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55564df93000 con 0x7f2e9c090070 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 93829163498784] v33'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ea000f610 con 0x7f2e9c090070 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 20 remove_watcher() i=7 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f2e9c090070 msgr2=0x7f2e9c098700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f2e9c090070 0x7f2e9c098700 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564df4b7e0 msgr2=0x55564df4dca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564df4b7e0 0x55564df4dca0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f2ea4017450 msgr2=0x7f2ea403ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f2ea4017450 0x7f2ea403ca60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df500e0 msgr2=0x55564df524d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.671+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df500e0 0x55564df524d0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d7403cfb0 msgr2=0x7f2d7403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d7403cfb0 0x7f2d7403f470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c050550 tx=0x7f2e9c091000 comp rx=0 tx=0).stop 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 msgr2=0x55564df3ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 --2- 172.21.15.28:0/3256343554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df399f0 0x55564df3ff10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4053710 tx=0x7f2ea4054820 comp rx=0 tx=0).stop 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2ea9d01640 1 -- 172.21.15.28:0/3256343554 reap_dead start 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 shutdown_connections 2024-09-19T11:24:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 >> 172.21.15.28:0/3256343554 conn(0x55564df34d80 msgr2=0x55564df32f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 shutdown_connections 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3256343554 wait complete. 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d8403cfb0 msgr2=0x7f2d8403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d8403cfb0 0x7f2d8403f470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55564dd8c780 tx=0x7f2e9c0504e0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df2bea0 msgr2=0x55564df2c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df2bea0 0x55564df2c2c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55564dd8cb20 tx=0x7f2ea40167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 shutdown_connections 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d8403cfb0 0x7f2d8403f470 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 --2- 172.21.15.28:0/3231548669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564df2bea0 0x55564df2c2c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 >> 172.21.15.28:0/3231548669 conn(0x55564df20250 msgr2=0x55564df20630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 shutdown_connections 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.672+0000 7f2eae920800 1 -- 172.21.15.28:0/3231548669 wait complete. 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564de18990 msgr2=0x55564df180d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564de18990 0x55564df180d0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df1aa30 msgr2=0x55564df1cea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df1aa30 0x55564df1cea0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d9003cfb0 msgr2=0x7f2d9003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d9003cfb0 0x7f2d9003f470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0001e30 tx=0x7f2ea0001390 comp rx=0 tx=0).stop 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 msgr2=0x55564dd8d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564dd8d0f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c067040 tx=0x7f2e9c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 shutdown_connections 2024-09-19T11:24:20.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55564df1aa30 0x55564df1cea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f2d9003cfb0 0x7f2d9003f470 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55564de18990 0x55564df180d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 --2- 172.21.15.28:0/3164265923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55564de17890 0x55564dd8d0f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 >> 172.21.15.28:0/3164265923 conn(0x55564de12c60 msgr2=0x55564df08660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 shutdown_connections 2024-09-19T11:24:20.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.673+0000 7f2eae920800 1 -- 172.21.15.28:0/3164265923 wait complete. 2024-09-19T11:24:20.683 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"anonymous","categories":[],"s3select":{}}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"s3select":{}},{"bucket":"myfoo2","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"anonymous","categories":[],"total":{"bytes_sent":0,"bytes_received":0,"ops":0,"successful_ops":0,"bytes_processed":0,"bytes_returned":0}},{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-09-19T11:24:20.684 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'anonymous', 'categories': [], 's3select': {}}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:24:20.684 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-09-19T11:24:20.684 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-09-19T11:24:20.684 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-09-19T11:24:20.807 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.727+0000 7f1827b3b800 1 Processor -- start 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.727+0000 7f1827b3b800 1 -- start start 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.727+0000 7f1827b3b800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558a9ebde190 0x558a9ebe62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.727+0000 7f1827b3b800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558a9ebe6820 con 0x558a9ebddd90 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.727+0000 7f1827b3b800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x558a9ea687d0 con 0x558a9ebde190 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.728+0000 7f1823024640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558a9ebde190 0x558a9ebe62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.728+0000 7f1823024640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558a9ebde190 0x558a9ebe62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56270/0 (socket says 172.21.15.28:56270) 2024-09-19T11:24:20.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.728+0000 7f1823024640 1 -- 172.21.15.28:0/437362594 learned_addr learned my addr 172.21.15.28:0/437362594 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.728+0000 7f1823825640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x558a9ebddd90 0x558a9ebe2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59290/0 (socket says 172.21.15.28:59290) 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.728+0000 7f1823024640 1 -- 172.21.15.28:0/437362594 >> v1:172.21.15.22:6789/0 conn(0x558a9ebddd90 legacy=0x558a9ebe2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.728+0000 7f1822823640 1 -- 172.21.15.28:0/437362594 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269928253 0 0) 0x558a9ebe6820 con 0x558a9ebddd90 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.728+0000 7f1823024640 1 -- 172.21.15.28:0/437362594 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9ebe7a00 con 0x558a9ebde190 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.730+0000 7f1823024640 1 --2- 172.21.15.28:0/437362594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebde190 0x558a9ebe62e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f180c00a520 tx=0x7f180c045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e7ec1058d715c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.730+0000 7f1822823640 1 -- 172.21.15.28:0/437362594 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f180c055070 con 0x558a9ebde190 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.730+0000 7f1822823640 1 -- 172.21.15.28:0/437362594 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180c0515b0 con 0x558a9ebde190 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.730+0000 7f1822823640 1 -- 172.21.15.28:0/437362594 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f180c0518d0 con 0x558a9ebde190 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.731+0000 7f1827b3b800 1 -- 172.21.15.28:0/437362594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebde190 msgr2=0x558a9ebe62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.731+0000 7f1827b3b800 1 --2- 172.21.15.28:0/437362594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebde190 0x558a9ebe62e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f180c00a520 tx=0x7f180c045ba0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.731+0000 7f1827b3b800 1 -- 172.21.15.28:0/437362594 shutdown_connections 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.731+0000 7f1827b3b800 1 --2- 172.21.15.28:0/437362594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebde190 0x558a9ebe62e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.731+0000 7f1827b3b800 1 -- 172.21.15.28:0/437362594 >> 172.21.15.28:0/437362594 conn(0x558a9ebd71b0 msgr2=0x558a9ebd9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.731+0000 7f1827b3b800 1 -- 172.21.15.28:0/437362594 shutdown_connections 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.731+0000 7f1827b3b800 1 -- 172.21.15.28:0/437362594 wait complete. 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.746+0000 7f1827b3b800 1 Processor -- start 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.746+0000 7f1827b3b800 1 -- start start 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.746+0000 7f1827b3b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9eccf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.746+0000 7f1827b3b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a9ea75dd0 con 0x558a9ebddd90 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1823825640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9eccf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1823825640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9eccf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56280/0 (socket says 172.21.15.28:56280) 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1823825640 1 -- 172.21.15.28:0/4294581158 learned_addr learned my addr 172.21.15.28:0/4294581158 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1823825640 1 -- 172.21.15.28:0/4294581158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9eccf910 con 0x558a9ebddd90 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1823825640 1 --2- 172.21.15.28:0/4294581158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9eccf320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fa0 tx=0x7f1814001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc57d9a365fe07e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1711e0b640 1 -- 172.21.15.28:0/4294581158 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1814003450 con 0x558a9ebddd90 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1711e0b640 1 -- 172.21.15.28:0/4294581158 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1814003610 con 0x558a9ebddd90 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.747+0000 7f1711e0b640 1 -- 172.21.15.28:0/4294581158 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1814003970 con 0x558a9ebddd90 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 -- 172.21.15.28:0/4294581158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 msgr2=0x558a9eccf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 --2- 172.21.15.28:0/4294581158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9eccf320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fa0 tx=0x7f1814001b00 comp rx=0 tx=0).stop 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 -- 172.21.15.28:0/4294581158 shutdown_connections 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 --2- 172.21.15.28:0/4294581158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9eccf320 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 -- 172.21.15.28:0/4294581158 >> 172.21.15.28:0/4294581158 conn(0x558a9ebd71b0 msgr2=0x558a9eccaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 -- 172.21.15.28:0/4294581158 shutdown_connections 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 -- 172.21.15.28:0/4294581158 wait complete. 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 Processor -- start 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.748+0000 7f1827b3b800 1 -- start start 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f1827b3b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9ecd6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f1827b3b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a9ebe6cf0 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f1823825640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9ecd6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f1823825640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9ecd6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56284/0 (socket says 172.21.15.28:56284) 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f1823825640 1 -- 172.21.15.28:0/3207532575 learned_addr learned my addr 172.21.15.28:0/3207532575 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f1823825640 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9ecd73d0 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f1823825640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9ecd6e70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1814001330 tx=0x7f1814001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f170bfff640 1 -- 172.21.15.28:0/3207532575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1814014a10 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.749+0000 7f170bfff640 1 -- 172.21.15.28:0/3207532575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1814014bb0 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.750+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a9ecd7660 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.750+0000 7f170bfff640 1 -- 172.21.15.28:0/3207532575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f181401e3f0 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.750+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9ecd8060 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.750+0000 7f170bfff640 1 -- 172.21.15.28:0/3207532575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1814010070 con 0x558a9ebddd90 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f170bfff640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f170c03d000 0x7f170c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f170bfff640 1 -- 172.21.15.28:0/3207532575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f1814018070 con 0x558a9ebddd90 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f1823024640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f170c03d000 0x7f170c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ebde190 0x558a9ecde4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ecdea00 con 0x558a9ebde190 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f1824026640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ebde190 0x558a9ecde4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f1823024640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f170c03d000 0x7f170c03f4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f180c057000 tx=0x7f180c052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.751+0000 7f1824026640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ebde190 0x558a9ecde4c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.752+0000 7f1824026640 1 -- 172.21.15.28:0/3207532575 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ebde190 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.752+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ecdeda0 con 0x558a9ebde190 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.752+0000 7f1824026640 1 -- 172.21.15.28:0/3207532575 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ebde190 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.752+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ece0860 0x558a9ece2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.752+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ece3460 con 0x558a9ece0860 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.752+0000 7f1823825640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ece0860 0x558a9ece2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.752+0000 7f1823825640 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ece0860 0x558a9ece2cd0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.753+0000 7f1823825640 1 -- 172.21.15.28:0/3207532575 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f181401b070 con 0x558a9ece0860 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.753+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ecdf5f0 con 0x558a9ebde190 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.753+0000 7f1824026640 1 -- 172.21.15.28:0/3207532575 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ebde190 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.753+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ecdf990 con 0x558a9ece0860 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.753+0000 7f1823825640 1 -- 172.21.15.28:0/3207532575 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f181401b070 con 0x558a9ece0860 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1827b3b800 1 Processor -- start 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1827b3b800 1 -- start start 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1827b3b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecf1e30 0x558a9ecf2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1827b3b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a9ead44a0 con 0x558a9ecf1e30 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1824026640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecf1e30 0x558a9ecf2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1824026640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecf1e30 0x558a9ecf2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56286/0 (socket says 172.21.15.28:56286) 2024-09-19T11:24:20.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1824026640 1 -- 172.21.15.28:0/2215885009 learned_addr learned my addr 172.21.15.28:0/2215885009 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.754+0000 7f1824026640 1 -- 172.21.15.28:0/2215885009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9ecf2890 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.755+0000 7f1824026640 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecf1e30 0x558a9ecf2250 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x558a9ecd8d50 tx=0x7f181c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.755+0000 7f17097fa640 1 -- 172.21.15.28:0/2215885009 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f181c01c020 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.755+0000 7f17097fa640 1 -- 172.21.15.28:0/2215885009 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181c007c90 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.755+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a9ecf30c0 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.755+0000 7f17097fa640 1 -- 172.21.15.28:0/2215885009 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f181c018440 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.755+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9ecf3660 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f17097fa640 1 -- 172.21.15.28:0/2215885009 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f181c01f020 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f17097fa640 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16fc03d070 0x7f16fc03f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f17097fa640 1 -- 172.21.15.28:0/2215885009 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f181c055b00 con 0x558a9ecf1e30 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f1823825640 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16fc03d070 0x7f16fc03f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f1827b3b800 1 Processor -- start 2024-09-19T11:24:20.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f1827b3b800 1 -- start start 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f1823825640 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16fc03d070 0x7f16fc03f530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1814003fd0 tx=0x7f1814003f60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f1827b3b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ecfed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.756+0000 7f1827b3b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a9eccfd40 con 0x558a9ecfe930 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1824026640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ecfed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1824026640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ecfed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56300/0 (socket says 172.21.15.28:56300) 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1824026640 1 -- 172.21.15.28:0/2297094584 learned_addr learned my addr 172.21.15.28:0/2297094584 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1824026640 1 -- 172.21.15.28:0/2297094584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9ecff320 con 0x558a9ecfe930 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1824026640 1 --2- 172.21.15.28:0/2297094584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ecfed30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f181c053d80 tx=0x7f181c054050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c448d74383391dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1702ffd640 1 -- 172.21.15.28:0/2297094584 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f181c052070 con 0x558a9ecfe930 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1702ffd640 1 -- 172.21.15.28:0/2297094584 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181c0156f0 con 0x558a9ecfe930 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.757+0000 7f1702ffd640 1 -- 172.21.15.28:0/2297094584 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f181c016c50 con 0x558a9ecfe930 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 -- 172.21.15.28:0/2297094584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 msgr2=0x558a9ecfed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2297094584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ecfed30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f181c053d80 tx=0x7f181c054050 comp rx=0 tx=0).stop 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 -- 172.21.15.28:0/2297094584 shutdown_connections 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2297094584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ecfed30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 -- 172.21.15.28:0/2297094584 >> 172.21.15.28:0/2297094584 conn(0x558a9ecfad30 msgr2=0x558a9ecf8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 -- 172.21.15.28:0/2297094584 shutdown_connections 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 -- 172.21.15.28:0/2297094584 wait complete. 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 Processor -- start 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 -- start start 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ed05ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.758+0000 7f1827b3b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558a9ecff910 con 0x558a9ecfe930 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f1824026640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ed05ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f1824026640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ed05ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56314/0 (socket says 172.21.15.28:56314) 2024-09-19T11:24:20.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 learned_addr learned my addr 172.21.15.28:0/2600480809 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9ed06400 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f1824026640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ed05ea0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f181c015a80 tx=0x7f181c053d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f17017fa640 1 -- 172.21.15.28:0/2600480809 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f181c013660 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f17017fa640 1 -- 172.21.15.28:0/2600480809 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181c02e070 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f17017fa640 1 -- 172.21.15.28:0/2600480809 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f181c0139e0 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558a9ed06690 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.759+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9ed07220 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.760+0000 7f17017fa640 1 -- 172.21.15.28:0/2600480809 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f181c016040 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.760+0000 7f17017fa640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16ec03d000 0x7f16ec03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.760+0000 7f17017fa640 1 -- 172.21.15.28:0/2600480809 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f181c034d80 con 0x558a9ecfe930 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.761+0000 7f1823825640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16ec03d000 0x7f16ec03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.761+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.761+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ed11840 0x558a9ed13d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.761+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed14450 con 0x558a9ed11840 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.761+0000 7f1823024640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ed11840 0x558a9ed13d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.761+0000 7f1823825640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16ec03d000 0x7f16ec03f4c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1814029970 tx=0x7f1814037420 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.761+0000 7f1823024640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ed11840 0x558a9ed13d00 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 realm 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed14880 con 0x558a9ed11840 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed15150 con 0x558a9ed11840 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed154f0 con 0x558a9ed11840 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ed16250 0x558a9ed18640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.762+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed18b80 con 0x558a9ed16250 2024-09-19T11:24:20.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1824026640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ed16250 0x558a9ed18640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1824026640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ed16250 0x558a9ed18640 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1827b3b800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1827b3b800 20 searching for the correct realm 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1824026640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f181c04e070 0x7f181c028300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.763+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.764+0000 7f1823825640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f181c04e070 0x7f181c028300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.764+0000 7f1823825640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f181c04e070 0x7f181c028300 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.764+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.764+0000 7f1823825640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f18140123c0 0x7f18140113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.764+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f18140123c0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.764+0000 7f1823024640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f18140123c0 0x7f18140113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.765+0000 7f1823024640 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f18140123c0 0x7f18140113e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.765+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.765+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.765+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.765+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f18140123c0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f18140123c0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.766+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.767+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.767+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.767+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.767+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f18140123c0 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.767+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.767+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.768+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.769+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f18140123c0 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.770+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f18140123c0 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f18140123c0 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.771+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed16250 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x558a9ed11840 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.772+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1a4f0 con 0x7f181c04e070 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed195c0 con 0x558a9ed11840 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed19200 con 0x558a9ed11840 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.773+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed19f20 con 0x558a9ed16250 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 zone default found 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 4 Realm: () 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 zonegroup default 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed19a90 con 0x7f18140123c0 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1cd90 con 0x558a9ed16250 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.774+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1cd90 con 0x558a9ed16250 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.775+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.775+0000 7f1827b3b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.775+0000 7f1827b3b800 20 started sync module instance, tier type = 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.775+0000 7f1827b3b800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.775+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1c220 con 0x7f181c04e070 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.776+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.776+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94053858395712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1d1f0 con 0x7f181c04e070 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.776+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1da40 con 0x558a9ed11840 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.777+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94053858395712] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.777+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.777+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94053858399632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1de60 con 0x558a9ed11840 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.777+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1e6f0 con 0x558a9ed16250 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.778+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.778+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94053858401888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1e2e0 con 0x558a9ed16250 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.778+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1f670 con 0x7f181c04e070 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.779+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94053858399632] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.780+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.780+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94053858406752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1f630 con 0x7f181c04e070 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.780+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94053858401888] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.780+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed20100 con 0x558a9ed11840 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.781+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94053858406752] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.781+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.781+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94053858400688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed20100 con 0x558a9ed11840 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.781+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed209a0 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.782+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.782+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94053858410752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed20a50 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.782+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed21550 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.782+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94053858400688] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.783+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94053858410752] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.783+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.783+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94053858411808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1eae0 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.783+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed21b30 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94053858411808] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94053858414656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed21b30 con 0x7f18140123c0 2024-09-19T11:24:20.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 20 add_watcher() i=0 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 20 add_watcher() i=1 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 20 add_watcher() i=2 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 20 add_watcher() i=3 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 20 add_watcher() i=4 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 20 add_watcher() i=5 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.784+0000 7f1827b3b800 20 add_watcher() i=6 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.785+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94053858414656] v33'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.785+0000 7f1827b3b800 20 add_watcher() i=7 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.785+0000 7f1827b3b800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.785+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed24db0 con 0x558a9ed11840 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.786+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.786+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94053858416624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed220c0 con 0x558a9ed11840 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94053858416624] v33'198 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f16d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:20.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f1827b3b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f1827b3b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f1827b3b800 5 note: GC not initialized 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f16d8fe1640 20 reqs_thread_entry: start 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f1827b3b800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f1827b3b800 20 Filter name: none 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f16527fc640 20 reqs_thread_entry: start 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.787+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58300 con 0x558a9ed11840 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58300 con 0x7f181c04e070 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed11840 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed16250 2024-09-19T11:24:20.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.788+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed11840 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed11840 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.789+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.790+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.790+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.790+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.790+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed58780 con 0x7f18140123c0 2024-09-19T11:24:20.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.790+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(72 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.790+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed11840 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f181c04e070 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed16250 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f18140123c0 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.791+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f18140123c0 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.792+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.792+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed16250 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.792+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.792+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f18140123c0 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.793+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(79 usage.17 [call out=185b] v0'0 uv5 ondisk = 0) ==== 152+0+185 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.793+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed11840 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.793+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.793+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed16250 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.793+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.793+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f18140123c0 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f181c04e070 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f18140123c0 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.794+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed16250 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed16250 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.795+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.796+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed16250 2024-09-19T11:24:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.796+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.796+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x558a9ed16250 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.796+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.796+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed59c40 con 0x7f18140123c0 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.796+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.797+0000 7f16d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.797+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94053858416624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed56da0 con 0x558a9ed11840 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.798+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94053858416624] v33'199 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.798+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94053858395712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed56da0 con 0x7f181c04e070 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.799+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94053858395712] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.799+0000 7f1827b3b800 20 remove_watcher() i=0 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.799+0000 7f1827b3b800 2 removed watcher, disabling cache 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.799+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94053858399632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed56da0 con 0x558a9ed11840 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.801+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94053858399632] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.801+0000 7f1827b3b800 20 remove_watcher() i=1 2024-09-19T11:24:20.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.801+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94053858401888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed56da0 con 0x558a9ed16250 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.802+0000 7f1824026640 1 -- 172.21.15.28:0/2600480809 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94053858401888] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181c0083e0 con 0x558a9ed16250 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.802+0000 7f1827b3b800 20 remove_watcher() i=2 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.802+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94053858406752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed56da0 con 0x7f181c04e070 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.803+0000 7f1823825640 1 -- 172.21.15.28:0/2600480809 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94053858406752] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f181401b070 con 0x7f181c04e070 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.803+0000 7f1827b3b800 20 remove_watcher() i=3 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.803+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94053858400688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed56da0 con 0x558a9ed11840 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.804+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94053858400688] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x558a9ed11840 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.804+0000 7f1827b3b800 20 remove_watcher() i=4 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.804+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94053858410752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1da40 con 0x7f18140123c0 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.805+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94053858410752] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.805+0000 7f1827b3b800 20 remove_watcher() i=5 2024-09-19T11:24:20.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.805+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94053858411808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1c640 con 0x7f18140123c0 2024-09-19T11:24:20.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.806+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94053858411808] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.806+0000 7f1827b3b800 20 remove_watcher() i=6 2024-09-19T11:24:20.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.806+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94053858414656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558a9ed1c640 con 0x7f18140123c0 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94053858414656] v33'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f180c057030 con 0x7f18140123c0 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 20 remove_watcher() i=7 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f18140123c0 msgr2=0x7f18140113e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f18140123c0 0x7f18140113e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ed11840 msgr2=0x558a9ed13d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ed11840 0x558a9ed13d00 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f181c04e070 msgr2=0x7f181c028300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f181c04e070 0x7f181c028300 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ed16250 msgr2=0x558a9ed18640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ed16250 0x558a9ed18640 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16ec03d000 msgr2=0x7f16ec03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16ec03d000 0x7f16ec03f4c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1814029970 tx=0x7f1814037420 comp rx=0 tx=0).stop 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 msgr2=0x558a9ed05ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2600480809 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecfe930 0x558a9ed05ea0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f181c015a80 tx=0x7f181c053d10 comp rx=0 tx=0).stop 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 reap_dead start 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1823024640 1 -- 172.21.15.28:0/2600480809 reap_dead start 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 shutdown_connections 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.807+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 >> 172.21.15.28:0/2600480809 conn(0x558a9ecfad30 msgr2=0x558a9ed00080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 shutdown_connections 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2600480809 wait complete. 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16fc03d070 msgr2=0x7f16fc03f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16fc03d070 0x7f16fc03f530 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1814003fd0 tx=0x7f1814003f60 comp rx=0 tx=0).stop 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecf1e30 msgr2=0x558a9ecf2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecf1e30 0x558a9ecf2250 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x558a9ecd8d50 tx=0x7f181c00bf30 comp rx=0 tx=0).stop 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 shutdown_connections 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f16fc03d070 0x7f16fc03f530 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/2215885009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ecf1e30 0x558a9ecf2250 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 >> 172.21.15.28:0/2215885009 conn(0x558a9ece61e0 msgr2=0x558a9ece65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 shutdown_connections 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/2215885009 wait complete. 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ebde190 msgr2=0x558a9ecde4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ebde190 0x558a9ecde4c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ece0860 msgr2=0x558a9ece2cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ece0860 0x558a9ece2cd0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f170c03d000 msgr2=0x7f170c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f170c03d000 0x7f170c03f4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f180c057000 tx=0x7f180c052000 comp rx=0 tx=0).stop 2024-09-19T11:24:20.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 msgr2=0x558a9ecd6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9ecd6e70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1814001330 tx=0x7f1814001250 comp rx=0 tx=0).stop 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.808+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 shutdown_connections 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.809+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558a9ece0860 0x558a9ece2cd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.809+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f170c03d000 0x7f170c03f4c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.809+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558a9ebde190 0x558a9ecde4c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.809+0000 7f1827b3b800 1 --2- 172.21.15.28:0/3207532575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558a9ebddd90 0x558a9ecd6e70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.809+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 >> 172.21.15.28:0/3207532575 conn(0x558a9ebd71b0 msgr2=0x558a9ecd0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.809+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 shutdown_connections 2024-09-19T11:24:20.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.809+0000 7f1827b3b800 1 -- 172.21.15.28:0/3207532575 wait complete. 2024-09-19T11:24:20.847 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo2","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo3","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo4","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]}],"summary":[{"user":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}],"total":{"bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1,"bytes_processed":0,"bytes_returned":0}},{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34,"bytes_processed":0,"bytes_returned":0}}]} 2024-09-19T11:24:20.847 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo2', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo3', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo4', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:24:20.848 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-09-19T11:24:20.848 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-09-19T11:24:20.848 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-09-19T11:24:20.893 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:20.893 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:20.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa80c525800 1 Processor -- start 2024-09-19T11:24:20.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa80c525800 1 -- start start 2024-09-19T11:24:20.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa80c525800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c514982a90 0x55c514984f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa80c525800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c5149855d0 con 0x55c51497ef80 2024-09-19T11:24:20.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa80c525800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55c5148097d0 con 0x55c514982a90 2024-09-19T11:24:20.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa807932640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c514982a90 0x55c514984f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa807932640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55c514982a90 0x55c514984f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56328/0 (socket says 172.21.15.28:56328) 2024-09-19T11:24:20.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa808133640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55c51497ef80 0x55c514981370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59300/0 (socket says 172.21.15.28:59300) 2024-09-19T11:24:20.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.893+0000 7fa807932640 1 -- 172.21.15.28:0/216139900 learned_addr learned my addr 172.21.15.28:0/216139900 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.894+0000 7fa807932640 1 -- 172.21.15.28:0/216139900 >> v1:172.21.15.22:6789/0 conn(0x55c51497ef80 legacy=0x55c514981370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.894+0000 7fa807932640 1 -- 172.21.15.28:0/216139900 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c514986800 con 0x55c514982a90 2024-09-19T11:24:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa807932640 1 --2- 172.21.15.28:0/216139900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514982a90 0x55c514984f50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00a520 tx=0x7fa7fc045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4afc2332b310245 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa807131640 1 -- 172.21.15.28:0/216139900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7fc055070 con 0x55c514982a90 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa807131640 1 -- 172.21.15.28:0/216139900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7fc0515b0 con 0x55c514982a90 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa807131640 1 -- 172.21.15.28:0/216139900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7fc0518d0 con 0x55c514982a90 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa80c525800 1 -- 172.21.15.28:0/216139900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514982a90 msgr2=0x55c514984f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa80c525800 1 --2- 172.21.15.28:0/216139900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514982a90 0x55c514984f50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00a520 tx=0x7fa7fc045ba0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa80c525800 1 -- 172.21.15.28:0/216139900 shutdown_connections 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa80c525800 1 --2- 172.21.15.28:0/216139900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514982a90 0x55c514984f50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa80c525800 1 -- 172.21.15.28:0/216139900 >> 172.21.15.28:0/216139900 conn(0x55c514978a60 msgr2=0x55c51497aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa80c525800 1 -- 172.21.15.28:0/216139900 shutdown_connections 2024-09-19T11:24:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.896+0000 7fa80c525800 1 -- 172.21.15.28:0/216139900 wait complete. 2024-09-19T11:24:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.911+0000 7fa80c525800 1 Processor -- start 2024-09-19T11:24:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.911+0000 7fa80c525800 1 -- start start 2024-09-19T11:24:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.911+0000 7fa80c525800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c51497e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.911+0000 7fa80c525800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c514816dd0 con 0x55c51497ef80 2024-09-19T11:24:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa808133640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c51497e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa808133640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c51497e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56330/0 (socket says 172.21.15.28:56330) 2024-09-19T11:24:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa808133640 1 -- 172.21.15.28:0/2052635898 learned_addr learned my addr 172.21.15.28:0/2052635898 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa808133640 1 -- 172.21.15.28:0/2052635898 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c51497ed00 con 0x55c51497ef80 2024-09-19T11:24:20.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa808133640 1 --2- 172.21.15.28:0/2052635898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c51497e710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b260 tx=0x7fa7f800b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674de7c6ba214516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa6f660c640 1 -- 172.21.15.28:0/2052635898 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7f800be60 con 0x55c51497ef80 2024-09-19T11:24:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa6f660c640 1 -- 172.21.15.28:0/2052635898 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f8006070 con 0x55c51497ef80 2024-09-19T11:24:20.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.912+0000 7fa6f660c640 1 -- 172.21.15.28:0/2052635898 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7f800e710 con 0x55c51497ef80 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.913+0000 7fa80c525800 1 -- 172.21.15.28:0/2052635898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 msgr2=0x55c51497e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.913+0000 7fa80c525800 1 --2- 172.21.15.28:0/2052635898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c51497e710 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b260 tx=0x7fa7f800b660 comp rx=0 tx=0).stop 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.913+0000 7fa80c525800 1 -- 172.21.15.28:0/2052635898 shutdown_connections 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.913+0000 7fa80c525800 1 --2- 172.21.15.28:0/2052635898 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c51497e710 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.913+0000 7fa80c525800 1 -- 172.21.15.28:0/2052635898 >> 172.21.15.28:0/2052635898 conn(0x55c514978a60 msgr2=0x55c514a69cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.913+0000 7fa80c525800 1 -- 172.21.15.28:0/2052635898 shutdown_connections 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.913+0000 7fa80c525800 1 -- 172.21.15.28:0/2052635898 wait complete. 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa80c525800 1 Processor -- start 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa80c525800 1 -- start start 2024-09-19T11:24:20.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa80c525800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c514a78080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa80c525800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c514985aa0 con 0x55c51497ef80 2024-09-19T11:24:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa808133640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c514a78080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa808133640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c514a78080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56336/0 (socket says 172.21.15.28:56336) 2024-09-19T11:24:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa808133640 1 -- 172.21.15.28:0/1697726268 learned_addr learned my addr 172.21.15.28:0/1697726268 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.914+0000 7fa808133640 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c514a785e0 con 0x55c51497ef80 2024-09-19T11:24:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.915+0000 7fa808133640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c514a78080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55c51497e240 tx=0x7fa7f8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.915+0000 7fa6f4e09640 1 -- 172.21.15.28:0/1697726268 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7f800ee10 con 0x55c51497ef80 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.915+0000 7fa6f4e09640 1 -- 172.21.15.28:0/1697726268 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f800fa30 con 0x55c51497ef80 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.915+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c514a78870 con 0x55c51497ef80 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.915+0000 7fa6f4e09640 1 -- 172.21.15.28:0/1697726268 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7f800fd30 con 0x55c51497ef80 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.915+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c514a79270 con 0x55c51497ef80 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa6f4e09640 1 -- 172.21.15.28:0/1697726268 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa7f8017070 con 0x55c51497ef80 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa6f4e09640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6f003d000 0x7fa6f003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa6f4e09640 1 -- 172.21.15.28:0/1697726268 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa7f8015020 con 0x55c51497ef80 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa807932640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6f003d000 0x7fa6f003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514982a90 0x55c514a7f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514a7fc10 con 0x55c514982a90 2024-09-19T11:24:20.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa808934640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514982a90 0x55c514a7f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.916+0000 7fa807932640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6f003d000 0x7fa6f003f4c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc057000 tx=0x7fa7fc052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.917+0000 7fa808934640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514982a90 0x55c514a7f6d0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.917+0000 7fa808934640 1 -- 172.21.15.28:0/1697726268 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514982a90 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.917+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514a7ffb0 con 0x55c514982a90 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.917+0000 7fa808934640 1 -- 172.21.15.28:0/1697726268 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514982a90 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.917+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514a81ab0 0x55c514a83f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.917+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514a846c0 con 0x55c514a81ab0 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.917+0000 7fa808133640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514a81ab0 0x55c514a83f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.918+0000 7fa808133640 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514a81ab0 0x55c514a83f30 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.918+0000 7fa808133640 1 -- 172.21.15.28:0/1697726268 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa7f802e070 con 0x55c514a81ab0 2024-09-19T11:24:20.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.918+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514a80800 con 0x55c514982a90 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.918+0000 7fa808934640 1 -- 172.21.15.28:0/1697726268 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514982a90 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.918+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514a80c50 con 0x55c514a81ab0 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.919+0000 7fa808133640 1 -- 172.21.15.28:0/1697726268 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa7f802e070 con 0x55c514a81ab0 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.919+0000 7fa80c525800 1 Processor -- start 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.919+0000 7fa80c525800 1 -- start start 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.919+0000 7fa80c525800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a93050 0x55c514a93470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.919+0000 7fa80c525800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c5148754a0 con 0x55c514a93050 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa808934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a93050 0x55c514a93470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa808934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a93050 0x55c514a93470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56348/0 (socket says 172.21.15.28:56348) 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa808934640 1 -- 172.21.15.28:0/852618659 learned_addr learned my addr 172.21.15.28:0/852618659 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa808934640 1 -- 172.21.15.28:0/852618659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c514a93ab0 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa808934640 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a93050 0x55c514a93470 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55c514a79080 tx=0x7fa8000167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa6edffb640 1 -- 172.21.15.28:0/852618659 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa800013640 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa6edffb640 1 -- 172.21.15.28:0/852618659 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8000137e0 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa6edffb640 1 -- 172.21.15.28:0/852618659 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa800013ae0 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c514a942e0 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.920+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c514a94880 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.921+0000 7fa6edffb640 1 -- 172.21.15.28:0/852618659 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa800011040 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.921+0000 7fa6edffb640 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6e003d000 0x7fa6e003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa6edffb640 1 -- 172.21.15.28:0/852618659 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa80000e040 con 0x55c514a93050 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa808133640 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6e003d000 0x7fa6e003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa80c525800 1 Processor -- start 2024-09-19T11:24:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa80c525800 1 -- start start 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa808133640 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6e003d000 0x7fa6e003f4c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8001530 tx=0x7fa7f801c790 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa80c525800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514a9fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa80c525800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c51497d160 con 0x55c514a9fad0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa808934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514a9fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa808934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514a9fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56360/0 (socket says 172.21.15.28:56360) 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.922+0000 7fa808934640 1 -- 172.21.15.28:0/2282253339 learned_addr learned my addr 172.21.15.28:0/2282253339 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa808934640 1 -- 172.21.15.28:0/2282253339 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c514aa04c0 con 0x55c514a9fad0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa808934640 1 --2- 172.21.15.28:0/2282253339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514a9fed0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa8000547f0 tx=0x7fa8000206e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95aad01e2be22962 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa6df7fe640 1 -- 172.21.15.28:0/2282253339 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa800051070 con 0x55c514a9fad0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa6df7fe640 1 -- 172.21.15.28:0/2282253339 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa800053440 con 0x55c514a9fad0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa6df7fe640 1 -- 172.21.15.28:0/2282253339 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa800021a90 con 0x55c514a9fad0 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa80c525800 1 -- 172.21.15.28:0/2282253339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 msgr2=0x55c514a9fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa80c525800 1 --2- 172.21.15.28:0/2282253339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514a9fed0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa8000547f0 tx=0x7fa8000206e0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa80c525800 1 -- 172.21.15.28:0/2282253339 shutdown_connections 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa80c525800 1 --2- 172.21.15.28:0/2282253339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514a9fed0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa80c525800 1 -- 172.21.15.28:0/2282253339 >> 172.21.15.28:0/2282253339 conn(0x55c514a9bed0 msgr2=0x55c514a9a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa80c525800 1 -- 172.21.15.28:0/2282253339 shutdown_connections 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.923+0000 7fa80c525800 1 -- 172.21.15.28:0/2282253339 wait complete. 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.924+0000 7fa80c525800 1 Processor -- start 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.924+0000 7fa80c525800 1 -- start start 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.924+0000 7fa80c525800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514aa70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.924+0000 7fa80c525800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c514aa0ae0 con 0x55c514a9fad0 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.924+0000 7fa808934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514aa70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.924+0000 7fa808934640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514aa70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56374/0 (socket says 172.21.15.28:56374) 2024-09-19T11:24:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.924+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 learned_addr learned my addr 172.21.15.28:0/2221732366 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.925+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c514aa7630 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.925+0000 7fa808934640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514aa70d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa800054750 tx=0x7fa800054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.925+0000 7fa6ddffb640 1 -- 172.21.15.28:0/2221732366 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa800021440 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.925+0000 7fa6ddffb640 1 -- 172.21.15.28:0/2221732366 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80004c070 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.925+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c514aa78c0 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.925+0000 7fa6ddffb640 1 -- 172.21.15.28:0/2221732366 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8000459c0 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.925+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c514aa8240 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.926+0000 7fa6ddffb640 1 -- 172.21.15.28:0/2221732366 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa800045b60 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.926+0000 7fa6ddffb640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6d003d000 0x7fa6d003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.926+0000 7fa6ddffb640 1 -- 172.21.15.28:0/2221732366 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa8000049e0 con 0x55c514a9fad0 2024-09-19T11:24:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.926+0000 7fa808133640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6d003d000 0x7fa6d003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.926+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.926+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514ab2960 0x55c514ab4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.926+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514ab5570 con 0x55c514ab2960 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa807932640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514ab2960 0x55c514ab4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa808133640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6d003d000 0x7fa6d003f4c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800bfb0 tx=0x7fa7f8051000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa807932640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514ab2960 0x55c514ab4e20 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 20 realm 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514ab59a0 con 0x55c514ab2960 2024-09-19T11:24:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.927+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514ab6270 con 0x55c514ab2960 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa80c525800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514ab6610 con 0x55c514ab2960 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa80c525800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514ab7370 0x55c514ab9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514ab9ca0 con 0x55c514ab7370 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa808934640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514ab7370 0x55c514ab9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.928+0000 7fa808934640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514ab7370 0x55c514ab9760 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa80c525800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa80c525800 20 searching for the correct realm 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa808934640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa80004d980 0x7fa800032a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa808133640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa80004d980 0x7fa800032a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.929+0000 7fa808133640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa80004d980 0x7fa800032a20 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.930+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.930+0000 7fa808133640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa7f800f070 0x7fa7f804d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.930+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa7f800f070 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.930+0000 7fa807932640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa7f800f070 0x7fa7f804d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.930+0000 7fa807932640 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa7f800f070 0x7fa7f804d7f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.930+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.930+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.931+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.931+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.931+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.931+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa7f800f070 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.931+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.931+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa7f800f070 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.932+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa7f800f070 2024-09-19T11:24:20.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.933+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.934+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa7f800f070 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.935+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa7f800f070 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.936+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa7f800f070 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab7370 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.937+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x55c514ab2960 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55c514abb640 con 0x7fa80004d980 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514aba710 con 0x55c514ab2960 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.938+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514abafb0 con 0x55c514ab2960 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514abbb10 con 0x55c514ab7370 2024-09-19T11:24:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 zone default found 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 4 Realm: () 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 zonegroup default 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514abbeb0 con 0x7fa7f800f070 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514abddd0 con 0x55c514ab7370 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.939+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa80c525800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514abe230 con 0x55c514ab7370 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa80c525800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa80c525800 20 started sync module instance, tier type = 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa80c525800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.940+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abd0e0 con 0x7fa80004d980 2024-09-19T11:24:20.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.941+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.941+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94304943721520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abe670 con 0x7fa80004d980 2024-09-19T11:24:20.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.941+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abeec0 con 0x55c514ab2960 2024-09-19T11:24:20.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.942+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94304943721520] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.942+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.942+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94304943729168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abf2e0 con 0x55c514ab2960 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.942+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abfb90 con 0x55c514ab7370 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.943+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94304943729168] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.944+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.944+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94304943730224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abf700 con 0x55c514ab7370 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.944+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac0660 con 0x7fa80004d980 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.945+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.945+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94304943730224] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.945+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94304943731280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abf670 con 0x7fa80004d980 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.945+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac1550 con 0x55c514ab2960 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.946+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94304943731280] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.946+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.946+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94304943739040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac1970 con 0x55c514ab2960 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.946+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac2330 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.947+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.947+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94304943742592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac2790 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.947+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac3270 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.947+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94304943739040] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.948+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94304943742592] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.948+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.949+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94304943740096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac2d80 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.949+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac3740 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94304943740096] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94304943743648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac3740 con 0x7fa7f800f070 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 20 add_watcher() i=0 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 20 add_watcher() i=1 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 20 add_watcher() i=2 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 20 add_watcher() i=3 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 20 add_watcher() i=4 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 20 add_watcher() i=5 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.950+0000 7fa80c525800 20 add_watcher() i=6 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.951+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94304943743648] v33'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.951+0000 7fa80c525800 20 add_watcher() i=7 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.951+0000 7fa80c525800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.951+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac6b40 con 0x55c514ab2960 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.951+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.951+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94304943748192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac4090 con 0x55c514ab2960 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.952+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94304943748192] v33'200 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.952+0000 7fa6be7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.952+0000 7fa80c525800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.952+0000 7fa80c525800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.952+0000 7fa80c525800 5 note: GC not initialized 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa6bd7e2640 20 reqs_thread_entry: start 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa80c525800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa80c525800 20 Filter name: none 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa62effd640 20 reqs_thread_entry: start 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa80c525800 20 rados->read ofs=0 len=0 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514af9510 con 0x55c514ab2960 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa80c525800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:24:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.953+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55c514afa930 con 0x7fa7f800f070 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.954+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.955+0000 7fa6be7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.955+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94304943748192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac5790 con 0x55c514ab2960 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.956+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94304943748192] v33'201 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.956+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94304943721520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514ac3430 con 0x7fa80004d980 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.957+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94304943721520] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.957+0000 7fa80c525800 20 remove_watcher() i=0 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.957+0000 7fa80c525800 2 removed watcher, disabling cache 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.957+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94304943729168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abcc30 con 0x55c514ab2960 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.958+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94304943729168] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.958+0000 7fa80c525800 20 remove_watcher() i=1 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.958+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94304943730224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abcc30 con 0x55c514ab7370 2024-09-19T11:24:20.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.959+0000 7fa808934640 1 -- 172.21.15.28:0/2221732366 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94304943730224] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa80000a4d0 con 0x55c514ab7370 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.959+0000 7fa80c525800 20 remove_watcher() i=2 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.959+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94304943731280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abcc30 con 0x7fa80004d980 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.960+0000 7fa808133640 1 -- 172.21.15.28:0/2221732366 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94304943731280] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7f802e070 con 0x7fa80004d980 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.960+0000 7fa80c525800 20 remove_watcher() i=3 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.960+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94304943739040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abcc30 con 0x55c514ab2960 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.961+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94304943739040] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x55c514ab2960 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.961+0000 7fa80c525800 20 remove_watcher() i=4 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.961+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94304943742592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abcc30 con 0x7fa7f800f070 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.962+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94304943742592] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.962+0000 7fa80c525800 20 remove_watcher() i=5 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.962+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94304943740096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abcc30 con 0x7fa7f800f070 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.963+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94304943740096] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.963+0000 7fa80c525800 20 remove_watcher() i=6 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.963+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94304943743648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55c514abcc30 con 0x7fa7f800f070 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94304943743648] v33'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa7fc057030 con 0x7fa7f800f070 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 20 remove_watcher() i=7 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa7f800f070 msgr2=0x7fa7f804d7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa7f800f070 0x7fa7f804d7f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514ab2960 msgr2=0x55c514ab4e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514ab2960 0x55c514ab4e20 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa80004d980 msgr2=0x7fa800032a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa80004d980 0x7fa800032a20 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514ab7370 msgr2=0x55c514ab9760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514ab7370 0x55c514ab9760 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6d003d000 msgr2=0x7fa6d003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6d003d000 0x7fa6d003f4c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800bfb0 tx=0x7fa7f8051000 comp rx=0 tx=0).stop 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.964+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 msgr2=0x55c514aa70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 --2- 172.21.15.28:0/2221732366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a9fad0 0x55c514aa70d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa800054750 tx=0x7fa800054780 comp rx=0 tx=0).stop 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 reap_dead start 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa807932640 1 -- 172.21.15.28:0/2221732366 reap_dead start 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 shutdown_connections 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 >> 172.21.15.28:0/2221732366 conn(0x55c514a9bed0 msgr2=0x55c514a9a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 shutdown_connections 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/2221732366 wait complete. 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6e003d000 msgr2=0x7fa6e003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6e003d000 0x7fa6e003f4c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8001530 tx=0x7fa7f801c790 comp rx=0 tx=0).stop 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a93050 msgr2=0x55c514a93470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a93050 0x55c514a93470 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55c514a79080 tx=0x7fa8000167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 shutdown_connections 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6e003d000 0x7fa6e003f4c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 --2- 172.21.15.28:0/852618659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c514a93050 0x55c514a93470 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 >> 172.21.15.28:0/852618659 conn(0x55c514a87400 msgr2=0x55c514a877e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 shutdown_connections 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/852618659 wait complete. 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514982a90 msgr2=0x55c514a7f6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514982a90 0x55c514a7f6d0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514a81ab0 msgr2=0x55c514a83f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.965+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514a81ab0 0x55c514a83f30 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6f003d000 msgr2=0x7fa6f003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6f003d000 0x7fa6f003f4c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc057000 tx=0x7fa7fc052000 comp rx=0 tx=0).stop 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 msgr2=0x55c514a78080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c514a78080 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55c51497e240 tx=0x7fa7f8000ed0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 shutdown_connections 2024-09-19T11:24:20.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55c514a81ab0 0x55c514a83f30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa6f003d000 0x7fa6f003f4c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55c514982a90 0x55c514a7f6d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 --2- 172.21.15.28:0/1697726268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c51497ef80 0x55c514a78080 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:20.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 >> 172.21.15.28:0/1697726268 conn(0x55c514978a60 msgr2=0x55c514a69800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:20.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 shutdown_connections 2024-09-19T11:24:20.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:20.966+0000 7fa80c525800 1 -- 172.21.15.28:0/1697726268 wait complete. 2024-09-19T11:24:20.980 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo2","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo3","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo4","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34,"bytes_processed":0,"bytes_returned":0}}]} 2024-09-19T11:24:20.980 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo2', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo3', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo4', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:24:20.981 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-09-19T11:24:20.981 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-09-19T11:24:20.981 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-09-19T11:24:21.028 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:21.029 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f367183e800 1 Processor -- start 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f367183e800 1 -- start start 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f367183e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x559782ecb880 0x559782ed43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f367183e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559782ed48e0 con 0x559782ec9d60 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f367183e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x559782d567d0 con 0x559782ecb880 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f3666b27640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x559782ecb880 0x559782ed43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f3666b27640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x559782ecb880 0x559782ed43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56384/0 (socket says 172.21.15.28:56384) 2024-09-19T11:24:21.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.028+0000 7f3666b27640 1 -- 172.21.15.28:0/3494265261 learned_addr learned my addr 172.21.15.28:0/3494265261 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.029+0000 7f3666b27640 1 -- 172.21.15.28:0/3494265261 >> v1:172.21.15.22:6789/0 conn(0x559782ec9d60 legacy=0x559782eca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.029+0000 7f3666b27640 1 -- 172.21.15.28:0/3494265261 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782ed5ac0 con 0x559782ecb880 2024-09-19T11:24:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f3666b27640 1 --2- 172.21.15.28:0/3494265261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ecb880 0x559782ed43a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f365800a520 tx=0x7f3658045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e623a806c5601a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f366eb27640 1 -- 172.21.15.28:0/3494265261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3658055070 con 0x559782ecb880 2024-09-19T11:24:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f366eb27640 1 -- 172.21.15.28:0/3494265261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36580515b0 con 0x559782ecb880 2024-09-19T11:24:21.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f366eb27640 1 -- 172.21.15.28:0/3494265261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36580518d0 con 0x559782ecb880 2024-09-19T11:24:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f367183e800 1 -- 172.21.15.28:0/3494265261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ecb880 msgr2=0x559782ed43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f367183e800 1 --2- 172.21.15.28:0/3494265261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ecb880 0x559782ed43a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f365800a520 tx=0x7f3658045ba0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f367183e800 1 -- 172.21.15.28:0/3494265261 shutdown_connections 2024-09-19T11:24:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f367183e800 1 --2- 172.21.15.28:0/3494265261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ecb880 0x559782ed43a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f367183e800 1 -- 172.21.15.28:0/3494265261 >> 172.21.15.28:0/3494265261 conn(0x559782ec5a70 msgr2=0x559782ec7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f367183e800 1 -- 172.21.15.28:0/3494265261 shutdown_connections 2024-09-19T11:24:21.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.031+0000 7f367183e800 1 -- 172.21.15.28:0/3494265261 wait complete. 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.046+0000 7f367183e800 1 Processor -- start 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.046+0000 7f367183e800 1 -- start start 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f367183e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fbd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f367183e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559782d63dd0 con 0x559782ec9d60 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f366f328640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fbd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f366f328640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fbd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56396/0 (socket says 172.21.15.28:56396) 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f366f328640 1 -- 172.21.15.28:0/3901470233 learned_addr learned my addr 172.21.15.28:0/3901470233 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f366f328640 1 -- 172.21.15.28:0/3901470233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782fbdba0 con 0x559782ec9d60 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f366f328640 1 --2- 172.21.15.28:0/3901470233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fbd5b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3660001d40 tx=0x7f366000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffae0178f3113060 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f355d60a640 1 -- 172.21.15.28:0/3901470233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f366000f400 con 0x559782ec9d60 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f355d60a640 1 -- 172.21.15.28:0/3901470233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366000f5a0 con 0x559782ec9d60 2024-09-19T11:24:21.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.047+0000 7f355d60a640 1 -- 172.21.15.28:0/3901470233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f366000f900 con 0x559782ec9d60 2024-09-19T11:24:21.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 -- 172.21.15.28:0/3901470233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 msgr2=0x559782fbd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 --2- 172.21.15.28:0/3901470233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fbd5b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3660001d40 tx=0x7f366000b5b0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 -- 172.21.15.28:0/3901470233 shutdown_connections 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 --2- 172.21.15.28:0/3901470233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fbd5b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 -- 172.21.15.28:0/3901470233 >> 172.21.15.28:0/3901470233 conn(0x559782ec5a70 msgr2=0x559782fb8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 -- 172.21.15.28:0/3901470233 shutdown_connections 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 -- 172.21.15.28:0/3901470233 wait complete. 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.048+0000 7f367183e800 1 Processor -- start 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f367183e800 1 -- start start 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f367183e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fc50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f367183e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559782ed4db0 con 0x559782ec9d60 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f366f328640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fc50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f366f328640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fc50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56404/0 (socket says 172.21.15.28:56404) 2024-09-19T11:24:21.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f366f328640 1 -- 172.21.15.28:0/607567058 learned_addr learned my addr 172.21.15.28:0/607567058 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f366f328640 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782fc5620 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f366f328640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fc50c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x559782fbe540 tx=0x7f3660000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f355be07640 1 -- 172.21.15.28:0/607567058 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f366001a7c0 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f355be07640 1 -- 172.21.15.28:0/607567058 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366001a980 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f355be07640 1 -- 172.21.15.28:0/607567058 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f366001aca0 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.049+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559782fc58b0 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.050+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559782fc64c0 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.050+0000 7f355be07640 1 -- 172.21.15.28:0/607567058 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3660017070 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f355be07640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f355403d000 0x7f355403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f355be07640 1 -- 172.21.15.28:0/607567058 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3660015020 con 0x559782ec9d60 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f3666b27640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f355403d000 0x7f355403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782ecb880 0x559782fcc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559782fccc50 con 0x559782ecb880 2024-09-19T11:24:21.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f366fb29640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782ecb880 0x559782fcc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f3666b27640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f355403d000 0x7f355403f4c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3658057000 tx=0x7f3658052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.051+0000 7f366fb29640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782ecb880 0x559782fcc710 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.052+0000 7f366fb29640 1 -- 172.21.15.28:0/607567058 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f366800a4d0 con 0x559782ecb880 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.052+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559782fccff0 con 0x559782ecb880 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.052+0000 7f366fb29640 1 -- 172.21.15.28:0/607567058 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f366800a4d0 con 0x559782ecb880 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.052+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559782fceaf0 0x559782fd0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.052+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559782fd1720 con 0x559782fceaf0 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.052+0000 7f366f328640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559782fceaf0 0x559782fd0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.052+0000 7f366f328640 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559782fceaf0 0x559782fd0f90 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.053+0000 7f366f328640 1 -- 172.21.15.28:0/607567058 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f366002e070 con 0x559782fceaf0 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.053+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559782fcd840 con 0x559782ecb880 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.053+0000 7f366fb29640 1 -- 172.21.15.28:0/607567058 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f366800a4d0 con 0x559782ecb880 2024-09-19T11:24:21.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.053+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559782fcdc60 con 0x559782fceaf0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.053+0000 7f366f328640 1 -- 172.21.15.28:0/607567058 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f366002e070 con 0x559782fceaf0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f367183e800 1 Processor -- start 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f367183e800 1 -- start start 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f367183e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fe00d0 0x559782fe04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f367183e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559782dc24a0 con 0x559782fe00d0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f366fb29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fe00d0 0x559782fe04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f366fb29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fe00d0 0x559782fe04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56416/0 (socket says 172.21.15.28:56416) 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f366fb29640 1 -- 172.21.15.28:0/126970452 learned_addr learned my addr 172.21.15.28:0/126970452 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.054+0000 7f366fb29640 1 -- 172.21.15.28:0/126970452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782fe0b30 con 0x559782fe00d0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.055+0000 7f366fb29640 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fe00d0 0x559782fe04f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x559782fc60c0 tx=0x7f36680167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.055+0000 7f3559602640 1 -- 172.21.15.28:0/126970452 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668013640 con 0x559782fe00d0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.055+0000 7f3559602640 1 -- 172.21.15.28:0/126970452 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36680137e0 con 0x559782fe00d0 2024-09-19T11:24:21.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.055+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559782fe1360 con 0x559782fe00d0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.055+0000 7f3559602640 1 -- 172.21.15.28:0/126970452 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668013ae0 con 0x559782fe00d0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.055+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559782fe1900 con 0x559782fe00d0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f3559602640 1 -- 172.21.15.28:0/126970452 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3668011040 con 0x559782fe00d0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f3559602640 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f354803d000 0x7f354803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f3559602640 1 -- 172.21.15.28:0/126970452 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f366800e040 con 0x559782fe00d0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f366f328640 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f354803d000 0x7f354803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f367183e800 1 Processor -- start 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f366f328640 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f354803d000 0x7f354803f4c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x559782fd1ac0 tx=0x7f3660046f90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f367183e800 1 -- start start 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f367183e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782fecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.056+0000 7f367183e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559782fbdfd0 con 0x559782fecb50 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f366fb29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782fecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f366fb29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782fecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56428/0 (socket says 172.21.15.28:56428) 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f366fb29640 1 -- 172.21.15.28:0/3156456977 learned_addr learned my addr 172.21.15.28:0/3156456977 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f366fb29640 1 -- 172.21.15.28:0/3156456977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782fed540 con 0x559782fecb50 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f366fb29640 1 --2- 172.21.15.28:0/3156456977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782fecf50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3668016cd0 tx=0x7f3668020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f2df809494ad0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f3552ffd640 1 -- 172.21.15.28:0/3156456977 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668051070 con 0x559782fecb50 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f3552ffd640 1 -- 172.21.15.28:0/3156456977 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366802adb0 con 0x559782fecb50 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.057+0000 7f3552ffd640 1 -- 172.21.15.28:0/3156456977 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668053310 con 0x559782fecb50 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 -- 172.21.15.28:0/3156456977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 msgr2=0x559782fecf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 --2- 172.21.15.28:0/3156456977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782fecf50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3668016cd0 tx=0x7f3668020460 comp rx=0 tx=0).stop 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 -- 172.21.15.28:0/3156456977 shutdown_connections 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 --2- 172.21.15.28:0/3156456977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782fecf50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 -- 172.21.15.28:0/3156456977 >> 172.21.15.28:0/3156456977 conn(0x559782fe8f50 msgr2=0x559782fe7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 -- 172.21.15.28:0/3156456977 shutdown_connections 2024-09-19T11:24:21.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 -- 172.21.15.28:0/3156456977 wait complete. 2024-09-19T11:24:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 Processor -- start 2024-09-19T11:24:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.058+0000 7f367183e800 1 -- start start 2024-09-19T11:24:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f367183e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782ff4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f367183e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559782fedb60 con 0x559782fecb50 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f366fb29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782ff4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f366fb29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782ff4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56436/0 (socket says 172.21.15.28:56436) 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 learned_addr learned my addr 172.21.15.28:0/2171450489 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559782ff46e0 con 0x559782fecb50 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f366fb29640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782ff4180 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3668020790 tx=0x7f36680204d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f35517fa640 1 -- 172.21.15.28:0/2171450489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3668021570 con 0x559782fecb50 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f35517fa640 1 -- 172.21.15.28:0/2171450489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366801f340 con 0x559782fecb50 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559782ff46e0 con 0x559782fecb50 2024-09-19T11:24:21.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.059+0000 7f35517fa640 1 -- 172.21.15.28:0/2171450489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36680459d0 con 0x559782fecb50 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.060+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559782ff46e0 con 0x559782fecb50 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.060+0000 7f35517fa640 1 -- 172.21.15.28:0/2171450489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3668045b90 con 0x559782fecb50 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f35517fa640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f353803d050 0x7f353803f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f35517fa640 1 -- 172.21.15.28:0/2171450489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3668004dc0 con 0x559782fecb50 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f366f328640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f353803d050 0x7f353803f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782fff630 0x559783001af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559783002240 con 0x559782fff630 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f3666b27640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782fff630 0x559783001af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f366f328640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f353803d050 0x7f353803f510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f366000ea80 tx=0x7f3660033000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.061+0000 7f3666b27640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782fff630 0x559783001af0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 realm 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559783002670 con 0x559782fff630 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559783002f40 con 0x559782fff630 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5597830032e0 con 0x559782fff630 2024-09-19T11:24:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559783004040 0x559783006430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.062+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559783006970 con 0x559783004040 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f366fb29640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559783004040 0x559783006430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f366fb29640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559783004040 0x559783006430 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f367183e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f367183e800 20 searching for the correct realm 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f366fb29640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3668004600 0x7f36680323a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.063+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.064+0000 7f366f328640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3668004600 0x7f36680323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.064+0000 7f366f328640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3668004600 0x7f36680323a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.064+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.064+0000 7f366f328640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f366003d040 0x7f366004b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.064+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f366003d040 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.064+0000 7f3666b27640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f366003d040 0x7f366004b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.065+0000 7f3666b27640 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f366003d040 0x7f366004b0e0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.065+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.065+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.065+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.065+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f366003d040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f366003d040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.066+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f366003d040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.067+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.068+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.069+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f366003d040 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.070+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f366003d040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f366003d040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.071+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559783004040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x559782fff630 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x559783008310 con 0x7f3668004600 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.072+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5597830073e0 con 0x559782fff630 2024-09-19T11:24:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559783007c80 con 0x559782fff630 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5597830087e0 con 0x559783004040 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 zone default found 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 4 Realm: () 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 zonegroup default 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.073+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559783008b80 con 0x7f366003d040 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55978300adc0 con 0x559783004040 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55978300b160 con 0x559783004040 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 20 started sync module instance, tier type = 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.074+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300a200 con 0x7f3668004600 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.075+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.075+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94109226280432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300aa10 con 0x7f3668004600 2024-09-19T11:24:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.075+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300bec0 con 0x559782fff630 2024-09-19T11:24:21.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.076+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94109226280432] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.077+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.077+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94109226284288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300c2e0 con 0x559782fff630 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.077+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300cd00 con 0x559783004040 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.078+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94109226284288] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.078+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.078+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94109226289136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300ccc0 con 0x559783004040 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.078+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300d7b0 con 0x7f3668004600 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.079+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94109226289136] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.079+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.079+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94109226291904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300dba0 con 0x7f3668004600 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.079+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300e5a0 con 0x559782fff630 2024-09-19T11:24:21.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.080+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94109226291904] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.080+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.080+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94109226285344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300e990 con 0x559782fff630 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.080+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300f3d0 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.081+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.081+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94109226285344] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.081+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94109226299040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300f390 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.082+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300e0d0 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.082+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94109226299040] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.083+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.083+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94109226295504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300fec0 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.083+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x559783010880 con 0x7f366003d040 2024-09-19T11:24:21.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.083+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94109226295504] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94109226301088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x559783010880 con 0x7f366003d040 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 20 add_watcher() i=0 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 20 add_watcher() i=1 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 20 add_watcher() i=2 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 20 add_watcher() i=3 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 20 add_watcher() i=4 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 20 add_watcher() i=5 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.084+0000 7f367183e800 20 add_watcher() i=6 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.085+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94109226301088] v33'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.085+0000 7f367183e800 20 add_watcher() i=7 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.085+0000 7f367183e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:21.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.085+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x559783013b90 con 0x559782fff630 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.085+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.085+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94109226304832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x559783010e10 con 0x559782fff630 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.086+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94109226304832] v33'202 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.086+0000 7f3525fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.086+0000 7f367183e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.086+0000 7f367183e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.086+0000 7f367183e800 5 note: GC not initialized 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f3524fe1640 20 reqs_thread_entry: start 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f367183e800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f367183e800 20 Filter name: none 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f349e7fc640 20 reqs_thread_entry: start 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f367183e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5597830465a0 con 0x559782fff630 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f367183e800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.087+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5597830479c0 con 0x7f366003d040 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.088+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.089+0000 7f3525fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.089+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94109226304832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x559783012660 con 0x559782fff630 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.090+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94109226304832] v33'203 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.090+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94109226280432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x559783010540 con 0x7f3668004600 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.091+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94109226280432] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.091+0000 7f367183e800 20 remove_watcher() i=0 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.091+0000 7f367183e800 2 removed watcher, disabling cache 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.091+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94109226284288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300a5f0 con 0x559782fff630 2024-09-19T11:24:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.092+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94109226284288] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.092+0000 7f367183e800 20 remove_watcher() i=1 2024-09-19T11:24:21.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.092+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94109226289136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300a5f0 con 0x559783004040 2024-09-19T11:24:21.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.093+0000 7f366fb29640 1 -- 172.21.15.28:0/2171450489 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94109226289136] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366800a4d0 con 0x559783004040 2024-09-19T11:24:21.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.093+0000 7f367183e800 20 remove_watcher() i=2 2024-09-19T11:24:21.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.093+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94109226291904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300c7f0 con 0x7f3668004600 2024-09-19T11:24:21.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.094+0000 7f366f328640 1 -- 172.21.15.28:0/2171450489 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94109226291904] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f366002e070 con 0x7f3668004600 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.094+0000 7f367183e800 20 remove_watcher() i=3 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.094+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94109226285344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300a5f0 con 0x559782fff630 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.095+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94109226285344] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x559782fff630 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.095+0000 7f367183e800 20 remove_watcher() i=4 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.095+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94109226299040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300a5f0 con 0x7f366003d040 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.096+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94109226299040] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.096+0000 7f367183e800 20 remove_watcher() i=5 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.096+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94109226295504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300a5f0 con 0x7f366003d040 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.097+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94109226295504] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.097+0000 7f367183e800 20 remove_watcher() i=6 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.097+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94109226301088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55978300a5f0 con 0x7f366003d040 2024-09-19T11:24:21.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94109226301088] v33'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3658057030 con 0x7f366003d040 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 20 remove_watcher() i=7 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f366003d040 msgr2=0x7f366004b0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f366003d040 0x7f366004b0e0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782fff630 msgr2=0x559783001af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782fff630 0x559783001af0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3668004600 msgr2=0x7f36680323a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3668004600 0x7f36680323a0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559783004040 msgr2=0x559783006430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559783004040 0x559783006430 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f353803d050 msgr2=0x7f353803f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f353803d050 0x7f353803f510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f366000ea80 tx=0x7f3660033000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 msgr2=0x559782ff4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 --2- 172.21.15.28:0/2171450489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fecb50 0x559782ff4180 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3668020790 tx=0x7f36680204d0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 reap_dead start 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f3666b27640 1 -- 172.21.15.28:0/2171450489 reap_dead start 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 shutdown_connections 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 >> 172.21.15.28:0/2171450489 conn(0x559782fe8f50 msgr2=0x559782fe7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 shutdown_connections 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.098+0000 7f367183e800 1 -- 172.21.15.28:0/2171450489 wait complete. 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f354803d000 msgr2=0x7f354803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f354803d000 0x7f354803f4c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x559782fd1ac0 tx=0x7f3660046f90 comp rx=0 tx=0).stop 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fe00d0 msgr2=0x559782fe04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fe00d0 0x559782fe04f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x559782fc60c0 tx=0x7f36680167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 shutdown_connections 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f354803d000 0x7f354803f4c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/126970452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782fe00d0 0x559782fe04f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 >> 172.21.15.28:0/126970452 conn(0x559782fd4480 msgr2=0x559782fd4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 shutdown_connections 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/126970452 wait complete. 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782ecb880 msgr2=0x559782fcc710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782ecb880 0x559782fcc710 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559782fceaf0 msgr2=0x559782fd0f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559782fceaf0 0x559782fd0f90 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f355403d000 msgr2=0x7f355403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f355403d000 0x7f355403f4c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3658057000 tx=0x7f3658052000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 msgr2=0x559782fc50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fc50c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x559782fbe540 tx=0x7f3660000bc0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 shutdown_connections 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x559782fceaf0 0x559782fd0f90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f355403d000 0x7f355403f4c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x559782ecb880 0x559782fcc710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 --2- 172.21.15.28:0/607567058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559782ec9d60 0x559782fc50c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.099+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 >> 172.21.15.28:0/607567058 conn(0x559782ec5a70 msgr2=0x559782fbeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.100+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 shutdown_connections 2024-09-19T11:24:21.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.100+0000 7f367183e800 1 -- 172.21.15.28:0/607567058 wait complete. 2024-09-19T11:24:21.110 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5}],"s3select":{}},{"bucket":"myfoo2","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo3","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo4","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8}],"total":{"bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8,"bytes_processed":0,"bytes_returned":0}}]} 2024-09-19T11:24:21.110 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:24:21.110 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-09-19T11:24:21.110 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-09-19T11:24:21.111 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-09-19T11:24:21.158 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:21.158 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:21.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4f53c800 1 Processor -- start 2024-09-19T11:24:21.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4f53c800 1 -- start start 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4f53c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562981af5e10 0x562981afb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4f53c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562981af62c0 con 0x562981af2cf0 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4f53c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x56298197f7d0 con 0x562981af5e10 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4aa80640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562981af5e10 0x562981afb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4b281640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x562981af2cf0 0x562981af5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59316/0 (socket says 172.21.15.28:59316) 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4b281640 1 -- 172.21.15.28:0/3448531100 learned_addr learned my addr 172.21.15.28:0/3448531100 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4aa80640 1 -- 172.21.15.28:0/3448531100 >> v1:172.21.15.22:6789/0 conn(0x562981af2cf0 legacy=0x562981af5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.158+0000 7f9b4aa80640 1 -- 172.21.15.28:0/3448531100 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562981af6490 con 0x562981af5e10 2024-09-19T11:24:21.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.160+0000 7f9b4aa80640 1 --2- 172.21.15.28:0/3448531100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af5e10 0x562981afb760 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400a470 tx=0x7f9b34045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0bc33b0c50099c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.160+0000 7f9b4a27f640 1 -- 172.21.15.28:0/3448531100 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b34053570 con 0x562981af5e10 2024-09-19T11:24:21.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.160+0000 7f9b4a27f640 1 -- 172.21.15.28:0/3448531100 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b34058070 con 0x562981af5e10 2024-09-19T11:24:21.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.160+0000 7f9b4a27f640 1 -- 172.21.15.28:0/3448531100 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b34053880 con 0x562981af5e10 2024-09-19T11:24:21.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.161+0000 7f9b4f53c800 1 -- 172.21.15.28:0/3448531100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af5e10 msgr2=0x562981afb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.161+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/3448531100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af5e10 0x562981afb760 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400a470 tx=0x7f9b34045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:21.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.161+0000 7f9b4f53c800 1 -- 172.21.15.28:0/3448531100 shutdown_connections 2024-09-19T11:24:21.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.161+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/3448531100 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af5e10 0x562981afb760 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.161+0000 7f9b4f53c800 1 -- 172.21.15.28:0/3448531100 >> 172.21.15.28:0/3448531100 conn(0x562981aee1d0 msgr2=0x562981af0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.161+0000 7f9b4f53c800 1 -- 172.21.15.28:0/3448531100 shutdown_connections 2024-09-19T11:24:21.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.161+0000 7f9b4f53c800 1 -- 172.21.15.28:0/3448531100 wait complete. 2024-09-19T11:24:21.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.176+0000 7f9b4f53c800 1 Processor -- start 2024-09-19T11:24:21.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.176+0000 7f9b4f53c800 1 -- start start 2024-09-19T11:24:21.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.176+0000 7f9b4f53c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981af4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.176+0000 7f9b4f53c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56298198cdd0 con 0x562981af2cf0 2024-09-19T11:24:21.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.176+0000 7f9b4b281640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981af4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.176+0000 7f9b4b281640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981af4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56448/0 (socket says 172.21.15.28:56448) 2024-09-19T11:24:21.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.176+0000 7f9b4b281640 1 -- 172.21.15.28:0/2870364967 learned_addr learned my addr 172.21.15.28:0/2870364967 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.177+0000 7f9b4b281640 1 -- 172.21.15.28:0/2870364967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562981af5440 con 0x562981af2cf0 2024-09-19T11:24:21.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.177+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2870364967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981af4e50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001e40 tx=0x7f9b3c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b428113b946933af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.177+0000 7f9a3960a640 1 -- 172.21.15.28:0/2870364967 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c012020 con 0x562981af2cf0 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.177+0000 7f9a3960a640 1 -- 172.21.15.28:0/2870364967 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00e5a0 con 0x562981af2cf0 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.177+0000 7f9a3960a640 1 -- 172.21.15.28:0/2870364967 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c00e8a0 con 0x562981af2cf0 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2870364967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 msgr2=0x562981af4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2870364967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981af4e50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001e40 tx=0x7f9b3c001e70 comp rx=0 tx=0).stop 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2870364967 shutdown_connections 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2870364967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981af4e50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2870364967 >> 172.21.15.28:0/2870364967 conn(0x562981aee1d0 msgr2=0x562981bdff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2870364967 shutdown_connections 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2870364967 wait complete. 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 Processor -- start 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 -- start start 2024-09-19T11:24:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981bee310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4f53c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562981afbfa0 con 0x562981af2cf0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4b281640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981bee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4b281640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981bee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56454/0 (socket says 172.21.15.28:56454) 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.178+0000 7f9b4b281640 1 -- 172.21.15.28:0/1326470075 learned_addr learned my addr 172.21.15.28:0/1326470075 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.179+0000 7f9b4b281640 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562981bee870 con 0x562981af2cf0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.179+0000 7f9b4b281640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981bee310 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c010 tx=0x7f9b3c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.179+0000 7f9a37e07640 1 -- 172.21.15.28:0/1326470075 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c012030 con 0x562981af2cf0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.179+0000 7f9a37e07640 1 -- 172.21.15.28:0/1326470075 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3c0014a0 con 0x562981af2cf0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.179+0000 7f9a37e07640 1 -- 172.21.15.28:0/1326470075 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b3c01a7a0 con 0x562981af2cf0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.179+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562981beeb00 con 0x562981af2cf0 2024-09-19T11:24:21.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.179+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562981bef500 con 0x562981af2cf0 2024-09-19T11:24:21.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.180+0000 7f9a37e07640 1 -- 172.21.15.28:0/1326470075 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9b3c015020 con 0x562981af2cf0 2024-09-19T11:24:21.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.180+0000 7f9a37e07640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a3003cfb0 0x7f9a3003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.180+0000 7f9a37e07640 1 -- 172.21.15.28:0/1326470075 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f9b3c020030 con 0x562981af2cf0 2024-09-19T11:24:21.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4aa80640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a3003cfb0 0x7f9a3003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981af5e10 0x562981bf5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981bf5ea0 con 0x562981af5e10 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981af5e10 0x562981bf5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4aa80640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a3003cfb0 0x7f9a3003f470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b34057000 tx=0x7f9b34052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981af5e10 0x562981bf5960 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4ba82640 1 -- 172.21.15.28:0/1326470075 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981af5e10 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.181+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981bf6240 con 0x562981af5e10 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.182+0000 7f9b4ba82640 1 -- 172.21.15.28:0/1326470075 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981af5e10 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.182+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981bf7d00 0x562981bfa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.182+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981bfa930 con 0x562981bf7d00 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.182+0000 7f9b4b281640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981bf7d00 0x562981bfa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.182+0000 7f9b4b281640 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981bf7d00 0x562981bfa1a0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.182+0000 7f9b4b281640 1 -- 172.21.15.28:0/1326470075 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9b3c01ab20 con 0x562981bf7d00 2024-09-19T11:24:21.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.182+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981bf6a90 con 0x562981af5e10 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.183+0000 7f9b4ba82640 1 -- 172.21.15.28:0/1326470075 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981af5e10 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.183+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981bf6e30 con 0x562981bf7d00 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.183+0000 7f9b4b281640 1 -- 172.21.15.28:0/1326470075 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9b3c01ab20 con 0x562981bf7d00 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.183+0000 7f9b4f53c800 1 Processor -- start 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.183+0000 7f9b4f53c800 1 -- start start 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.184+0000 7f9b4f53c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c092c0 0x562981c096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.184+0000 7f9b4f53c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629819eb4a0 con 0x562981c092c0 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.184+0000 7f9b4ba82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c092c0 0x562981c096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.184+0000 7f9b4ba82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c092c0 0x562981c096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56460/0 (socket says 172.21.15.28:56460) 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.184+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2438744548 learned_addr learned my addr 172.21.15.28:0/2438744548 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.184+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2438744548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562981c09d20 con 0x562981c092c0 2024-09-19T11:24:21.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.184+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c092c0 0x562981c096e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x562981bef1a0 tx=0x7f9b440167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.185+0000 7f9a35602640 1 -- 172.21.15.28:0/2438744548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44013640 con 0x562981c092c0 2024-09-19T11:24:21.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.185+0000 7f9a35602640 1 -- 172.21.15.28:0/2438744548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b440137e0 con 0x562981c092c0 2024-09-19T11:24:21.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.185+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562981c0a550 con 0x562981c092c0 2024-09-19T11:24:21.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.185+0000 7f9a35602640 1 -- 172.21.15.28:0/2438744548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44013b00 con 0x562981c092c0 2024-09-19T11:24:21.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.185+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562981c0aaf0 con 0x562981c092c0 2024-09-19T11:24:21.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.185+0000 7f9a35602640 1 -- 172.21.15.28:0/2438744548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9b44011040 con 0x562981c092c0 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9a35602640 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a2403d070 0x7f9a2403f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9a35602640 1 -- 172.21.15.28:0/2438744548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f9b4400e040 con 0x562981c092c0 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a2403d070 0x7f9a2403f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9b4f53c800 1 Processor -- start 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9b4f53c800 1 -- start start 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a2403d070 0x7f9a2403f530 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x562981bf0410 tx=0x7f9b3c0529c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9b4f53c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c16140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.186+0000 7f9b4f53c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562981af3860 con 0x562981c15d40 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9b4ba82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c16140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9b4ba82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c16140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56464/0 (socket says 172.21.15.28:56464) 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9b4ba82640 1 -- 172.21.15.28:0/1852373556 learned_addr learned my addr 172.21.15.28:0/1852373556 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9b4ba82640 1 -- 172.21.15.28:0/1852373556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562981c16730 con 0x562981c15d40 2024-09-19T11:24:21.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/1852373556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c16140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400afe0 tx=0x7f9b44020f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b485a1e6907c867d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9a2effd640 1 -- 172.21.15.28:0/1852373556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4402aa40 con 0x562981c15d40 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9a2effd640 1 -- 172.21.15.28:0/1852373556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4404de50 con 0x562981c15d40 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.187+0000 7f9a2effd640 1 -- 172.21.15.28:0/1852373556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44013dd0 con 0x562981c15d40 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1852373556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 msgr2=0x562981c16140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1852373556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c16140 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400afe0 tx=0x7f9b44020f00 comp rx=0 tx=0).stop 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1852373556 shutdown_connections 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1852373556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c16140 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1852373556 >> 172.21.15.28:0/1852373556 conn(0x562981c12140 msgr2=0x562981c10380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1852373556 shutdown_connections 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1852373556 wait complete. 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 Processor -- start 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 -- start start 2024-09-19T11:24:21.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c1d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4f53c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562981c16d50 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4ba82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c1d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4ba82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c1d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56480/0 (socket says 172.21.15.28:56480) 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.188+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 learned_addr learned my addr 172.21.15.28:0/2877893222 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.189+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562981c1d850 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.189+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c1d2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400afe0 tx=0x7f9b440547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.189+0000 7f9a2d7fa640 1 -- 172.21.15.28:0/2877893222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4402a3f0 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.189+0000 7f9a2d7fa640 1 -- 172.21.15.28:0/2877893222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b4402a590 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.189+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562981c1dae0 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.189+0000 7f9a2d7fa640 1 -- 172.21.15.28:0/2877893222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44045b20 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.189+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562981c1e460 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.190+0000 7f9a2d7fa640 1 -- 172.21.15.28:0/2877893222 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9b44045cc0 con 0x562981c15d40 2024-09-19T11:24:21.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.190+0000 7f9a2d7fa640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a1403cf90 0x7f9a1403f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.190+0000 7f9a2d7fa640 1 -- 172.21.15.28:0/2877893222 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f9b440539a0 con 0x562981c15d40 2024-09-19T11:24:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.190+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a1403cf90 0x7f9a1403f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.191+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a1403cf90 0x7f9a1403f450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001250 tx=0x7f9b3c051000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.191+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.191+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981c28c70 0x562981c2b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.191+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c2b880 con 0x562981c28c70 2024-09-19T11:24:21.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.191+0000 7f9b4aa80640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981c28c70 0x562981c2b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.191+0000 7f9b4aa80640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981c28c70 0x562981c2b130 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.191+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 realm 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c2bcb0 con 0x562981c28c70 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c2c580 con 0x562981c28c70 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c2c920 con 0x562981c28c70 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981c2d680 0x562981c2fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c2ffb0 con 0x562981c2d680 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.192+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981c2d680 0x562981c2fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981c2d680 0x562981c2fa70 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4f53c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4f53c800 20 searching for the correct realm 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4ba82640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9b4404d990 0x7f9b4404d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.193+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9b4404d990 0x7f9b4404d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.194+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9b4404d990 0x7f9b4404d2d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.194+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.194+0000 7f9b4b281640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9b3c017050 0x7f9b3c04b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.194+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b3c017050 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.194+0000 7f9b4aa80640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9b3c017050 0x7f9b3c04b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.194+0000 7f9b4aa80640 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9b3c017050 0x7f9b3c04b890 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.195+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.195+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.195+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.195+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.195+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.195+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b3c017050 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.195+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b3c017050 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.196+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.197+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b3c017050 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.197+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.197+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.197+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.197+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.197+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.197+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.198+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.199+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.199+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.199+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.199+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b3c017050 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.199+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.199+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.200+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.200+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.200+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.200+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.200+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.200+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b3c017050 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.201+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.201+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.201+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.201+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b3c017050 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.201+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.201+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c2d680 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x562981c28c70 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.202+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562981c31920 con 0x7f9b4404d990 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:21.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c309f0 con 0x562981c28c70 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c31290 con 0x562981c28c70 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c30630 con 0x562981c2d680 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 zone default found 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 4 Realm: () 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 zonegroup default 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.203+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c30eb0 con 0x7f9b3c017050 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c33d40 con 0x562981c2d680 2024-09-19T11:24:21.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c341b0 con 0x562981c2d680 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 20 started sync module instance, tier type = 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.204+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c331b0 con 0x7f9b4404d990 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.206+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.206+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94736270701984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c34aa0 con 0x7f9b4404d990 2024-09-19T11:24:21.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.206+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c352f0 con 0x562981c28c70 2024-09-19T11:24:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.207+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94736270701984] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.207+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.207+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94736270706160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c35390 con 0x562981c28c70 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.207+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c35eb0 con 0x562981c2d680 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.208+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94736270706160] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.208+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.208+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94736270711168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c35e70 con 0x562981c2d680 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.208+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c36930 con 0x7f9b4404d990 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.209+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94736270711168] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.209+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.209+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94736270713920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c36930 con 0x7f9b4404d990 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.209+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c373c0 con 0x562981c28c70 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.210+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94736270713920] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.211+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.211+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94736270716560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c37380 con 0x562981c28c70 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.211+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c37e50 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.212+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.212+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94736270719360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c37e50 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.212+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c38df0 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.212+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94736270716560] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.213+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94736270719360] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.213+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.213+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94736270723264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c38240 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.213+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c393d0 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94736270724896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c397f0 con 0x7f9b3c017050 2024-09-19T11:24:21.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 20 add_watcher() i=0 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 20 add_watcher() i=1 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 20 add_watcher() i=2 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 20 add_watcher() i=3 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 20 add_watcher() i=4 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 20 add_watcher() i=5 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94736270723264] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.214+0000 7f9b4f53c800 20 add_watcher() i=6 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.215+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94736270724896] v33'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.215+0000 7f9b4f53c800 20 add_watcher() i=7 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.215+0000 7f9b4f53c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.215+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c397f0 con 0x562981c28c70 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.215+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.215+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94736270725952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c39760 con 0x562981c28c70 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.216+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94736270725952] v33'204 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.216+0000 7f9a01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.216+0000 7f9b4f53c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.216+0000 7f9b4f53c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.216+0000 7f9b4f53c800 5 note: GC not initialized 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.216+0000 7f9a00fe1640 20 reqs_thread_entry: start 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f9b4f53c800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f9b4f53c800 20 Filter name: none 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f997a7fc640 20 reqs_thread_entry: start 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f9b4f53c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c6f150 con 0x562981c28c70 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f9b4f53c800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.217+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562981c70570 con 0x7f9b3c017050 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.218+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.219+0000 7f9a01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.219+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94736270725952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c6ea70 con 0x562981c28c70 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.220+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94736270725952] v33'205 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.220+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94736270701984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c39340 con 0x7f9b4404d990 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.221+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94736270701984] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.221+0000 7f9b4f53c800 20 remove_watcher() i=0 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.221+0000 7f9b4f53c800 2 removed watcher, disabling cache 2024-09-19T11:24:21.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.221+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94736270706160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c39340 con 0x562981c28c70 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.222+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94736270706160] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.222+0000 7f9b4f53c800 20 remove_watcher() i=1 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.222+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94736270711168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c345f0 con 0x562981c2d680 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.223+0000 7f9b4ba82640 1 -- 172.21.15.28:0/2877893222 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94736270711168] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b4400a4d0 con 0x562981c2d680 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.223+0000 7f9b4f53c800 20 remove_watcher() i=2 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.223+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94736270713920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c39340 con 0x7f9b4404d990 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.224+0000 7f9b4b281640 1 -- 172.21.15.28:0/2877893222 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94736270713920] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b3c01ab20 con 0x7f9b4404d990 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.224+0000 7f9b4f53c800 20 remove_watcher() i=3 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.224+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94736270716560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c35980 con 0x562981c28c70 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.225+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94736270716560] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x562981c28c70 2024-09-19T11:24:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.225+0000 7f9b4f53c800 20 remove_watcher() i=4 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.225+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94736270719360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c36440 con 0x7f9b3c017050 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.226+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94736270719360] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.226+0000 7f9b4f53c800 20 remove_watcher() i=5 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.226+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94736270723264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c39340 con 0x7f9b3c017050 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.227+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94736270723264] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.227+0000 7f9b4f53c800 20 remove_watcher() i=6 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.227+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94736270724896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562981c36e90 con 0x7f9b3c017050 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94736270724896] v33'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b34051070 con 0x7f9b3c017050 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 20 remove_watcher() i=7 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9b3c017050 msgr2=0x7f9b3c04b890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f9b3c017050 0x7f9b3c04b890 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981c28c70 msgr2=0x562981c2b130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981c28c70 0x562981c2b130 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9b4404d990 msgr2=0x7f9b4404d2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f9b4404d990 0x7f9b4404d2d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981c2d680 msgr2=0x562981c2fa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.228+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981c2d680 0x562981c2fa70 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a1403cf90 msgr2=0x7f9a1403f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a1403cf90 0x7f9a1403f450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c001250 tx=0x7f9b3c051000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4aa80640 1 -- 172.21.15.28:0/2877893222 reap_dead start 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 msgr2=0x562981c1d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c1d2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400afe0 tx=0x7f9b440547b0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 shutdown_connections 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2877893222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c15d40 0x562981c1d2f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 >> 172.21.15.28:0/2877893222 conn(0x562981c12140 msgr2=0x562981c10350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 shutdown_connections 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2877893222 wait complete. 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a2403d070 msgr2=0x7f9a2403f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a2403d070 0x7f9a2403f530 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x562981bf0410 tx=0x7f9b3c0529c0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c092c0 msgr2=0x562981c096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c092c0 0x562981c096e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x562981bef1a0 tx=0x7f9b440167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 shutdown_connections 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a2403d070 0x7f9a2403f530 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/2438744548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981c092c0 0x562981c096e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 >> 172.21.15.28:0/2438744548 conn(0x562981bfd670 msgr2=0x562981bfda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 shutdown_connections 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.229+0000 7f9b4f53c800 1 -- 172.21.15.28:0/2438744548 wait complete. 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981af5e10 msgr2=0x562981bf5960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981af5e10 0x562981bf5960 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981bf7d00 msgr2=0x562981bfa1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981bf7d00 0x562981bfa1a0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a3003cfb0 msgr2=0x7f9a3003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a3003cfb0 0x7f9a3003f470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9b34057000 tx=0x7f9b34052000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 msgr2=0x562981bee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981bee310 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00c010 tx=0x7f9b3c000ed0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 shutdown_connections 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562981bf7d00 0x562981bfa1a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f9a3003cfb0 0x7f9a3003f470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562981af5e10 0x562981bf5960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 --2- 172.21.15.28:0/1326470075 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562981af2cf0 0x562981bee310 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 >> 172.21.15.28:0/1326470075 conn(0x562981aee1d0 msgr2=0x562981bdfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 shutdown_connections 2024-09-19T11:24:21.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.230+0000 7f9b4f53c800 1 -- 172.21.15.28:0/1326470075 wait complete. 2024-09-19T11:24:21.240 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{}},{"bucket":"myfoo2","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11,"bytes_processed":0,"bytes_returned":0}}]} 2024-09-19T11:24:21.240 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:24:21.241 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-09-19T11:24:21.241 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-09-19T11:24:21.241 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-09-19T11:24:21.289 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:21.290 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:21.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.289+0000 7fbb78d19800 1 Processor -- start 2024-09-19T11:24:21.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.289+0000 7fbb78d19800 1 -- start start 2024-09-19T11:24:21.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.289+0000 7fbb78d19800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564ca566fa90 0x564ca5671f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.289+0000 7fbb78d19800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564ca56725d0 con 0x564ca566bf20 2024-09-19T11:24:21.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.289+0000 7fbb78d19800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x564ca54f67d0 con 0x564ca566fa90 2024-09-19T11:24:21.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.290+0000 7fbb76009640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564ca566fa90 0x564ca5671f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.290+0000 7fbb7680a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x564ca566bf20 0x564ca566e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59318/0 (socket says 172.21.15.28:59318) 2024-09-19T11:24:21.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.290+0000 7fbb76009640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564ca566fa90 0x564ca5671f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56492/0 (socket says 172.21.15.28:56492) 2024-09-19T11:24:21.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.290+0000 7fbb7680a640 1 -- 172.21.15.28:0/2981810846 learned_addr learned my addr 172.21.15.28:0/2981810846 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:21.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.290+0000 7fbb76009640 1 -- 172.21.15.28:0/2981810846 >> v1:172.21.15.22:6789/0 conn(0x564ca566bf20 legacy=0x564ca566e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.290+0000 7fbb76009640 1 -- 172.21.15.28:0/2981810846 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca5673800 con 0x564ca566fa90 2024-09-19T11:24:21.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb76009640 1 --2- 172.21.15.28:0/2981810846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566fa90 0x564ca5671f50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000a470 tx=0x7fbb60045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc849a80597a74e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb75808640 1 -- 172.21.15.28:0/2981810846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb60053570 con 0x564ca566fa90 2024-09-19T11:24:21.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb75808640 1 -- 172.21.15.28:0/2981810846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb60058070 con 0x564ca566fa90 2024-09-19T11:24:21.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb75808640 1 -- 172.21.15.28:0/2981810846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb60053880 con 0x564ca566fa90 2024-09-19T11:24:21.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb78d19800 1 -- 172.21.15.28:0/2981810846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566fa90 msgr2=0x564ca5671f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2981810846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566fa90 0x564ca5671f50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000a470 tx=0x7fbb60045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:21.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb78d19800 1 -- 172.21.15.28:0/2981810846 shutdown_connections 2024-09-19T11:24:21.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2981810846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566fa90 0x564ca5671f50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb78d19800 1 -- 172.21.15.28:0/2981810846 >> 172.21.15.28:0/2981810846 conn(0x564ca5665a60 msgr2=0x564ca5667ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb78d19800 1 -- 172.21.15.28:0/2981810846 shutdown_connections 2024-09-19T11:24:21.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.292+0000 7fbb78d19800 1 -- 172.21.15.28:0/2981810846 wait complete. 2024-09-19T11:24:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.308+0000 7fbb78d19800 1 Processor -- start 2024-09-19T11:24:21.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.308+0000 7fbb78d19800 1 -- start start 2024-09-19T11:24:21.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.308+0000 7fbb78d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca566b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.308+0000 7fbb78d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564ca5503dd0 con 0x564ca566bf20 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fbb7680a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca566b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fbb7680a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca566b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56494/0 (socket says 172.21.15.28:56494) 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fbb7680a640 1 -- 172.21.15.28:0/3661064617 learned_addr learned my addr 172.21.15.28:0/3661064617 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fbb7680a640 1 -- 172.21.15.28:0/3661064617 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca566bc80 con 0x564ca566bf20 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fbb7680a640 1 --2- 172.21.15.28:0/3661064617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca566b690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb68001800 tx=0x7fbb68001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcbfffa14f8a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fba64e09640 1 -- 172.21.15.28:0/3661064617 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb68004520 con 0x564ca566bf20 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fba64e09640 1 -- 172.21.15.28:0/3661064617 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb680046c0 con 0x564ca566bf20 2024-09-19T11:24:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.309+0000 7fba64e09640 1 -- 172.21.15.28:0/3661064617 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb680049e0 con 0x564ca566bf20 2024-09-19T11:24:21.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 -- 172.21.15.28:0/3661064617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 msgr2=0x564ca566b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 --2- 172.21.15.28:0/3661064617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca566b690 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fbb68001800 tx=0x7fbb68001cc0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 -- 172.21.15.28:0/3661064617 shutdown_connections 2024-09-19T11:24:21.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 --2- 172.21.15.28:0/3661064617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca566b690 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 -- 172.21.15.28:0/3661064617 >> 172.21.15.28:0/3661064617 conn(0x564ca5665a60 msgr2=0x564ca5756d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 -- 172.21.15.28:0/3661064617 shutdown_connections 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 -- 172.21.15.28:0/3661064617 wait complete. 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.310+0000 7fbb78d19800 1 Processor -- start 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.311+0000 7fbb78d19800 1 -- start start 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.311+0000 7fbb78d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca5765040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.311+0000 7fbb78d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564ca5672aa0 con 0x564ca566bf20 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.311+0000 7fbb7680a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca5765040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.311+0000 7fbb7680a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca5765040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56496/0 (socket says 172.21.15.28:56496) 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.311+0000 7fbb7680a640 1 -- 172.21.15.28:0/2611090296 learned_addr learned my addr 172.21.15.28:0/2611090296 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.311+0000 7fbb7680a640 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca57655a0 con 0x564ca566bf20 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.312+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca5765040 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbb680066b0 tx=0x7fbb68001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.312+0000 7fba63606640 1 -- 172.21.15.28:0/2611090296 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb68022020 con 0x564ca566bf20 2024-09-19T11:24:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.312+0000 7fba63606640 1 -- 172.21.15.28:0/2611090296 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb680148c0 con 0x564ca566bf20 2024-09-19T11:24:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.312+0000 7fba63606640 1 -- 172.21.15.28:0/2611090296 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb68014be0 con 0x564ca566bf20 2024-09-19T11:24:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.312+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ca5765890 con 0x564ca566bf20 2024-09-19T11:24:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.312+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca5766440 con 0x564ca566bf20 2024-09-19T11:24:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.313+0000 7fba63606640 1 -- 172.21.15.28:0/2611090296 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbb68025020 con 0x564ca566bf20 2024-09-19T11:24:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.313+0000 7fba63606640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5c03cfb0 0x7fba5c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.313+0000 7fba63606640 1 -- 172.21.15.28:0/2611090296 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbb68018030 con 0x564ca566bf20 2024-09-19T11:24:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.313+0000 7fbb76009640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5c03cfb0 0x7fba5c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.313+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca566fa90 0x564ca576c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.313+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca576cbd0 con 0x564ca566fa90 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.313+0000 7fbb7700b640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca566fa90 0x564ca576c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb76009640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5c03cfb0 0x7fba5c03f470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb60057000 tx=0x7fbb60052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb7700b640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca566fa90 0x564ca576c690 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb7700b640 1 -- 172.21.15.28:0/2611090296 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca566fa90 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca576cf70 con 0x564ca566fa90 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb7700b640 1 -- 172.21.15.28:0/2611090296 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca566fa90 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca576eab0 0x564ca5770f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57716e0 con 0x564ca576eab0 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.314+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca576eab0 0x564ca5770f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.315+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca576eab0 0x564ca5770f50 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.315+0000 7fbb7680a640 1 -- 172.21.15.28:0/2611090296 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbb6801b070 con 0x564ca576eab0 2024-09-19T11:24:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.315+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca576d7c0 con 0x564ca566fa90 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.315+0000 7fbb7700b640 1 -- 172.21.15.28:0/2611090296 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca566fa90 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.315+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca576dbe0 con 0x564ca576eab0 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.315+0000 7fbb7680a640 1 -- 172.21.15.28:0/2611090296 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbb6801b070 con 0x564ca576eab0 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb78d19800 1 Processor -- start 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb78d19800 1 -- start start 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb78d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca5780080 0x564ca57804a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb78d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564ca55624a0 con 0x564ca5780080 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb7700b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca5780080 0x564ca57804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb7700b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca5780080 0x564ca57804a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56506/0 (socket says 172.21.15.28:56506) 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb7700b640 1 -- 172.21.15.28:0/1941910500 learned_addr learned my addr 172.21.15.28:0/1941910500 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.316+0000 7fbb7700b640 1 -- 172.21.15.28:0/1941910500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca5780ae0 con 0x564ca5780080 2024-09-19T11:24:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.317+0000 7fbb7700b640 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca5780080 0x564ca57804a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x564ca5766040 tx=0x7fbb7000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.317+0000 7fba60e01640 1 -- 172.21.15.28:0/1941910500 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb7001c020 con 0x564ca5780080 2024-09-19T11:24:21.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.317+0000 7fba60e01640 1 -- 172.21.15.28:0/1941910500 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb70007c90 con 0x564ca5780080 2024-09-19T11:24:21.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.317+0000 7fba60e01640 1 -- 172.21.15.28:0/1941910500 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb70018440 con 0x564ca5780080 2024-09-19T11:24:21.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.317+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ca5781310 con 0x564ca5780080 2024-09-19T11:24:21.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.317+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca57818b0 con 0x564ca5780080 2024-09-19T11:24:21.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fba60e01640 1 -- 172.21.15.28:0/1941910500 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbb7001f020 con 0x564ca5780080 2024-09-19T11:24:21.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fba60e01640 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5003cfb0 0x7fba5003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fba60e01640 1 -- 172.21.15.28:0/1941910500 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbb70055a80 con 0x564ca5780080 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fbb7680a640 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5003cfb0 0x7fba5003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fbb78d19800 1 Processor -- start 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fbb7680a640 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5003cfb0 0x7fba5003f470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x564ca576db60 tx=0x7fbb68014e90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fbb78d19800 1 -- start start 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fbb78d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca578cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.318+0000 7fbb78d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564ca566a0d0 con 0x564ca578cae0 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fbb7700b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca578cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fbb7700b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca578cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56520/0 (socket says 172.21.15.28:56520) 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fbb7700b640 1 -- 172.21.15.28:0/1693038583 learned_addr learned my addr 172.21.15.28:0/1693038583 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fbb7700b640 1 -- 172.21.15.28:0/1693038583 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca578d4d0 con 0x564ca578cae0 2024-09-19T11:24:21.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fbb7700b640 1 --2- 172.21.15.28:0/1693038583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca578cee0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbb70053df0 tx=0x7fbb70015820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29953a8662166c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fba5a7fc640 1 -- 172.21.15.28:0/1693038583 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb700174d0 con 0x564ca578cae0 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fba5a7fc640 1 -- 172.21.15.28:0/1693038583 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb70052070 con 0x564ca578cae0 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.319+0000 7fba5a7fc640 1 -- 172.21.15.28:0/1693038583 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb70016d80 con 0x564ca578cae0 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 -- 172.21.15.28:0/1693038583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 msgr2=0x564ca578cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 --2- 172.21.15.28:0/1693038583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca578cee0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fbb70053df0 tx=0x7fbb70015820 comp rx=0 tx=0).stop 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 -- 172.21.15.28:0/1693038583 shutdown_connections 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 --2- 172.21.15.28:0/1693038583 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca578cee0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 -- 172.21.15.28:0/1693038583 >> 172.21.15.28:0/1693038583 conn(0x564ca5788ee0 msgr2=0x564ca5787120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 -- 172.21.15.28:0/1693038583 shutdown_connections 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 -- 172.21.15.28:0/1693038583 wait complete. 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 Processor -- start 2024-09-19T11:24:21.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 -- start start 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca57940c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb78d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564ca578daf0 con 0x564ca578cae0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb7700b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca57940c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb7700b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca57940c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56534/0 (socket says 172.21.15.28:56534) 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.320+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 learned_addr learned my addr 172.21.15.28:0/2359701960 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.321+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca5794620 con 0x564ca578cae0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.321+0000 7fbb7700b640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca57940c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbb700206f0 tx=0x7fbb700088f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.321+0000 7fba58ff9640 1 -- 172.21.15.28:0/2359701960 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb7002f070 con 0x564ca578cae0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.321+0000 7fba58ff9640 1 -- 172.21.15.28:0/2359701960 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb700189d0 con 0x564ca578cae0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.321+0000 7fba58ff9640 1 -- 172.21.15.28:0/2359701960 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb700208f0 con 0x564ca578cae0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.321+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ca57948b0 con 0x564ca578cae0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.321+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca57948b0 con 0x564ca578cae0 2024-09-19T11:24:21.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fba58ff9640 1 -- 172.21.15.28:0/2359701960 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbb70016040 con 0x564ca578cae0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fba58ff9640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4003d000 0x7fba4003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fba58ff9640 1 -- 172.21.15.28:0/2359701960 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbb7003b880 con 0x564ca578cae0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4003d000 0x7fba4003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca579f830 0x564ca57a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a23f0 con 0x564ca579f830 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.322+0000 7fbb76009640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca579f830 0x564ca57a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4003d000 0x7fba4003f4c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbb68052ba0 tx=0x7fbb68046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb76009640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca579f830 0x564ca57a1cf0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 20 realm 2024-09-19T11:24:21.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a2820 con 0x564ca579f830 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.323+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a30f0 con 0x564ca579f830 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb78d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a3490 con 0x564ca579f830 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb78d19800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca57a41f0 0x564ca57a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a6b20 con 0x564ca57a41f0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb7700b640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca57a41f0 0x564ca57a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.324+0000 7fbb7700b640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca57a41f0 0x564ca57a65e0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb78d19800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb78d19800 20 searching for the correct realm 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb7700b640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb70021020 0x7fbb7000a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb70021020 0x7fbb7000a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.325+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb70021020 0x7fbb7000a6e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.326+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.326+0000 7fbb7680a640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb68052c20 0x7fbb68054050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.326+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb68052c20 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.326+0000 7fbb76009640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb68052c20 0x7fbb68054050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.326+0000 7fbb76009640 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb68052c20 0x7fbb68054050 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.326+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.326+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb68052c20 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb68052c20 2024-09-19T11:24:21.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.327+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb68052c20 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.328+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.329+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.330+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb68052c20 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.331+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb68052c20 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.332+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.332+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.332+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.332+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb68052c20 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.332+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.332+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca57a41f0 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x564ca579f830 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.333+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8490 con 0x7fbb70021020 2024-09-19T11:24:21.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a7590 con 0x564ca579f830 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a7e30 con 0x564ca579f830 2024-09-19T11:24:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a71d0 con 0x564ca57a41f0 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 zone default found 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 4 Realm: () 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 zonegroup default 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.334+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a8e10 con 0x7fbb68052c20 2024-09-19T11:24:21.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57aa900 con 0x564ca57a41f0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57aa950 con 0x564ca57a41f0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 20 started sync module instance, tier type = 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.335+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a9c40 con 0x7fbb70021020 2024-09-19T11:24:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.336+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.336+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94887193778064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57aae20 con 0x7fbb70021020 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.336+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ab670 con 0x564ca579f830 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.337+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94887193778064] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.338+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.338+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94887193784768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57aba90 con 0x564ca579f830 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.338+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ac320 con 0x564ca57a41f0 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.339+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94887193784768] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.339+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.339+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94887193785824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57abeb0 con 0x564ca57a41f0 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.339+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57acd80 con 0x7fbb70021020 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.340+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.340+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94887193790592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ad620 con 0x7fbb70021020 2024-09-19T11:24:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.340+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ae120 con 0x564ca579f830 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.340+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94887193785824] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.341+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94887193790592] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.341+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.341+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94887193795600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ad170 con 0x564ca579f830 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.341+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ae9f0 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.342+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94887193795600] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.342+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.342+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94887193786880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ae920 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.342+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57af170 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.343+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94887193786880] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.343+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.343+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94887193797936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57aefa0 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.343+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57afa90 con 0x7fbb68052c20 2024-09-19T11:24:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94887193802080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57aff00 con 0x7fbb68052c20 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 20 add_watcher() i=0 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 20 add_watcher() i=1 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 20 add_watcher() i=2 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 20 add_watcher() i=3 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 20 add_watcher() i=4 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 20 add_watcher() i=5 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94887193797936] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.344+0000 7fbb78d19800 20 add_watcher() i=6 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.345+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94887193802080] v33'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.345+0000 7fbb78d19800 20 add_watcher() i=7 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.345+0000 7fbb78d19800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.345+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ae9f0 con 0x564ca579f830 2024-09-19T11:24:21.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.346+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.346+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94887193799152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57b0410 con 0x564ca579f830 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.346+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94887193799152] v33'206 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fba2d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fbb78d19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fbb78d19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fbb78d19800 5 note: GC not initialized 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fb9a7fff640 20 reqs_thread_entry: start 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fbb78d19800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fbb78d19800 20 Filter name: none 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fb9a5ffb640 20 reqs_thread_entry: start 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fbb78d19800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.347+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57e5920 con 0x564ca579f830 2024-09-19T11:24:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.348+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.348+0000 7fbb78d19800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.348+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564ca57e6d40 con 0x7fbb68052c20 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.348+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.349+0000 7fba2d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.349+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94887193799152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57e3fc0 con 0x564ca579f830 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.350+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94887193799152] v33'207 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.350+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94887193778064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57ae9f0 con 0x7fbb70021020 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.351+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94887193778064] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.351+0000 7fbb78d19800 20 remove_watcher() i=0 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.351+0000 7fbb78d19800 2 removed watcher, disabling cache 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.351+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94887193784768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a9790 con 0x564ca579f830 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.352+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94887193784768] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.352+0000 7fbb78d19800 20 remove_watcher() i=1 2024-09-19T11:24:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.352+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94887193785824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a9790 con 0x564ca57a41f0 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.354+0000 7fbb7700b640 1 -- 172.21.15.28:0/2359701960 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94887193785824] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb700083e0 con 0x564ca57a41f0 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.354+0000 7fbb78d19800 20 remove_watcher() i=2 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.354+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94887193790592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a9790 con 0x7fbb70021020 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.354+0000 7fbb7680a640 1 -- 172.21.15.28:0/2359701960 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94887193790592] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb6801b070 con 0x7fbb70021020 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.355+0000 7fbb78d19800 20 remove_watcher() i=3 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.355+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94887193795600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57a9790 con 0x564ca579f830 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.356+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94887193795600] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x564ca579f830 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.356+0000 7fbb78d19800 20 remove_watcher() i=4 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.356+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94887193786880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57adc10 con 0x7fbb68052c20 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.357+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94887193786880] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.357+0000 7fbb78d19800 20 remove_watcher() i=5 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.357+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94887193797936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57adc10 con 0x7fbb68052c20 2024-09-19T11:24:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.358+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94887193797936] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.358+0000 7fbb78d19800 20 remove_watcher() i=6 2024-09-19T11:24:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.358+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94887193802080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564ca57adc10 con 0x7fbb68052c20 2024-09-19T11:24:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94887193802080] v33'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb60051070 con 0x7fbb68052c20 2024-09-19T11:24:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 20 remove_watcher() i=7 2024-09-19T11:24:21.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb68052c20 msgr2=0x7fbb68054050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbb68052c20 0x7fbb68054050 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca579f830 msgr2=0x564ca57a1cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca579f830 0x564ca57a1cf0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb70021020 msgr2=0x7fbb7000a6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbb70021020 0x7fbb7000a6e0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca57a41f0 msgr2=0x564ca57a65e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca57a41f0 0x564ca57a65e0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4003d000 msgr2=0x7fba4003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba4003d000 0x7fba4003f4c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbb68052ba0 tx=0x7fbb68046000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 msgr2=0x564ca57940c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb76009640 1 -- 172.21.15.28:0/2359701960 reap_dead start 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca57940c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fbb700206f0 tx=0x7fbb700088f0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 shutdown_connections 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2359701960 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca578cae0 0x564ca57940c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 >> 172.21.15.28:0/2359701960 conn(0x564ca5788ee0 msgr2=0x564ca57870f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 shutdown_connections 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.359+0000 7fbb78d19800 1 -- 172.21.15.28:0/2359701960 wait complete. 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5003cfb0 msgr2=0x7fba5003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5003cfb0 0x7fba5003f470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x564ca576db60 tx=0x7fbb68014e90 comp rx=0 tx=0).stop 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca5780080 msgr2=0x564ca57804a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca5780080 0x564ca57804a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x564ca5766040 tx=0x7fbb7000bf30 comp rx=0 tx=0).stop 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 shutdown_connections 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5003cfb0 0x7fba5003f470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/1941910500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca5780080 0x564ca57804a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 >> 172.21.15.28:0/1941910500 conn(0x564ca5774430 msgr2=0x564ca5774810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 shutdown_connections 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/1941910500 wait complete. 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca566fa90 msgr2=0x564ca576c690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca566fa90 0x564ca576c690 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca576eab0 msgr2=0x564ca5770f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca576eab0 0x564ca5770f50 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5c03cfb0 msgr2=0x7fba5c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5c03cfb0 0x7fba5c03f470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb60057000 tx=0x7fbb60052000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 msgr2=0x564ca5765040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca5765040 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbb680066b0 tx=0x7fbb68001250 comp rx=0 tx=0).stop 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 shutdown_connections 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564ca576eab0 0x564ca5770f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fba5c03cfb0 0x7fba5c03f470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564ca566fa90 0x564ca576c690 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 --2- 172.21.15.28:0/2611090296 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ca566bf20 0x564ca5765040 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 >> 172.21.15.28:0/2611090296 conn(0x564ca5665a60 msgr2=0x564ca5756860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 shutdown_connections 2024-09-19T11:24:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.360+0000 7fbb78d19800 1 -- 172.21.15.28:0/2611090296 wait complete. 2024-09-19T11:24:21.371 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"s3select":{}},{"bucket":"myfoo2","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-09-19T11:24:21.371 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:24:21.371 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-09-19T11:24:21.371 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-09-19T11:24:21.372 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-09-19T11:24:21.419 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:21.420 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:21.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.419+0000 7fdb8a12c800 1 Processor -- start 2024-09-19T11:24:21.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.419+0000 7fdb8a12c800 1 -- start start 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.419+0000 7fdb8a12c800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560ae42e3eb0 0x560ae42e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.419+0000 7fdb8a12c800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560ae42ea5f0 con 0x560ae42e68d0 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.419+0000 7fdb8a12c800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x560ae416e7d0 con 0x560ae42e3eb0 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.420+0000 7fdb85e4f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560ae42e3eb0 0x560ae42e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.420+0000 7fdb85e4f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x560ae42e3eb0 0x560ae42e62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56544/0 (socket says 172.21.15.28:56544) 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.420+0000 7fdb8564e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x560ae42e68d0 0x560ae42e8d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59330/0 (socket says 172.21.15.28:59330) 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.420+0000 7fdb85e4f640 1 -- 172.21.15.28:0/3810492684 learned_addr learned my addr 172.21.15.28:0/3810492684 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.420+0000 7fdb85e4f640 1 -- 172.21.15.28:0/3810492684 >> v1:172.21.15.22:6789/0 conn(0x560ae42e68d0 legacy=0x560ae42e8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.420+0000 7fdb84e4d640 1 -- 172.21.15.28:0/3810492684 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643838521 0 0) 0x560ae42ea5f0 con 0x560ae42e68d0 2024-09-19T11:24:21.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.420+0000 7fdb85e4f640 1 -- 172.21.15.28:0/3810492684 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae42eb820 con 0x560ae42e3eb0 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/3810492684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e62a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800a520 tx=0x7fdb78045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b5d302dd8af4be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb84e4d640 1 -- 172.21.15.28:0/3810492684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb78055070 con 0x560ae42e3eb0 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb84e4d640 1 -- 172.21.15.28:0/3810492684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb780515b0 con 0x560ae42e3eb0 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb84e4d640 1 -- 172.21.15.28:0/3810492684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb780518d0 con 0x560ae42e3eb0 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3810492684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 msgr2=0x560ae42e62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/3810492684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e62a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800a520 tx=0x7fdb78045ba0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3810492684 shutdown_connections 2024-09-19T11:24:21.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/3810492684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e62a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3810492684 >> 172.21.15.28:0/3810492684 conn(0x560ae42dda70 msgr2=0x560ae42dfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3810492684 shutdown_connections 2024-09-19T11:24:21.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.422+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3810492684 wait complete. 2024-09-19T11:24:21.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb8a12c800 1 Processor -- start 2024-09-19T11:24:21.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb8a12c800 1 -- start start 2024-09-19T11:24:21.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb8a12c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb8a12c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ae417bdd0 con 0x560ae42e3eb0 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb85e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb85e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56554/0 (socket says 172.21.15.28:56554) 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb85e4f640 1 -- 172.21.15.28:0/3379113415 learned_addr learned my addr 172.21.15.28:0/3379113415 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.438+0000 7fdb85e4f640 1 -- 172.21.15.28:0/3379113415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae42e3bf0 con 0x560ae42e3eb0 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.439+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/3379113415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e3600 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdb78000c00 tx=0x7fdb78051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb23b043db86ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.439+0000 7fda73e07640 1 -- 172.21.15.28:0/3379113415 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb780519d0 con 0x560ae42e3eb0 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.439+0000 7fda73e07640 1 -- 172.21.15.28:0/3379113415 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb78051b70 con 0x560ae42e3eb0 2024-09-19T11:24:21.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.439+0000 7fda73e07640 1 -- 172.21.15.28:0/3379113415 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb78051e50 con 0x560ae42e3eb0 2024-09-19T11:24:21.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3379113415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 msgr2=0x560ae42e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/3379113415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e3600 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdb78000c00 tx=0x7fdb78051000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3379113415 shutdown_connections 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/3379113415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae42e3600 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3379113415 >> 172.21.15.28:0/3379113415 conn(0x560ae42dda70 msgr2=0x560ae43ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3379113415 shutdown_connections 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 -- 172.21.15.28:0/3379113415 wait complete. 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 Processor -- start 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 -- start start 2024-09-19T11:24:21.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae43dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb8a12c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ae42eaac0 con 0x560ae42e3eb0 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb85e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae43dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb85e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae43dcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56558/0 (socket says 172.21.15.28:56558) 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.440+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1256168138 learned_addr learned my addr 172.21.15.28:0/1256168138 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.441+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae43dd2f0 con 0x560ae42e3eb0 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.441+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae43dcd90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdb78005570 tx=0x7fdb78002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.441+0000 7fda72604640 1 -- 172.21.15.28:0/1256168138 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb78073070 con 0x560ae42e3eb0 2024-09-19T11:24:21.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.441+0000 7fda72604640 1 -- 172.21.15.28:0/1256168138 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7805a580 con 0x560ae42e3eb0 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.441+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ae43dd580 con 0x560ae42e3eb0 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.441+0000 7fda72604640 1 -- 172.21.15.28:0/1256168138 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb7805a8a0 con 0x560ae42e3eb0 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.441+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae43de190 con 0x560ae42e3eb0 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.442+0000 7fda72604640 1 -- 172.21.15.28:0/1256168138 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdb78076070 con 0x560ae42e3eb0 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.442+0000 7fda72604640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6c03d000 0x7fda6c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.442+0000 7fda72604640 1 -- 172.21.15.28:0/1256168138 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fdb78062070 con 0x560ae42e3eb0 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.442+0000 7fdb8564e640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6c03d000 0x7fda6c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.442+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae42e68d0 0x560ae43e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.442+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae43e4920 con 0x560ae42e68d0 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb86650640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae42e68d0 0x560ae43e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb8564e640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6c03d000 0x7fda6c03f4c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000c00 tx=0x7fdb7c0013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb86650640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae42e68d0 0x560ae43e43e0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb86650640 1 -- 172.21.15.28:0/1256168138 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae42e68d0 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae43e4cc0 con 0x560ae42e68d0 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb86650640 1 -- 172.21.15.28:0/1256168138 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae42e68d0 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae43e6780 0x560ae43e8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.443+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae43e93b0 con 0x560ae43e6780 2024-09-19T11:24:21.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.444+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae43e6780 0x560ae43e8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.444+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae43e6780 0x560ae43e8c20 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.444+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1256168138 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fdb7807f070 con 0x560ae43e6780 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.444+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae43e5510 con 0x560ae42e68d0 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.444+0000 7fdb86650640 1 -- 172.21.15.28:0/1256168138 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae42e68d0 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.444+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae43e5960 con 0x560ae43e6780 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.445+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1256168138 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fdb7807f070 con 0x560ae43e6780 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.445+0000 7fdb8a12c800 1 Processor -- start 2024-09-19T11:24:21.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.445+0000 7fdb8a12c800 1 -- start start 2024-09-19T11:24:21.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.445+0000 7fdb8a12c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae43f7d40 0x560ae43f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.445+0000 7fdb8a12c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ae41da4a0 con 0x560ae43f7d40 2024-09-19T11:24:21.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fdb86650640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae43f7d40 0x560ae43f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fdb86650640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae43f7d40 0x560ae43f8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56568/0 (socket says 172.21.15.28:56568) 2024-09-19T11:24:21.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fdb86650640 1 -- 172.21.15.28:0/2910672939 learned_addr learned my addr 172.21.15.28:0/2910672939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fdb86650640 1 -- 172.21.15.28:0/2910672939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae43f87a0 con 0x560ae43f7d40 2024-09-19T11:24:21.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fdb86650640 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae43f7d40 0x560ae43f8160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x560ae43ddd90 tx=0x7fdb800167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fda6b7fe640 1 -- 172.21.15.28:0/2910672939 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb80013640 con 0x560ae43f7d40 2024-09-19T11:24:21.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fda6b7fe640 1 -- 172.21.15.28:0/2910672939 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb800137e0 con 0x560ae43f7d40 2024-09-19T11:24:21.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ae43f8fd0 con 0x560ae43f7d40 2024-09-19T11:24:21.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fda6b7fe640 1 -- 172.21.15.28:0/2910672939 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb80013ac0 con 0x560ae43f7d40 2024-09-19T11:24:21.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.446+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae43f9570 con 0x560ae43f7d40 2024-09-19T11:24:21.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.447+0000 7fda6b7fe640 1 -- 172.21.15.28:0/2910672939 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdb80011040 con 0x560ae43f7d40 2024-09-19T11:24:21.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.447+0000 7fda6b7fe640 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6003d000 0x7fda6003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fda6b7fe640 1 -- 172.21.15.28:0/2910672939 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fdb8000e040 con 0x560ae43f7d40 2024-09-19T11:24:21.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6003d000 0x7fda6003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb8a12c800 1 Processor -- start 2024-09-19T11:24:21.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb8a12c800 1 -- start start 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6003d000 0x7fda6003f4c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb78051fd0 tx=0x7fdb78064f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb8a12c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae4404c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb8a12c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ae42e1f50 con 0x560ae4404800 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb86650640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae4404c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb86650640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae4404c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56576/0 (socket says 172.21.15.28:56576) 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.448+0000 7fdb86650640 1 -- 172.21.15.28:0/1836180136 learned_addr learned my addr 172.21.15.28:0/1836180136 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb86650640 1 -- 172.21.15.28:0/1836180136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae44051f0 con 0x560ae4404800 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb86650640 1 --2- 172.21.15.28:0/1836180136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae4404c00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdb80016cd0 tx=0x7fdb80020ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8183e861f18345 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fda697fa640 1 -- 172.21.15.28:0/1836180136 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb80021440 con 0x560ae4404800 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fda697fa640 1 -- 172.21.15.28:0/1836180136 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb80052070 con 0x560ae4404800 2024-09-19T11:24:21.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fda697fa640 1 -- 172.21.15.28:0/1836180136 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb80021620 con 0x560ae4404800 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1836180136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 msgr2=0x560ae4404c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1836180136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae4404c00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdb80016cd0 tx=0x7fdb80020ad0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1836180136 shutdown_connections 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1836180136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae4404c00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1836180136 >> 172.21.15.28:0/1836180136 conn(0x560ae4400c00 msgr2=0x560ae43fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1836180136 shutdown_connections 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.449+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1836180136 wait complete. 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb8a12c800 1 Processor -- start 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb8a12c800 1 -- start start 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb8a12c800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae440bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb8a12c800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ae4405810 con 0x560ae4404800 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb86650640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae440bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb86650640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae440bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56582/0 (socket says 172.21.15.28:56582) 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 learned_addr learned my addr 172.21.15.28:0/1105513012 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.450+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae440c310 con 0x560ae4404800 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.451+0000 7fdb86650640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae440bdb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdb800547b0 tx=0x7fdb80054810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.451+0000 7fda577fe640 1 -- 172.21.15.28:0/1105513012 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb800560d0 con 0x560ae4404800 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.451+0000 7fda577fe640 1 -- 172.21.15.28:0/1105513012 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb80056270 con 0x560ae4404800 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.451+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ae440c5a0 con 0x560ae4404800 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.451+0000 7fda577fe640 1 -- 172.21.15.28:0/1105513012 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb8001fa70 con 0x560ae4404800 2024-09-19T11:24:21.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.451+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae440d130 con 0x560ae4404800 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.451+0000 7fda577fe640 1 -- 172.21.15.28:0/1105513012 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdb8001fc10 con 0x560ae4404800 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fda577fe640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda5003cfb0 0x7fda5003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fda577fe640 1 -- 172.21.15.28:0/1105513012 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fdb80017450 con 0x560ae4404800 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda5003cfb0 0x7fda5003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae44176c0 0x560ae4419b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441a2d0 con 0x560ae44176c0 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fdb8564e640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae44176c0 0x560ae4419b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.452+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda5003cfb0 0x7fda5003f470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb78096dc0 tx=0x7fdb78092000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8564e640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae44176c0 0x560ae4419b80 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdb7c00f5f0 con 0x560ae44176c0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 20 realm 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae4416a10 con 0x560ae44176c0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdb7c00f5f0 con 0x560ae44176c0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441adb0 con 0x560ae44176c0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdb7c00f5f0 con 0x560ae44176c0 2024-09-19T11:24:21.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.453+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441b680 con 0x560ae44176c0 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdb7c00f5f0 con 0x560ae44176c0 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8a12c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae441bf80 0x560ae441e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441eaf0 con 0x560ae441bf80 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb86650640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae441bf80 0x560ae441e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb86650640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae441bf80 0x560ae441e390 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8a12c800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:21.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8a12c800 20 searching for the correct realm 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb86650640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdb80033440 0x7fdb80045a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.454+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.455+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdb80033440 0x7fdb80045a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.455+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdb80033440 0x7fdb80045a50 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.455+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.455+0000 7fdb85e4f640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdb780a5040 0x7fdb78096210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.455+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb780a5040 2024-09-19T11:24:21.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.455+0000 7fdb8564e640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdb780a5040 0x7fdb78096210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.456+0000 7fdb8564e640 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdb780a5040 0x7fdb78096210 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.456+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c00f5f0 con 0x7fdb780a5040 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.456+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.456+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c00f5f0 con 0x560ae44176c0 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.456+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.457+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.457+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb780a5040 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.457+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x7fdb780a5040 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.457+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb780a5040 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.457+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x7fdb780a5040 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.457+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb780a5040 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x7fdb780a5040 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.458+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.459+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.460+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.461+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.461+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb780a5040 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.461+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x7fdb780a5040 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.461+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.461+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.461+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb780a5040 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x7fdb780a5040 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.462+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb780a5040 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7c02b040 con 0x7fdb780a5040 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae441bf80 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.463+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x560ae44176c0 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x560ae44204d0 con 0x7fdb80033440 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:21.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441f5a0 con 0x560ae44176c0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441f1e0 con 0x560ae44176c0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdb7c02b040 con 0x560ae44176c0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.464+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441ff00 con 0x560ae441bf80 2024-09-19T11:24:21.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 zone default found 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 4 Realm: () 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 zonegroup default 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae441fa40 con 0x7fdb780a5040 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fdb7c02b040 con 0x7fdb780a5040 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae4422d40 con 0x560ae441bf80 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.465+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae44230e0 con 0x560ae441bf80 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.466+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.466+0000 7fdb8a12c800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.466+0000 7fdb8a12c800 20 started sync module instance, tier type = 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.466+0000 7fdb8a12c800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.466+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44221d0 con 0x7fdb80033440 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.467+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.467+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94604779201984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44239a0 con 0x7fdb80033440 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.467+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44241f0 con 0x560ae44176c0 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.468+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94604779201984] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.468+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.468+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94604779208000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4424610 con 0x560ae44176c0 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.468+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4424e60 con 0x560ae441bf80 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.469+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.469+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94604779210736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44252c0 con 0x560ae441bf80 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.469+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4425de0 con 0x7fdb80033440 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.469+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94604779208000] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.470+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94604779210736] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.470+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.470+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94604779215024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4424d10 con 0x7fdb80033440 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.470+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44267f0 con 0x560ae44176c0 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.471+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94604779215024] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.472+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.472+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94604779209056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44267f0 con 0x560ae44176c0 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.472+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44271b0 con 0x7fdb780a5040 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.473+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94604779209056] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.473+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.473+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94604779218208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44271b0 con 0x7fdb780a5040 2024-09-19T11:24:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.473+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4427b70 con 0x7fdb780a5040 2024-09-19T11:24:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.473+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94604779218208] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.474+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.474+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94604779223200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44271b0 con 0x7fdb780a5040 2024-09-19T11:24:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.474+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4428580 con 0x7fdb780a5040 2024-09-19T11:24:21.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.474+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94604779223200] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94604779220704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4428580 con 0x7fdb780a5040 2024-09-19T11:24:21.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 20 add_watcher() i=0 2024-09-19T11:24:21.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 20 add_watcher() i=1 2024-09-19T11:24:21.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 20 add_watcher() i=2 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 20 add_watcher() i=3 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 20 add_watcher() i=4 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 20 add_watcher() i=5 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.475+0000 7fdb8a12c800 20 add_watcher() i=6 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.476+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94604779220704] v33'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.476+0000 7fdb8a12c800 20 add_watcher() i=7 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.476+0000 7fdb8a12c800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.476+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae442b870 con 0x560ae44176c0 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.476+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.476+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94604779225664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4428b10 con 0x560ae44176c0 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94604779225664] v33'208 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fda377fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fdb8a12c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fdb8a12c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fdb8a12c800 5 note: GC not initialized 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fda367fc640 20 reqs_thread_entry: start 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fdb8a12c800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.477+0000 7fdb8a12c800 20 Filter name: none 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.478+0000 7fd9abfff640 20 reqs_thread_entry: start 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.478+0000 7fdb8a12c800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.478+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae445e310 con 0x560ae44176c0 2024-09-19T11:24:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.478+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.478+0000 7fdb8a12c800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.478+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560ae445f730 con 0x7fdb780a5040 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.479+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.480+0000 7fda377fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.480+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94604779225664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae442a050 con 0x560ae44176c0 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.481+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94604779225664] v33'209 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.481+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94604779201984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4428240 con 0x7fdb80033440 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.482+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94604779201984] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.482+0000 7fdb8a12c800 20 remove_watcher() i=0 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.482+0000 7fdb8a12c800 2 removed watcher, disabling cache 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.482+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94604779208000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4428240 con 0x560ae44176c0 2024-09-19T11:24:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.483+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94604779208000] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.483+0000 7fdb8a12c800 20 remove_watcher() i=1 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.483+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94604779210736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4423d40 con 0x560ae441bf80 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.484+0000 7fdb86650640 1 -- 172.21.15.28:0/1105513012 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94604779210736] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb8000a4d0 con 0x560ae441bf80 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.484+0000 7fdb8a12c800 20 remove_watcher() i=2 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.484+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94604779215024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4428240 con 0x7fdb80033440 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.485+0000 7fdb85e4f640 1 -- 172.21.15.28:0/1105513012 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94604779215024] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7807f070 con 0x7fdb80033440 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.485+0000 7fdb8a12c800 20 remove_watcher() i=3 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.485+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94604779209056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae4423d40 con 0x560ae44176c0 2024-09-19T11:24:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.486+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94604779209056] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x560ae44176c0 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.486+0000 7fdb8a12c800 20 remove_watcher() i=4 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.486+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94604779218208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44258b0 con 0x7fdb780a5040 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.487+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94604779218208] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.487+0000 7fdb8a12c800 20 remove_watcher() i=5 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.487+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94604779223200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44258b0 con 0x7fdb780a5040 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.488+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94604779223200] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.488+0000 7fdb8a12c800 20 remove_watcher() i=6 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.488+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94604779220704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560ae44258b0 con 0x7fdb780a5040 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94604779220704] v33'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdb7c03b040 con 0x7fdb780a5040 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 20 remove_watcher() i=7 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdb780a5040 msgr2=0x7fdb78096210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fdb780a5040 0x7fdb78096210 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae44176c0 msgr2=0x560ae4419b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae44176c0 0x560ae4419b80 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdb80033440 msgr2=0x7fdb80045a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fdb80033440 0x7fdb80045a50 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae441bf80 msgr2=0x560ae441e390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae441bf80 0x560ae441e390 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda5003cfb0 msgr2=0x7fda5003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda5003cfb0 0x7fda5003f470 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb78096dc0 tx=0x7fdb78092000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8564e640 1 -- 172.21.15.28:0/1105513012 reap_dead start 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 msgr2=0x560ae440bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae440bdb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdb800547b0 tx=0x7fdb80054810 comp rx=0 tx=0).stop 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 shutdown_connections 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1105513012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae4404800 0x560ae440bdb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 >> 172.21.15.28:0/1105513012 conn(0x560ae4400c00 msgr2=0x560ae43fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 shutdown_connections 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.489+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1105513012 wait complete. 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6003d000 msgr2=0x7fda6003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6003d000 0x7fda6003f4c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb78051fd0 tx=0x7fdb78064f50 comp rx=0 tx=0).stop 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae43f7d40 msgr2=0x560ae43f8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae43f7d40 0x560ae43f8160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x560ae43ddd90 tx=0x7fdb800167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 shutdown_connections 2024-09-19T11:24:21.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6003d000 0x7fda6003f4c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/2910672939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae43f7d40 0x560ae43f8160 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 >> 172.21.15.28:0/2910672939 conn(0x560ae43ec0f0 msgr2=0x560ae43ec4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 shutdown_connections 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/2910672939 wait complete. 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae42e68d0 msgr2=0x560ae43e43e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae42e68d0 0x560ae43e43e0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae43e6780 msgr2=0x560ae43e8c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae43e6780 0x560ae43e8c20 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6c03d000 msgr2=0x7fda6c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6c03d000 0x7fda6c03f4c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c000c00 tx=0x7fdb7c0013b0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 msgr2=0x560ae43dcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae43dcd90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdb78005570 tx=0x7fdb78002fb0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 shutdown_connections 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560ae43e6780 0x560ae43e8c20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda6c03d000 0x7fda6c03f4c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x560ae42e68d0 0x560ae43e43e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 --2- 172.21.15.28:0/1256168138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ae42e3eb0 0x560ae43dcd90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 >> 172.21.15.28:0/1256168138 conn(0x560ae42dda70 msgr2=0x560ae43ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 shutdown_connections 2024-09-19T11:24:21.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.490+0000 7fdb8a12c800 1 -- 172.21.15.28:0/1256168138 wait complete. 2024-09-19T11:24:21.512 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo2","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo3","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo4","time":"2024-09-19T11:00:00.000000Z","epoch":1726743600,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4}],"total":{"bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-09-19T11:24:21.512 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:24:21.513 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-09-19T11:24:21.513 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-09-19T11:24:21.513 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-09-19T11:24:21.567 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:21.567 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.566+0000 7fa9aaa94800 1 Processor -- start 2024-09-19T11:24:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9aaa94800 1 -- start start 2024-09-19T11:24:21.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9aaa94800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55fa8d295e40 0x55fa8d29b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9aaa94800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55fa8d2962f0 con 0x55fa8d292d20 2024-09-19T11:24:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9aaa94800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55fa8d11f7d0 con 0x55fa8d295e40 2024-09-19T11:24:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9a7d7d640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55fa8d295e40 0x55fa8d29b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9a857e640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55fa8d292d20 0x55fa8d295730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59344/0 (socket says 172.21.15.28:59344) 2024-09-19T11:24:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9a857e640 1 -- 172.21.15.28:0/2460957086 learned_addr learned my addr 172.21.15.28:0/2460957086 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/2460957086 >> v1:172.21.15.22:6789/0 conn(0x55fa8d292d20 legacy=0x55fa8d295730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.567+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/2460957086 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8d2964c0 con 0x55fa8d295e40 2024-09-19T11:24:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.569+0000 7fa9a7d7d640 1 --2- 172.21.15.28:0/2460957086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d295e40 0x55fa8d29b790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa99000a470 tx=0x7fa990045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f6717be1509e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.569+0000 7fa9a757c640 1 -- 172.21.15.28:0/2460957086 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa990053570 con 0x55fa8d295e40 2024-09-19T11:24:21.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.569+0000 7fa9a757c640 1 -- 172.21.15.28:0/2460957086 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa990058070 con 0x55fa8d295e40 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.569+0000 7fa9a757c640 1 -- 172.21.15.28:0/2460957086 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa990053880 con 0x55fa8d295e40 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.570+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2460957086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d295e40 msgr2=0x55fa8d29b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.570+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2460957086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d295e40 0x55fa8d29b790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fa99000a470 tx=0x7fa990045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.570+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2460957086 shutdown_connections 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.570+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2460957086 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d295e40 0x55fa8d29b790 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.570+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2460957086 >> 172.21.15.28:0/2460957086 conn(0x55fa8d28e200 msgr2=0x55fa8d290680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.570+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2460957086 shutdown_connections 2024-09-19T11:24:21.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.570+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2460957086 wait complete. 2024-09-19T11:24:21.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.585+0000 7fa9aaa94800 1 Processor -- start 2024-09-19T11:24:21.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.585+0000 7fa9aaa94800 1 -- start start 2024-09-19T11:24:21.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.585+0000 7fa9aaa94800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d384660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.585+0000 7fa9aaa94800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fa8d12cdd0 con 0x55fa8d292d20 2024-09-19T11:24:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa9a857e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d384660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa9a857e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d384660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56588/0 (socket says 172.21.15.28:56588) 2024-09-19T11:24:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa9a857e640 1 -- 172.21.15.28:0/2832803402 learned_addr learned my addr 172.21.15.28:0/2832803402 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa9a857e640 1 -- 172.21.15.28:0/2832803402 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8d384c50 con 0x55fa8d292d20 2024-09-19T11:24:21.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa9a857e640 1 --2- 172.21.15.28:0/2832803402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d384660 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa998001800 tx=0x7fa998001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874d926bce5c2f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa89660c640 1 -- 172.21.15.28:0/2832803402 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa998004520 con 0x55fa8d292d20 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa89660c640 1 -- 172.21.15.28:0/2832803402 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9980046c0 con 0x55fa8d292d20 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.586+0000 7fa89660c640 1 -- 172.21.15.28:0/2832803402 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9980049e0 con 0x55fa8d292d20 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2832803402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 msgr2=0x55fa8d384660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2832803402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d384660 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fa998001800 tx=0x7fa998001cc0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2832803402 shutdown_connections 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2832803402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d384660 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2832803402 >> 172.21.15.28:0/2832803402 conn(0x55fa8d28e200 msgr2=0x55fa8d37ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2832803402 shutdown_connections 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2832803402 wait complete. 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.587+0000 7fa9aaa94800 1 Processor -- start 2024-09-19T11:24:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9aaa94800 1 -- start start 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9aaa94800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d38c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9aaa94800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fa8d29bfd0 con 0x55fa8d292d20 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9a857e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d38c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9a857e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d38c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56594/0 (socket says 172.21.15.28:56594) 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9a857e640 1 -- 172.21.15.28:0/2038641457 learned_addr learned my addr 172.21.15.28:0/2038641457 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9a857e640 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8d38c730 con 0x55fa8d292d20 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.588+0000 7fa9a857e640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d38c1d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa998000f80 tx=0x7fa998001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.589+0000 7fa894e09640 1 -- 172.21.15.28:0/2038641457 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa998022020 con 0x55fa8d292d20 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.589+0000 7fa894e09640 1 -- 172.21.15.28:0/2038641457 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa998014890 con 0x55fa8d292d20 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.589+0000 7fa894e09640 1 -- 172.21.15.28:0/2038641457 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa998014bb0 con 0x55fa8d292d20 2024-09-19T11:24:21.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.589+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fa8d38c9c0 con 0x55fa8d292d20 2024-09-19T11:24:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.589+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8d38d3c0 con 0x55fa8d292d20 2024-09-19T11:24:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.589+0000 7fa894e09640 1 -- 172.21.15.28:0/2038641457 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa998025020 con 0x55fa8d292d20 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa894e09640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa89003cfb0 0x7fa89003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa894e09640 1 -- 172.21.15.28:0/2038641457 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa998018030 con 0x55fa8d292d20 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa9a7d7d640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa89003cfb0 0x7fa89003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d295e40 0x55fa8d393820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d393d60 con 0x55fa8d295e40 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d295e40 0x55fa8d393820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa9a7d7d640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa89003cfb0 0x7fa89003f470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa990057000 tx=0x7fa990052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.590+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d295e40 0x55fa8d393820 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.591+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/2038641457 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa9a00083e0 con 0x55fa8d295e40 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.591+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d394100 con 0x55fa8d295e40 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.591+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/2038641457 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa9a00083e0 con 0x55fa8d295e40 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.591+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d395c00 0x55fa8d398080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.591+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d398810 con 0x55fa8d395c00 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.591+0000 7fa9a857e640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d395c00 0x55fa8d398080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.591+0000 7fa9a857e640 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d395c00 0x55fa8d398080 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.592+0000 7fa9a857e640 1 -- 172.21.15.28:0/2038641457 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa99801b070 con 0x55fa8d395c00 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.592+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d394950 con 0x55fa8d295e40 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.592+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/2038641457 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa9a00083e0 con 0x55fa8d295e40 2024-09-19T11:24:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.592+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d394da0 con 0x55fa8d395c00 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.592+0000 7fa9a857e640 1 -- 172.21.15.28:0/2038641457 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa99801b070 con 0x55fa8d395c00 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9aaa94800 1 Processor -- start 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9aaa94800 1 -- start start 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9aaa94800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3a71e0 0x55fa8d3a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9aaa94800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fa8d18b4a0 con 0x55fa8d3a71e0 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9a8d7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3a71e0 0x55fa8d3a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9a8d7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3a71e0 0x55fa8d3a7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56598/0 (socket says 172.21.15.28:56598) 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/556992745 learned_addr learned my addr 172.21.15.28:0/556992745 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.593+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/556992745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8d3a7c40 con 0x55fa8d3a71e0 2024-09-19T11:24:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.594+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3a71e0 0x55fa8d3a7600 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55fa8d38d1d0 tx=0x7fa9a000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.594+0000 7fa88dffb640 1 -- 172.21.15.28:0/556992745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a001c020 con 0x55fa8d3a71e0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.594+0000 7fa88dffb640 1 -- 172.21.15.28:0/556992745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a0007c90 con 0x55fa8d3a71e0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.594+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fa8d3a8470 con 0x55fa8d3a71e0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.594+0000 7fa88dffb640 1 -- 172.21.15.28:0/556992745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a0018440 con 0x55fa8d3a71e0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.594+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8d3a8a10 con 0x55fa8d3a71e0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa88dffb640 1 -- 172.21.15.28:0/556992745 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa9a001f020 con 0x55fa8d3a71e0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa88dffb640 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa88003cfb0 0x7fa88003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa88dffb640 1 -- 172.21.15.28:0/556992745 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa9a0055b40 con 0x55fa8d3a71e0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa9a857e640 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa88003cfb0 0x7fa88003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa9aaa94800 1 Processor -- start 2024-09-19T11:24:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa9a857e640 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa88003cfb0 0x7fa88003f470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa998052b50 tx=0x7fa998011f40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa9aaa94800 1 -- start start 2024-09-19T11:24:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa9aaa94800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.595+0000 7fa9aaa94800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fa8d385080 con 0x55fa8d3b3c20 2024-09-19T11:24:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa9a8d7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa9a8d7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56612/0 (socket says 172.21.15.28:56612) 2024-09-19T11:24:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/74754641 learned_addr learned my addr 172.21.15.28:0/74754641 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/74754641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8d3b4610 con 0x55fa8d3b3c20 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/74754641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3b4020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0053df0 tx=0x7fa9a0015970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a8ee9216ec70a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa8877fe640 1 -- 172.21.15.28:0/74754641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a004a070 con 0x55fa8d3b3c20 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa8877fe640 1 -- 172.21.15.28:0/74754641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a0015e70 con 0x55fa8d3b3c20 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.596+0000 7fa8877fe640 1 -- 172.21.15.28:0/74754641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a00174d0 con 0x55fa8d3b3c20 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 -- 172.21.15.28:0/74754641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 msgr2=0x55fa8d3b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/74754641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3b4020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0053df0 tx=0x7fa9a0015970 comp rx=0 tx=0).stop 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 -- 172.21.15.28:0/74754641 shutdown_connections 2024-09-19T11:24:21.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/74754641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3b4020 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 -- 172.21.15.28:0/74754641 >> 172.21.15.28:0/74754641 conn(0x55fa8d3b0040 msgr2=0x55fa8d3ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 -- 172.21.15.28:0/74754641 shutdown_connections 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 -- 172.21.15.28:0/74754641 wait complete. 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 Processor -- start 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 -- start start 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3bb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.597+0000 7fa9aaa94800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fa8d3b4c30 con 0x55fa8d3b3c20 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa9a8d7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa9a8d7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3bb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56628/0 (socket says 172.21.15.28:56628) 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 learned_addr learned my addr 172.21.15.28:0/3861027731 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8d3bb780 con 0x55fa8d3b3c20 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3bb220 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00206f0 tx=0x7fa9a0053d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa885ffb640 1 -- 172.21.15.28:0/3861027731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a004a070 con 0x55fa8d3b3c20 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa885ffb640 1 -- 172.21.15.28:0/3861027731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9a004b040 con 0x55fa8d3b3c20 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fa8d3bba10 con 0x55fa8d3b3c20 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa885ffb640 1 -- 172.21.15.28:0/3861027731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a0017730 con 0x55fa8d3b3c20 2024-09-19T11:24:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.598+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8d3bc5a0 con 0x55fa8d3b3c20 2024-09-19T11:24:21.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.599+0000 7fa885ffb640 1 -- 172.21.15.28:0/3861027731 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa9a0016040 con 0x55fa8d3b3c20 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.599+0000 7fa885ffb640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa87803cf60 0x7fa87803f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.599+0000 7fa885ffb640 1 -- 172.21.15.28:0/3861027731 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa9a0055040 con 0x55fa8d3b3c20 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.599+0000 7fa9a857e640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa87803cf60 0x7fa87803f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.600+0000 7fa9a857e640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa87803cf60 0x7fa87803f420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa998052db0 tx=0x7fa998046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.600+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.600+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d3c6b40 0x55fa8d3c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.600+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3c9750 con 0x55fa8d3c6b40 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.600+0000 7fa9a7d7d640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d3c6b40 0x55fa8d3c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9a7d7d640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d3c6b40 0x55fa8d3c9000 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 20 realm 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3c9b80 con 0x55fa8d3c6b40 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3ca450 con 0x55fa8d3c6b40 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.601+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3ca7f0 con 0x55fa8d3c6b40 2024-09-19T11:24:21.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9aaa94800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d3cb550 0x55fa8d3cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cdf50 con 0x55fa8d3cb550 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d3cb550 0x55fa8d3cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d3cb550 0x55fa8d3cd940 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa9a00083e0 con 0x55fa8d3cb550 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9aaa94800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9aaa94800 20 searching for the correct realm 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a00083e0 con 0x55fa8d3cb550 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9a8d7f640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa9a0021020 0x7fa9a0024ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.602+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.603+0000 7fa9a857e640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa9a0021020 0x7fa9a0024ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.603+0000 7fa9a857e640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa9a0021020 0x7fa9a0024ab0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.603+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.603+0000 7fa9a857e640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa998027020 0x7fa998054930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.603+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa998027020 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.603+0000 7fa9a7d7d640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa998027020 0x7fa998054930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.604+0000 7fa9a7d7d640 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa998027020 0x7fa998054930 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.604+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.604+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.604+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.604+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.604+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.604+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa998027020 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa998027020 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a00083e0 con 0x55fa8d3cb550 2024-09-19T11:24:21.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.605+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa998027020 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.606+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a00083e0 con 0x55fa8d3cb550 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.607+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa998027020 2024-09-19T11:24:21.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.608+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa998027020 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.609+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa998027020 2024-09-19T11:24:21.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3cb550 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.610+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x55fa8d3c6b40 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf880 con 0x7fa9a0021020 2024-09-19T11:24:21.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3ce910 con 0x55fa8d3c6b40 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.611+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cf160 con 0x55fa8d3c6b40 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3cfd00 con 0x55fa8d3cb550 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 20 zone default found 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 4 Realm: () 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 20 zonegroup default 2024-09-19T11:24:21.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.612+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d00a0 con 0x7fa998027020 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d1f40 con 0x55fa8d3cb550 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d22e0 con 0x55fa8d3cb550 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 20 started sync module instance, tier type = 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.613+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d1380 con 0x7fa9a0021020 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.614+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.614+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94534599776112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d1b90 con 0x7fa9a0021020 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.614+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d2c40 con 0x55fa8d3c6b40 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.615+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94534599776112] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.615+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.615+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94534599779520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d3030 con 0x55fa8d3c6b40 2024-09-19T11:24:21.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.615+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d3b00 con 0x55fa8d3cb550 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.616+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94534599779520] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.617+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.617+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94534599783920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d3ac0 con 0x55fa8d3cb550 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.617+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d4570 con 0x7fa9a0021020 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.618+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94534599783920] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.618+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.618+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94534599786688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d4990 con 0x7fa9a0021020 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.618+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d53d0 con 0x55fa8d3c6b40 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.619+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.619+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94534599790240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d5c70 con 0x55fa8d3c6b40 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.619+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d6730 con 0x7fa998027020 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.619+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94534599786688] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.620+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.620+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94534599795200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d66f0 con 0x7fa998027020 2024-09-19T11:24:21.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.620+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d7140 con 0x7fa998027020 2024-09-19T11:24:21.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.620+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94534599790240] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.621+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.621+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94534599792576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d7140 con 0x7fa998027020 2024-09-19T11:24:21.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.621+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d7b50 con 0x7fa998027020 2024-09-19T11:24:21.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.621+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94534599795200] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94534599792576] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94534599800480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d7140 con 0x7fa998027020 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 20 add_watcher() i=0 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 20 add_watcher() i=1 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 20 add_watcher() i=2 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 20 add_watcher() i=3 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 20 add_watcher() i=4 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 20 add_watcher() i=5 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.622+0000 7fa9aaa94800 20 add_watcher() i=6 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.623+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94534599800480] v33'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.623+0000 7fa9aaa94800 20 add_watcher() i=7 2024-09-19T11:24:21.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.623+0000 7fa9aaa94800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.623+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d7b30 con 0x55fa8d3c6b40 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.624+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.624+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94534599797904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d3d7fe0 con 0x55fa8d3c6b40 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.625+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94534599797904] v33'210 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.625+0000 7fa85e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.625+0000 7fa9aaa94800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.625+0000 7fa9aaa94800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.625+0000 7fa9aaa94800 5 note: GC not initialized 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.625+0000 7fa85d7e2640 20 reqs_thread_entry: start 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 20 Filter name: none 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa7ceffd640 20 reqs_thread_entry: start 2024-09-19T11:24:21.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40d860 con 0x7fa998027020 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40bf00 con 0x55fa8d3c6b40 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.626+0000 7fa9aaa94800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-19T11:24:21.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.627+0000 7fa9aaa94800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.627+0000 7fa9aaa94800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.627+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40c7d0 con 0x7fa998027020 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~380] v33'11 uv11 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa9aaa94800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa9aaa94800 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa9aaa94800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94534600016672 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d4111a0 con 0x7fa998027020 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== watch-notify(notify (1) cookie 94534599800480 notify 141733920797 ret 0) ==== 598+0+0 (crc 0 0 0) 0x55fa8d40cb70 con 0x7fa998027020 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa8877fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920797 cookie 94534599800480 notifier 4726 bl.length()=556 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa8877fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(65 notify.7 [notify cookie 94534600016672 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa8877fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.628+0000 7fa8877fe640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fa86c003ea0 con 0x7fa998027020 2024-09-19T11:24:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.629+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.629+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== watch-notify(notify_complete (2) cookie 94534600016672 notify 141733920797 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55fa8d40d1b0 con 0x7fa998027020 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.629+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40c2d0 con 0x55fa8d3c6b40 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v33'3 uv3 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9aaa94800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9aaa94800 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9aaa94800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94534600031456 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40c2d0 con 0x7fa998027020 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== watch-notify(notify (1) cookie 94534599795200 notify 141733920798 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55fa8d40d1b0 con 0x7fa998027020 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa8877fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920798 cookie 94534599795200 notifier 4726 bl.length()=177 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa8877fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa8877fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa8877fe640 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fa86c003ea0 con 0x7fa998027020 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(68 notify.5 [notify cookie 94534600031456 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.630+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.631+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== watch-notify(notify_complete (2) cookie 94534600031456 notify 141733920798 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fa990081040 con 0x7fa998027020 2024-09-19T11:24:21.648 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"bar","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bar","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-09-19T11:24:21.627791Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:24:21.627767Z","tags":[],"group_ids":[]} 2024-09-19T11:24:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.632+0000 7fa85e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.632+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94534599797904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x55fa8d3c6b40 2024-09-19T11:24:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.632+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94534599797904] v33'211 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.632+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94534599776112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x7fa9a0021020 2024-09-19T11:24:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.633+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94534599776112] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.633+0000 7fa9aaa94800 20 remove_watcher() i=0 2024-09-19T11:24:21.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.633+0000 7fa9aaa94800 2 removed watcher, disabling cache 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.633+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94534599779520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x55fa8d3c6b40 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.635+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94534599779520] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.635+0000 7fa9aaa94800 20 remove_watcher() i=1 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.635+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94534599783920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x55fa8d3cb550 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.636+0000 7fa9a8d7f640 1 -- 172.21.15.28:0/3861027731 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94534599783920] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa9a0062040 con 0x55fa8d3cb550 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.636+0000 7fa9aaa94800 20 remove_watcher() i=2 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.636+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94534599786688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x7fa9a0021020 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.637+0000 7fa9a857e640 1 -- 172.21.15.28:0/3861027731 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94534599786688] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa99801b070 con 0x7fa9a0021020 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.637+0000 7fa9aaa94800 20 remove_watcher() i=3 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.637+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94534599790240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x55fa8d3c6b40 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.638+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94534599790240] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x55fa8d3c6b40 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.638+0000 7fa9aaa94800 20 remove_watcher() i=4 2024-09-19T11:24:21.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.638+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94534599795200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x7fa998027020 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.639+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94534599795200] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.639+0000 7fa9aaa94800 20 remove_watcher() i=5 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.639+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94534599792576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x7fa998027020 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.640+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94534599792576] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.640+0000 7fa9aaa94800 20 remove_watcher() i=6 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.640+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94534599800480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fa8d40b630 con 0x7fa998027020 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94534599800480] v33'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa990051070 con 0x7fa998027020 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 20 remove_watcher() i=7 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa998027020 msgr2=0x7fa998054930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa998027020 0x7fa998054930 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d3c6b40 msgr2=0x55fa8d3c9000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d3c6b40 0x55fa8d3c9000 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa9a0021020 msgr2=0x7fa9a0024ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa9a0021020 0x7fa9a0024ab0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d3cb550 msgr2=0x55fa8d3cd940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d3cb550 0x55fa8d3cd940 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa87803cf60 msgr2=0x7fa87803f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.641+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa87803cf60 0x7fa87803f420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa998052db0 tx=0x7fa998046000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 msgr2=0x55fa8d3bb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/3861027731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3b3c20 0x55fa8d3bb220 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00206f0 tx=0x7fa9a0053d50 comp rx=0 tx=0).stop 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 reap_dead start 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9a7d7d640 1 -- 172.21.15.28:0/3861027731 reap_dead start 2024-09-19T11:24:21.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 shutdown_connections 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 >> 172.21.15.28:0/3861027731 conn(0x55fa8d3b0040 msgr2=0x55fa8d3ae250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 shutdown_connections 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/3861027731 wait complete. 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa88003cfb0 msgr2=0x7fa88003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa88003cfb0 0x7fa88003f470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa998052b50 tx=0x7fa998011f40 comp rx=0 tx=0).stop 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3a71e0 msgr2=0x55fa8d3a7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3a71e0 0x55fa8d3a7600 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55fa8d38d1d0 tx=0x7fa9a000bf30 comp rx=0 tx=0).stop 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 shutdown_connections 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa88003cfb0 0x7fa88003f470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/556992745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d3a71e0 0x55fa8d3a7600 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 >> 172.21.15.28:0/556992745 conn(0x55fa8d39b590 msgr2=0x55fa8d39b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 shutdown_connections 2024-09-19T11:24:21.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/556992745 wait complete. 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d295e40 msgr2=0x55fa8d393820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d295e40 0x55fa8d393820 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d395c00 msgr2=0x55fa8d398080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d395c00 0x55fa8d398080 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa89003cfb0 msgr2=0x7fa89003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa89003cfb0 0x7fa89003f470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa990057000 tx=0x7fa990052000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 msgr2=0x55fa8d38c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.642+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d38c1d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa998000f80 tx=0x7fa998001250 comp rx=0 tx=0).stop 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 shutdown_connections 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fa8d395c00 0x55fa8d398080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa89003cfb0 0x7fa89003f470 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fa8d295e40 0x55fa8d393820 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 --2- 172.21.15.28:0/2038641457 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fa8d292d20 0x55fa8d38c1d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 >> 172.21.15.28:0/2038641457 conn(0x55fa8d28e200 msgr2=0x55fa8d385b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 shutdown_connections 2024-09-19T11:24:21.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.643+0000 7fa9aaa94800 1 -- 172.21.15.28:0/2038641457 wait complete. 2024-09-19T11:24:21.659 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-09-19T11:24:21.627791Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:24:21.627767Z', 'tags': [], 'group_ids': []} 2024-09-19T11:24:21.679 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:24:21.684 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-09-19T11:24:21.684 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-09-19T11:24:21.685 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-09-19T11:24:21.685 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-09-19T11:24:21.732 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:21.732 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f8911b2e800 1 Processor -- start 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f8911b2e800 1 -- start start 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f8911b2e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x556b50085e10 0x556b5008b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f8911b2e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556b500862c0 con 0x556b50082cf0 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f8911b2e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x556b4ff0f7d0 con 0x556b50085e10 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f890d0dc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x556b50085e10 0x556b5008b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f890d8dd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x556b50082cf0 0x556b50085700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59356/0 (socket says 172.21.15.28:59356) 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f890d8dd640 1 -- 172.21.15.28:0/3638455427 learned_addr learned my addr 172.21.15.28:0/3638455427 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f890d0dc640 1 -- 172.21.15.28:0/3638455427 >> v1:172.21.15.22:6789/0 conn(0x556b50082cf0 legacy=0x556b50085700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.732+0000 7f890d0dc640 1 -- 172.21.15.28:0/3638455427 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b50086490 con 0x556b50085e10 2024-09-19T11:24:21.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.734+0000 7f890d0dc640 1 --2- 172.21.15.28:0/3638455427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50085e10 0x556b5008b760 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f890400a470 tx=0x7f8904045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebe0e5e844ce0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.734+0000 7f890c8db640 1 -- 172.21.15.28:0/3638455427 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904053570 con 0x556b50085e10 2024-09-19T11:24:21.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.734+0000 7f890c8db640 1 -- 172.21.15.28:0/3638455427 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8904058070 con 0x556b50085e10 2024-09-19T11:24:21.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.734+0000 7f890c8db640 1 -- 172.21.15.28:0/3638455427 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8904053880 con 0x556b50085e10 2024-09-19T11:24:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.735+0000 7f8911b2e800 1 -- 172.21.15.28:0/3638455427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50085e10 msgr2=0x556b5008b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.735+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3638455427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50085e10 0x556b5008b760 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f890400a470 tx=0x7f8904045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.735+0000 7f8911b2e800 1 -- 172.21.15.28:0/3638455427 shutdown_connections 2024-09-19T11:24:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.735+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3638455427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50085e10 0x556b5008b760 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.735+0000 7f8911b2e800 1 -- 172.21.15.28:0/3638455427 >> 172.21.15.28:0/3638455427 conn(0x556b5007e1d0 msgr2=0x556b50080650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.735+0000 7f8911b2e800 1 -- 172.21.15.28:0/3638455427 shutdown_connections 2024-09-19T11:24:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.735+0000 7f8911b2e800 1 -- 172.21.15.28:0/3638455427 wait complete. 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.750+0000 7f8911b2e800 1 Processor -- start 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.750+0000 7f8911b2e800 1 -- start start 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.750+0000 7f8911b2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b50084e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.750+0000 7f8911b2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556b4ff1cdd0 con 0x556b50082cf0 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.750+0000 7f890d8dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b50084e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.750+0000 7f890d8dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b50084e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56644/0 (socket says 172.21.15.28:56644) 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.750+0000 7f890d8dd640 1 -- 172.21.15.28:0/259667385 learned_addr learned my addr 172.21.15.28:0/259667385 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.751+0000 7f890d8dd640 1 -- 172.21.15.28:0/259667385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b50085460 con 0x556b50082cf0 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.751+0000 7f890d8dd640 1 --2- 172.21.15.28:0/259667385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b50084e70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8900001e40 tx=0x7f8900001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a89d5b4143a5cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.751+0000 7f87fbe07640 1 -- 172.21.15.28:0/259667385 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900012020 con 0x556b50082cf0 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.751+0000 7f87fbe07640 1 -- 172.21.15.28:0/259667385 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890000e5a0 con 0x556b50082cf0 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.751+0000 7f87fbe07640 1 -- 172.21.15.28:0/259667385 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f890000e8a0 con 0x556b50082cf0 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 -- 172.21.15.28:0/259667385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 msgr2=0x556b50084e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 --2- 172.21.15.28:0/259667385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b50084e70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8900001e40 tx=0x7f8900001e70 comp rx=0 tx=0).stop 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 -- 172.21.15.28:0/259667385 shutdown_connections 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 --2- 172.21.15.28:0/259667385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b50084e70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 -- 172.21.15.28:0/259667385 >> 172.21.15.28:0/259667385 conn(0x556b5007e1d0 msgr2=0x556b5016fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 -- 172.21.15.28:0/259667385 shutdown_connections 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 -- 172.21.15.28:0/259667385 wait complete. 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 Processor -- start 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 -- start start 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b5017e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.752+0000 7f8911b2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556b5008bfa0 con 0x556b50082cf0 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f890d8dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b5017e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f890d8dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b5017e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56652/0 (socket says 172.21.15.28:56652) 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f890d8dd640 1 -- 172.21.15.28:0/3063424222 learned_addr learned my addr 172.21.15.28:0/3063424222 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f890d8dd640 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b5017e7d0 con 0x556b50082cf0 2024-09-19T11:24:21.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f890d8dd640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b5017e270 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f890000c010 tx=0x7f8900000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f87fa604640 1 -- 172.21.15.28:0/3063424222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8900012030 con 0x556b50082cf0 2024-09-19T11:24:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f87fa604640 1 -- 172.21.15.28:0/3063424222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89000014a0 con 0x556b50082cf0 2024-09-19T11:24:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f87fa604640 1 -- 172.21.15.28:0/3063424222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f890001a7a0 con 0x556b50082cf0 2024-09-19T11:24:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b5017eac0 con 0x556b50082cf0 2024-09-19T11:24:21.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.753+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5017f670 con 0x556b50082cf0 2024-09-19T11:24:21.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.754+0000 7f87fa604640 1 -- 172.21.15.28:0/3063424222 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8900015020 con 0x556b50082cf0 2024-09-19T11:24:21.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.754+0000 7f87fa604640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87f403d000 0x7f87f403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.754+0000 7f87fa604640 1 -- 172.21.15.28:0/3063424222 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f8900020030 con 0x556b50082cf0 2024-09-19T11:24:21.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f890d0dc640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87f403d000 0x7f87f403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b50085e10 0x556b501858c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50185e00 con 0x556b50085e10 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f890e0de640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b50085e10 0x556b501858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f890d0dc640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87f403d000 0x7f87f403f4c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8904057000 tx=0x7f8904052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f890e0de640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b50085e10 0x556b501858c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f890e0de640 1 -- 172.21.15.28:0/3063424222 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f890800a4d0 con 0x556b50085e10 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.755+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501861a0 con 0x556b50085e10 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.756+0000 7f890e0de640 1 -- 172.21.15.28:0/3063424222 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f890800a4d0 con 0x556b50085e10 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.756+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b50187c60 0x556b5018a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.756+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b5018a890 con 0x556b50187c60 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.756+0000 7f890d8dd640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b50187c60 0x556b5018a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.756+0000 7f890d8dd640 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b50187c60 0x556b5018a100 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.756+0000 7f890d8dd640 1 -- 172.21.15.28:0/3063424222 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f89000187b0 con 0x556b50187c60 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.756+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501869f0 con 0x556b50085e10 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.757+0000 7f890e0de640 1 -- 172.21.15.28:0/3063424222 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f890800a4d0 con 0x556b50085e10 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.757+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50186d90 con 0x556b50187c60 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.757+0000 7f890d8dd640 1 -- 172.21.15.28:0/3063424222 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f89000187b0 con 0x556b50187c60 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.757+0000 7f8911b2e800 1 Processor -- start 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.757+0000 7f8911b2e800 1 -- start start 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.757+0000 7f8911b2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50199210 0x556b50199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.757+0000 7f8911b2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556b4ff7b4a0 con 0x556b50199210 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f890e0de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50199210 0x556b50199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f890e0de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50199210 0x556b50199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56664/0 (socket says 172.21.15.28:56664) 2024-09-19T11:24:21.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f890e0de640 1 -- 172.21.15.28:0/2088901266 learned_addr learned my addr 172.21.15.28:0/2088901266 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f890e0de640 1 -- 172.21.15.28:0/2088901266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b50199c70 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f890e0de640 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50199210 0x556b50199630 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x556b5017f100 tx=0x7f89080167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f87f37fe640 1 -- 172.21.15.28:0/2088901266 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8908013640 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f87f37fe640 1 -- 172.21.15.28:0/2088901266 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89080137e0 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b5019a4a0 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f87f37fe640 1 -- 172.21.15.28:0/2088901266 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8908013ae0 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.758+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b5019aa40 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.759+0000 7f87f37fe640 1 -- 172.21.15.28:0/2088901266 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8908011040 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.759+0000 7f87f37fe640 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87e803d000 0x7f87e803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.759+0000 7f87f37fe640 1 -- 172.21.15.28:0/2088901266 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f890800e040 con 0x556b50199210 2024-09-19T11:24:21.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f890d8dd640 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87e803d000 0x7f87e803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f8911b2e800 1 Processor -- start 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f8911b2e800 1 -- start start 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f890d8dd640 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87e803d000 0x7f87e803f4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f890000c720 tx=0x7f8900052920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f8911b2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f8911b2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556b500838b0 con 0x556b501a5cf0 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f890e0de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f890e0de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56676/0 (socket says 172.21.15.28:56676) 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f890e0de640 1 -- 172.21.15.28:0/1327858210 learned_addr learned my addr 172.21.15.28:0/1327858210 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.760+0000 7f890e0de640 1 -- 172.21.15.28:0/1327858210 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b501a66e0 con 0x556b501a5cf0 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f890e0de640 1 --2- 172.21.15.28:0/1327858210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501a60f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8908054860 tx=0x7f890801f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7d4cd8188e23c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f87f17fa640 1 -- 172.21.15.28:0/1327858210 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8908044030 con 0x556b501a5cf0 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f87f17fa640 1 -- 172.21.15.28:0/1327858210 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8908021e20 con 0x556b501a5cf0 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f87f17fa640 1 -- 172.21.15.28:0/1327858210 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f890804de60 con 0x556b501a5cf0 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 -- 172.21.15.28:0/1327858210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 msgr2=0x556b501a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 --2- 172.21.15.28:0/1327858210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501a60f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8908054860 tx=0x7f890801f400 comp rx=0 tx=0).stop 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 -- 172.21.15.28:0/1327858210 shutdown_connections 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 --2- 172.21.15.28:0/1327858210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501a60f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 -- 172.21.15.28:0/1327858210 >> 172.21.15.28:0/1327858210 conn(0x556b501a20f0 msgr2=0x556b501a0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 -- 172.21.15.28:0/1327858210 shutdown_connections 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 -- 172.21.15.28:0/1327858210 wait complete. 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.761+0000 7f8911b2e800 1 Processor -- start 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f8911b2e800 1 -- start start 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f8911b2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f8911b2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556b501a6d00 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f890e0de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f890e0de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:56682/0 (socket says 172.21.15.28:56682) 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 learned_addr learned my addr 172.21.15.28:0/2330362174 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b501ad820 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f890e0de640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501ad2c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8908016cd0 tx=0x7f89080547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f87df7fe640 1 -- 172.21.15.28:0/2330362174 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8908020bd0 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f87df7fe640 1 -- 172.21.15.28:0/2330362174 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8908020d70 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f87df7fe640 1 -- 172.21.15.28:0/2330362174 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f890801fa90 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.762+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556b501adab0 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.763+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b501ae640 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.763+0000 7f87df7fe640 1 -- 172.21.15.28:0/2330362174 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f890801fc50 con 0x556b501a5cf0 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.763+0000 7f87df7fe640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87d803cfb0 0x7f87d803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.764+0000 7f87df7fe640 1 -- 172.21.15.28:0/2330362174 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f890804c030 con 0x556b501a5cf0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.764+0000 7f890d8dd640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87d803cfb0 0x7f87d803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.764+0000 7f890d8dd640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87d803cfb0 0x7f87d803f470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f890001af70 tx=0x7f8900046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.764+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.764+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b501b8bf0 0x556b501bb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.764+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501bb800 con 0x556b501b8bf0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.764+0000 7f890d0dc640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b501b8bf0 0x556b501bb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f890d0dc640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b501b8bf0 0x556b501bb0b0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 realm 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501bbc30 con 0x556b501b8bf0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501bc500 con 0x556b501b8bf0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501bc8a0 con 0x556b501b8bf0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.765+0000 7f8911b2e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b501bd600 0x556b501bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501c0000 con 0x556b501bd600 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f890e0de640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b501bd600 0x556b501bf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f890e0de640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b501bd600 0x556b501bf9f0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f8911b2e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f8911b2e800 20 searching for the correct realm 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f890e0de640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8908056070 0x7f89080539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.766+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.767+0000 7f890d8dd640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8908056070 0x7f89080539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.767+0000 7f890d8dd640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8908056070 0x7f89080539b0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.767+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.767+0000 7f890d8dd640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8900017050 0x7f890004b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.767+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8900017050 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.767+0000 7f890d0dc640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8900017050 0x7f890004b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.768+0000 7f890d0dc640 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8900017050 0x7f890004b7f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.768+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.768+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.768+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.768+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.768+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.768+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8900017050 2024-09-19T11:24:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8900017050 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.769+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8900017050 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.770+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.771+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.772+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.772+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.772+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.772+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8900017050 2024-09-19T11:24:21.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.772+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.772+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.772+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.773+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.773+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.773+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.773+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.773+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8900017050 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.773+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.773+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8900017050 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501bd600 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.774+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x556b501b8bf0 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1930 con 0x7f8908056070 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:21.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501c09f0 con 0x556b501b8bf0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.775+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1240 con 0x556b501b8bf0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501c1db0 con 0x556b501bd600 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 zone default found 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 4 Realm: () 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 zonegroup default 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501c2150 con 0x7f8900017050 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501c3ff0 con 0x556b501bd600 2024-09-19T11:24:21.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.776+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501c4390 con 0x556b501bd600 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.777+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.777+0000 7f8911b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.777+0000 7f8911b2e800 20 started sync module instance, tier type = 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.777+0000 7f8911b2e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.777+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c3430 con 0x7f8908056070 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.778+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.778+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93919393888336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c3090 con 0x7f8908056070 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.778+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c5090 con 0x556b501b8bf0 2024-09-19T11:24:21.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.779+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93919393888336] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.779+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.779+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93919393892624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c5480 con 0x556b501b8bf0 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.779+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c5ec0 con 0x556b501bd600 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.780+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93919393892624] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.780+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.780+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93919393896848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c5e80 con 0x556b501bd600 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.780+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c69f0 con 0x7f8908056070 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.782+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93919393896848] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.782+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.782+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93919393899776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c6de0 con 0x7f8908056070 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.782+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c7800 con 0x556b501b8bf0 2024-09-19T11:24:21.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.783+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.783+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93919393899776] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.783+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93919393893776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c7c20 con 0x556b501b8bf0 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.783+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c8620 con 0x7f8900017050 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.784+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93919393893776] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.784+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.784+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93919393903344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c85e0 con 0x7f8900017050 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.784+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c90a0 con 0x7f8900017050 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.785+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 93919393903344] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.785+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.785+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93919393909680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c9280 con 0x7f8900017050 2024-09-19T11:24:21.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.785+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c9ad0 con 0x7f8900017050 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 93919393909680] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93919393911424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501c9b50 con 0x7f8900017050 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 20 add_watcher() i=0 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 20 add_watcher() i=1 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 20 add_watcher() i=2 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 20 add_watcher() i=3 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 20 add_watcher() i=4 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 20 add_watcher() i=5 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.786+0000 7f8911b2e800 20 add_watcher() i=6 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.787+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93919393911424] v33'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.787+0000 7f8911b2e800 20 add_watcher() i=7 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.787+0000 7f8911b2e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:21.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.787+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501cce30 con 0x556b501b8bf0 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.787+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.787+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93919393912848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b501ca0e0 con 0x556b501b8bf0 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.788+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93919393912848] v33'212 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.788+0000 7f87bf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 5 note: GC not initialized 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f87be7fc640 20 reqs_thread_entry: start 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 20 Filter name: none 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f872ffff640 20 reqs_thread_entry: start 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-09-19T11:24:21.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.789+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b501ffcc0 con 0x7f8900017050 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.790+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 bar [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.790+0000 7f8911b2e800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.790+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.790+0000 7f8911b2e800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.790+0000 7f8911b2e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.790+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b50203e30 con 0x7f8908056070 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.791+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(63 bud [create,call,writefull 0~206] v33'1 uv1 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.791+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.791+0000 7f8911b2e800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.791+0000 7f8911b2e800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-09-19T11:24:21.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.791+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 93919394146400 in=394b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50204b20 con 0x7f8900017050 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== watch-notify(notify (1) cookie 93919393903344 notify 141733920799 ret 0) ==== 424+0+0 (crc 0 0 0) 0x7f8904081040 con 0x7f8900017050 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f87f0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920799 cookie 93919393903344 notifier 4733 bl.length()=382 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f87f0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f87f0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(64 notify.5 [notify cookie 93919394146400 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f87f0ff9640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f87d0003ea0 con 0x7f8900017050 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== watch-notify(notify_complete (2) cookie 93919394146400 notify 141733920799 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556b50086490 con 0x7f8900017050 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.792+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b502049c0 con 0x556b501b8bf0 2024-09-19T11:24:21.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 : miss 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 20 rados->read ofs=0 len=0 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 8.19 8:99df436e:root::.bucket.meta.myfoo6%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202090 con 0x556b501b8bf0 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(67 .bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 [call out=48b,stat out=16b,read 0~388 out=388b,getxattrs out=218b] v0'0 uv5 ondisk = 0) ==== 333+0+670 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 20 rados_obj.operate() r=0 bl.length=388 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 info.flags=0x17 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 to cache LRU end 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.793+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 8.19 8:99df436e:root::.bucket.meta.myfoo6%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.9:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~388 in=388b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b5020bc00 con 0x556b501b8bf0 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.794+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 .bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 [delete,create,call,call,writefull 0~388,setxattr (117)] v33'6 uv6 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.794+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 info.flags=0x17 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.794+0000 7f8911b2e800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 to cache LRU end 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.794+0000 7f8911b2e800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.794+0000 7f8911b2e800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9, ofs0, ns] 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.794+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify cookie 93919394146016 in=768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b5020a8d0 con 0x7f8908056070 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== watch-notify(notify (1) cookie 93919393888336 notify 141733920790 ret 0) ==== 798+0+0 (crc 0 0 0) 0x7f8900033040 con 0x7f8908056070 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f87f17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920790 cookie 93919393888336 notifier 4733 bl.length()=756 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f87f17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 info.flags=0x17 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f87f17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 to cache LRU end 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f87f17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(69 notify.0 [notify cookie 93919394146016 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f87f17fa640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f8738003530 con 0x7f8908056070 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== watch-notify(notify_complete (2) cookie 93919394146016 notify 141733920790 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556b5017f670 con 0x7f8908056070 2024-09-19T11:24:21.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(70 notify.0 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): orig_sources= new_sources= 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): potential sources added=[] removed=[] 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): orig_dests= new_dests= 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): potential dests added=[] removed=[] 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.795+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502026e0 con 0x556b501b8bf0 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.796+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(71 bud.buckets [call] v33'7 uv0 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.796+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502026e0 con 0x556b501b8bf0 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.797+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(72 bud.buckets [call] v33'8 uv8 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.797+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:73 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502026e0 con 0x556b501b8bf0 2024-09-19T11:24:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(73 myfoo6 [delete,create,call,writefull 0~166] v33'19 uv19 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-09-19T11:24:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f8911b2e800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-19T11:24:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f8911b2e800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-09-19T11:24:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [notify cookie 93919394146016 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50205150 con 0x7f8908056070 2024-09-19T11:24:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== watch-notify(notify (1) cookie 93919393899776 notify 141733920791 ret 0) ==== 358+0+0 (crc 0 0 0) 0x556b5017eac0 con 0x7f8908056070 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f87f0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920791 cookie 93919393899776 notifier 4733 bl.length()=316 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f87f0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f87f0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f87f0ff9640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:75 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f87d0003ea0 con 0x7f8908056070 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.798+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(74 notify.3 [notify cookie 93919394146016 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.799+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(75 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.799+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== watch-notify(notify_complete (2) cookie 93919394146016 notify 141733920791 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556b5017e7d0 con 0x7f8908056070 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.799+0000 7f8911b2e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 : hit (requested=0x17, cached=0x17) 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.799+0000 7f8911b2e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.799+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 8.19 8:99df436e:root::.bucket.meta.myfoo6%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.9:head [delete,create,call version.check_conds in=122b,call version.inc in=38b,writefull 0~388 in=388b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b50208600 con 0x556b501b8bf0 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(76 .bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 [delete,create,call,call,writefull 0~388,setxattr (117)] v33'9 uv9 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 info.flags=0x17 2024-09-19T11:24:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f8911b2e800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 to cache LRU end 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f8911b2e800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f8911b2e800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9, ofs0, ns] 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:77 7.1f 7:f95f44c2:::notify.0:head [notify cookie 93919394146016 in=768b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b502055d0 con 0x7f8908056070 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== watch-notify(notify (1) cookie 93919393888336 notify 141733920792 ret 0) ==== 798+0+0 (crc 0 0 0) 0x556b5017e7d0 con 0x7f8908056070 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f87f17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920792 cookie 93919393888336 notifier 4733 bl.length()=756 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f87f17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 info.flags=0x17 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(77 notify.0 [notify cookie 93919394146016 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f87f17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 to cache LRU end 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f87f17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.800+0000 7f87f17fa640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f8738003510 con 0x7f8908056070 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(78 notify.0 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== watch-notify(notify_complete (2) cookie 93919394146016 notify 141733920792 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556b50085460 con 0x7f8908056070 2024-09-19T11:24:21.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): orig_sources= new_sources= 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): potential sources added=[] removed=[] 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): orig_dests= new_dests= 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f8911b2e800 20 handle_bi_update(): bucket=:myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]): potential dests added=[] removed=[] 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 4.5 4:a33d4c10:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501bd600 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(79 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 4.2f 4:f73deb55:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501b8bf0 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.801+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(80 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.802+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:81 4.2d 4:b59ac719:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x7f8900017050 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.802+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(81 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.802+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:82 4.a 4:511fdb22:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x7f8908056070 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.803+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(82 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3 [call out=89b] v0'0 uv7 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.803+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:83 4.32 4:4c487552:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.803+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(83 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.803+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:84 4.2a 4:572cf71d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.803+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(84 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5 [call out=89b] v0'0 uv6 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.803+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 4.25 4:a58dc530:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501bd600 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.804+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(85 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.804+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:86 4.20 4:066e08cc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.804+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(86 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7 [call out=89b] v0'0 uv9 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.804+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:87 4.22 4:44a8aa8f:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.804+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(87 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.804+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x7f8900017050 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(88 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [call out=381b] v0'0 uv6 ondisk = 0) ==== 194+0+381 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 10 cls_bucket_list_unordered: got thirteen 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:89 4.c 4:3127bc01:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b50202b90 con 0x556b501b8bf0 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(89 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10 [call out=89b] v0'0 uv7 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 get_obj_state: octx=0x556b50204430 obj=myfoo6:thirteen state=0x556b502031a8 s->prefetch_data=0 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 3.24s0 3:26bc51ef:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b5020a8d0 con 0x7f8900017050 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(90 293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen [call out=24b,getxattrs out=854b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 280+0+894 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 10 manifest: total_size = 8 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.9054116028426185591 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.acl 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.content_type 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.etag 2024-09-19T11:24:21.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.idtag 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.manifest 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.source_zone 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 get_obj_state: octx=0x556b50204430 obj=myfoo6:thirteen state=0x556b502031a8 s->prefetch_data=0 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 : hit (requested=0x17, cached=0x17) 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.805+0000 7f8911b2e800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.806+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:91 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b5020f2d0 con 0x7f8900017050 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.807+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(91 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [stat,call,call] v33'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.807+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 3.24s0 3:26bc51ef:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen:head [cmpxattr user.rgw.idtag (62) op 1 mode 1 in=76b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b5020d2b0 con 0x7f8900017050 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.808+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(92 293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen [cmpxattr (62) op 1 mode 1,setxattr (117),setxattr (33)] v33'4 uv4 ondisk = 0) ==== 280+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.808+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:93 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=323b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b5020ed50 con 0x7f8900017050 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:1 objects processed in :myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9]). Next marker thirteen 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.808+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8908056070 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.809+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(93 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [stat,call,call] v33'8 uv8 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.809+0000 7f87f17fa640 20 handle_completion(): completion ok for obj=thirteen 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.809+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(94 bud [delete,create,call,writefull 0~380] v33'2 uv2 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.809+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.809+0000 7f8911b2e800 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.809+0000 7f8911b2e800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.809+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [notify cookie 93919394146016 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8900017050 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== watch-notify(notify (1) cookie 93919393903344 notify 141733920800 ret 0) ==== 598+0+0 (crc 0 0 0) 0x556b50204f50 con 0x7f8900017050 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f87f0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920800 cookie 93919393903344 notifier 4733 bl.length()=556 2024-09-19T11:24:21.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f87f0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f87f0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(95 notify.5 [notify cookie 93919394146016 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f87f0ff9640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f87d0003ea0 con 0x7f8900017050 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(96 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== watch-notify(notify_complete (2) cookie 93919394146016 notify 141733920800 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556b50208f90 con 0x7f8900017050 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.810+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:97 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x556b501b8bf0 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(97 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v33'4 uv4 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f8911b2e800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f8911b2e800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f8911b2e800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify cookie 93919394146016 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8900017050 2024-09-19T11:24:21.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== watch-notify(notify (1) cookie 93919393903344 notify 141733920801 ret 0) ==== 219+0+0 (crc 0 0 0) 0x556b50208f90 con 0x7f8900017050 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f87f0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920801 cookie 93919393903344 notifier 4733 bl.length()=177 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f87f0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f87f0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(98 notify.5 [notify cookie 93919394146016 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.811+0000 7f87f0ff9640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f87d0003ea0 con 0x7f8900017050 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.812+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(99 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.812+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== watch-notify(notify_complete (2) cookie 93919394146016 notify 141733920801 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556b50202480 con 0x7f8900017050 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.812+0000 7f8911b2e800 10 removing user index: bar 2024-09-19T11:24:21.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.812+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8900017050 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(100 bar [delete] v33'12 uv12 ondisk = 0) ==== 147+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f8911b2e800 10 removing default.rgw.meta+users.uid+bar from cache 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f8911b2e800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 7.9 7:93e5b521:::notify.7:head [notify cookie 93919394146016 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8900017050 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== watch-notify(notify (1) cookie 93919393911424 notify 141733920802 ret 0) ==== 194+0+0 (crc 0 0 0) 0x556b50202480 con 0x7f8900017050 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f87f17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920802 cookie 93919393911424 notifier 4733 bl.length()=152 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f87f17fa640 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f8738003510 con 0x7f8900017050 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== osd_op_reply(101 notify.7 [notify cookie 93919394146016 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(102 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.813+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== watch-notify(notify_complete (2) cookie 93919394146016 notify 141733920802 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556b50201850 con 0x7f8900017050 2024-09-19T11:24:21.833 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"bud","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bud","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-09-19T11:24:21.627791Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:24:21.627767Z","tags":[],"group_ids":[]} 2024-09-19T11:24:21.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.814+0000 7f87bf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:21.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.814+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93919393912848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x556b501b8bf0 2024-09-19T11:24:21.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.815+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 93919393912848] v33'213 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.815+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93919393888336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8908056070 2024-09-19T11:24:21.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.816+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 93919393888336] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.816+0000 7f8911b2e800 20 remove_watcher() i=0 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.816+0000 7f8911b2e800 2 removed watcher, disabling cache 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.816+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93919393892624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x556b501b8bf0 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.817+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 93919393892624] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.817+0000 7f8911b2e800 20 remove_watcher() i=1 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.817+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93919393896848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x556b501bd600 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.818+0000 7f890e0de640 1 -- 172.21.15.28:0/2330362174 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 93919393896848] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8908053020 con 0x556b501bd600 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.818+0000 7f8911b2e800 20 remove_watcher() i=2 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.818+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93919393899776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8908056070 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.819+0000 7f890d8dd640 1 -- 172.21.15.28:0/2330362174 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 93919393899776] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f89000187b0 con 0x7f8908056070 2024-09-19T11:24:21.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.819+0000 7f8911b2e800 20 remove_watcher() i=3 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.819+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93919393893776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x556b501b8bf0 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.820+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 93919393893776] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x556b501b8bf0 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.820+0000 7f8911b2e800 20 remove_watcher() i=4 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.820+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93919393903344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8900017050 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.821+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 93919393903344] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.822+0000 7f8911b2e800 20 remove_watcher() i=5 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.822+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93919393909680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8900017050 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.823+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 93919393909680] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.823+0000 7f8911b2e800 20 remove_watcher() i=6 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.823+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93919393911424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x556b502044b0 con 0x7f8900017050 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 93919393911424] v33'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8904051070 con 0x7f8900017050 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 20 remove_watcher() i=7 2024-09-19T11:24:21.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8900017050 msgr2=0x7f890004b7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f8900017050 0x7f890004b7f0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b501b8bf0 msgr2=0x556b501bb0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b501b8bf0 0x556b501bb0b0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8908056070 msgr2=0x7f89080539b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8908056070 0x7f89080539b0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b501bd600 msgr2=0x556b501bf9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b501bd600 0x556b501bf9f0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87d803cfb0 msgr2=0x7f87d803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87d803cfb0 0x7f87d803f470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f890001af70 tx=0x7f8900046000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 msgr2=0x556b501ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2330362174 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b501a5cf0 0x556b501ad2c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8908016cd0 tx=0x7f89080547c0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 reap_dead start 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f890d0dc640 1 -- 172.21.15.28:0/2330362174 reap_dead start 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 shutdown_connections 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 >> 172.21.15.28:0/2330362174 conn(0x556b501a20f0 msgr2=0x556b501a0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 shutdown_connections 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2330362174 wait complete. 2024-09-19T11:24:21.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87e803d000 msgr2=0x7f87e803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87e803d000 0x7f87e803f4c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f890000c720 tx=0x7f8900052920 comp rx=0 tx=0).stop 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50199210 msgr2=0x556b50199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50199210 0x556b50199630 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x556b5017f100 tx=0x7f89080167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 shutdown_connections 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87e803d000 0x7f87e803f4c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 --2- 172.21.15.28:0/2088901266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50199210 0x556b50199630 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.824+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 >> 172.21.15.28:0/2088901266 conn(0x556b5018d5c0 msgr2=0x556b5018d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 shutdown_connections 2024-09-19T11:24:21.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/2088901266 wait complete. 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b50085e10 msgr2=0x556b501858c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b50085e10 0x556b501858c0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b50187c60 msgr2=0x556b5018a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b50187c60 0x556b5018a100 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87f403d000 msgr2=0x7f87f403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87f403d000 0x7f87f403f4c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8904057000 tx=0x7f8904052000 comp rx=0 tx=0).stop 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 msgr2=0x556b5017e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b5017e270 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f890000c010 tx=0x7f8900000ed0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 shutdown_connections 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x556b50187c60 0x556b5018a100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f87f403d000 0x7f87f403f4c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x556b50085e10 0x556b501858c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 --2- 172.21.15.28:0/3063424222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556b50082cf0 0x556b5017e270 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:21.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 >> 172.21.15.28:0/3063424222 conn(0x556b5007e1d0 msgr2=0x556b5016fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:21.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 shutdown_connections 2024-09-19T11:24:21.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:21.825+0000 7f8911b2e800 1 -- 172.21.15.28:0/3063424222 wait complete. 2024-09-19T11:24:21.845 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-09-19T11:24:21.627791Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:24:21.627767Z', 'tags': [], 'group_ids': []} 2024-09-19T11:24:26.853 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:24:26.853 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-09-19T11:24:26.854 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-09-19T11:24:26.854 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-09-19T11:24:26.854 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-09-19T11:24:26.916 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:26.916 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:26.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.915+0000 7f920ba2f800 1 Processor -- start 2024-09-19T11:24:26.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.915+0000 7f920ba2f800 1 -- start start 2024-09-19T11:24:26.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.915+0000 7f920ba2f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55cdc271bdb0 0x55cdc2721950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.915+0000 7f920ba2f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55cdc271b0f0 con 0x55cdc271aa40 2024-09-19T11:24:26.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.915+0000 7f920ba2f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55cdc25a37d0 con 0x55cdc271bdb0 2024-09-19T11:24:26.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.916+0000 7f9208d1f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55cdc271bdb0 0x55cdc2721950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.916+0000 7f9209520640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55cdc271aa40 0x55cdc271b6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:54812/0 (socket says 172.21.15.28:54812) 2024-09-19T11:24:26.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.916+0000 7f9208d1f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55cdc271bdb0 0x55cdc2721950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46080/0 (socket says 172.21.15.28:46080) 2024-09-19T11:24:26.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.916+0000 7f9209520640 1 -- 172.21.15.28:0/4278478794 learned_addr learned my addr 172.21.15.28:0/4278478794 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:26.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.916+0000 7f9208d1f640 1 -- 172.21.15.28:0/4278478794 >> v1:172.21.15.22:6789/0 conn(0x55cdc271aa40 legacy=0x55cdc271b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:26.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.916+0000 7f9208d1f640 1 -- 172.21.15.28:0/4278478794 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc271c260 con 0x55cdc271bdb0 2024-09-19T11:24:26.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.918+0000 7f9208d1f640 1 --2- 172.21.15.28:0/4278478794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271bdb0 0x55cdc2721950 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f91f400a470 tx=0x7f91f4045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245616848e68f85f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.918+0000 7f9203fff640 1 -- 172.21.15.28:0/4278478794 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91f4053570 con 0x55cdc271bdb0 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f9203fff640 1 -- 172.21.15.28:0/4278478794 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f4058070 con 0x55cdc271bdb0 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f9203fff640 1 -- 172.21.15.28:0/4278478794 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91f4053880 con 0x55cdc271bdb0 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f920ba2f800 1 -- 172.21.15.28:0/4278478794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271bdb0 msgr2=0x55cdc2721950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f920ba2f800 1 --2- 172.21.15.28:0/4278478794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271bdb0 0x55cdc2721950 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f91f400a470 tx=0x7f91f4045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f920ba2f800 1 -- 172.21.15.28:0/4278478794 shutdown_connections 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f920ba2f800 1 --2- 172.21.15.28:0/4278478794 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271bdb0 0x55cdc2721950 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f920ba2f800 1 -- 172.21.15.28:0/4278478794 >> 172.21.15.28:0/4278478794 conn(0x55cdc268ddb0 msgr2=0x55cdc268e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f920ba2f800 1 -- 172.21.15.28:0/4278478794 shutdown_connections 2024-09-19T11:24:26.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.919+0000 7f920ba2f800 1 -- 172.21.15.28:0/4278478794 wait complete. 2024-09-19T11:24:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.936+0000 7f920ba2f800 1 Processor -- start 2024-09-19T11:24:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.936+0000 7f920ba2f800 1 -- start start 2024-09-19T11:24:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.937+0000 7f920ba2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc280a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.937+0000 7f920ba2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cdc25b0dd0 con 0x55cdc271aa40 2024-09-19T11:24:26.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.937+0000 7f9209520640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc280a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.937+0000 7f9209520640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc280a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46086/0 (socket says 172.21.15.28:46086) 2024-09-19T11:24:26.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.937+0000 7f9209520640 1 -- 172.21.15.28:0/763967191 learned_addr learned my addr 172.21.15.28:0/763967191 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:26.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.937+0000 7f9209520640 1 -- 172.21.15.28:0/763967191 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc280abb0 con 0x55cdc271aa40 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.937+0000 7f9209520640 1 --2- 172.21.15.28:0/763967191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc280a5c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f91fc001df0 tx=0x7f91fc00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2ee71d83d490e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f90f7606640 1 -- 172.21.15.28:0/763967191 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91fc012020 con 0x55cdc271aa40 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f90f7606640 1 -- 172.21.15.28:0/763967191 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91fc00e470 con 0x55cdc271aa40 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f90f7606640 1 -- 172.21.15.28:0/763967191 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91fc00e7f0 con 0x55cdc271aa40 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f920ba2f800 1 -- 172.21.15.28:0/763967191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 msgr2=0x55cdc280a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f920ba2f800 1 --2- 172.21.15.28:0/763967191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc280a5c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f91fc001df0 tx=0x7f91fc00b6c0 comp rx=0 tx=0).stop 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f920ba2f800 1 -- 172.21.15.28:0/763967191 shutdown_connections 2024-09-19T11:24:26.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f920ba2f800 1 --2- 172.21.15.28:0/763967191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc280a5c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f920ba2f800 1 -- 172.21.15.28:0/763967191 >> 172.21.15.28:0/763967191 conn(0x55cdc268ddb0 msgr2=0x55cdc2805f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f920ba2f800 1 -- 172.21.15.28:0/763967191 shutdown_connections 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.938+0000 7f920ba2f800 1 -- 172.21.15.28:0/763967191 wait complete. 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.939+0000 7f920ba2f800 1 Processor -- start 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.939+0000 7f920ba2f800 1 -- start start 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.939+0000 7f920ba2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc2812110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.939+0000 7f920ba2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cdc27220a0 con 0x55cdc271aa40 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.939+0000 7f9209520640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc2812110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.939+0000 7f9209520640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc2812110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46088/0 (socket says 172.21.15.28:46088) 2024-09-19T11:24:26.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.939+0000 7f9209520640 1 -- 172.21.15.28:0/1983233316 learned_addr learned my addr 172.21.15.28:0/1983233316 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:26.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.940+0000 7f9209520640 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc2812670 con 0x55cdc271aa40 2024-09-19T11:24:26.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.940+0000 7f9209520640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc2812110 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55cdc280b550 tx=0x7f91fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.940+0000 7f90f5e03640 1 -- 172.21.15.28:0/1983233316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91fc012030 con 0x55cdc271aa40 2024-09-19T11:24:26.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.940+0000 7f90f5e03640 1 -- 172.21.15.28:0/1983233316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91fc001450 con 0x55cdc271aa40 2024-09-19T11:24:26.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.940+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cdc2812900 con 0x55cdc271aa40 2024-09-19T11:24:26.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.940+0000 7f90f5e03640 1 -- 172.21.15.28:0/1983233316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91fc01a7a0 con 0x55cdc271aa40 2024-09-19T11:24:26.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.940+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc2813300 con 0x55cdc271aa40 2024-09-19T11:24:26.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.941+0000 7f90f5e03640 1 -- 172.21.15.28:0/1983233316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f91fc015020 con 0x55cdc271aa40 2024-09-19T11:24:26.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.941+0000 7f90f5e03640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90f003cfb0 0x7f90f003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.941+0000 7f90f5e03640 1 -- 172.21.15.28:0/1983233316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f91fc020030 con 0x55cdc271aa40 2024-09-19T11:24:26.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.941+0000 7f9208d1f640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90f003cfb0 0x7f90f003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.942+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc271bdb0 0x55cdc2819760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.942+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2819ca0 con 0x55cdc271bdb0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.942+0000 7f9209d21640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc271bdb0 0x55cdc2819760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.942+0000 7f9208d1f640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90f003cfb0 0x7f90f003f470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f91f4057000 tx=0x7f91f4052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.942+0000 7f9209d21640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc271bdb0 0x55cdc2819760 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.942+0000 7f9209d21640 1 -- 172.21.15.28:0/1983233316 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc271bdb0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.942+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc281a040 con 0x55cdc271bdb0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.943+0000 7f9209d21640 1 -- 172.21.15.28:0/1983233316 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc271bdb0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.943+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc281bb80 0x55cdc281dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.943+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc281e780 con 0x55cdc281bb80 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.943+0000 7f9209520640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc281bb80 0x55cdc281dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.943+0000 7f9209520640 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc281bb80 0x55cdc281dff0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.943+0000 7f9209520640 1 -- 172.21.15.28:0/1983233316 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f91fc01ab20 con 0x55cdc281bb80 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.943+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc281a890 con 0x55cdc271bdb0 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.944+0000 7f9209d21640 1 -- 172.21.15.28:0/1983233316 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc271bdb0 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.944+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc281acb0 con 0x55cdc281bb80 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.944+0000 7f9209520640 1 -- 172.21.15.28:0/1983233316 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f91fc01ab20 con 0x55cdc281bb80 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.944+0000 7f920ba2f800 1 Processor -- start 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.944+0000 7f920ba2f800 1 -- start start 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.945+0000 7f920ba2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc282d0e0 0x55cdc282d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.945+0000 7f920ba2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cdc260f4a0 con 0x55cdc282d0e0 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.945+0000 7f9209d21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc282d0e0 0x55cdc282d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.945+0000 7f9209d21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc282d0e0 0x55cdc282d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46092/0 (socket says 172.21.15.28:46092) 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.945+0000 7f9209d21640 1 -- 172.21.15.28:0/3129502549 learned_addr learned my addr 172.21.15.28:0/3129502549 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.945+0000 7f9209d21640 1 -- 172.21.15.28:0/3129502549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc282db40 con 0x55cdc282d0e0 2024-09-19T11:24:26.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.945+0000 7f9209d21640 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc282d0e0 0x55cdc282d500 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55cdc28134d0 tx=0x7f92040167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.946+0000 7f90eeffd640 1 -- 172.21.15.28:0/3129502549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9204013640 con 0x55cdc282d0e0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.946+0000 7f90eeffd640 1 -- 172.21.15.28:0/3129502549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92040137e0 con 0x55cdc282d0e0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.946+0000 7f90eeffd640 1 -- 172.21.15.28:0/3129502549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9204013ae0 con 0x55cdc282d0e0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.946+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cdc282e370 con 0x55cdc282d0e0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.946+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc282e910 con 0x55cdc282d0e0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.947+0000 7f90eeffd640 1 -- 172.21.15.28:0/3129502549 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9204011040 con 0x55cdc282d0e0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.947+0000 7f90eeffd640 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90e003d000 0x7f90e003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.947+0000 7f90eeffd640 1 -- 172.21.15.28:0/3129502549 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f920400e040 con 0x55cdc282d0e0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.947+0000 7f9209520640 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90e003d000 0x7f90e003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.947+0000 7f920ba2f800 1 Processor -- start 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.947+0000 7f9209520640 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90e003d000 0x7f90e003f4c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55cdc281eb20 tx=0x7f91fc018f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.947+0000 7f920ba2f800 1 -- start start 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f920ba2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2839f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f920ba2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cdc280afe0 con 0x55cdc2839b60 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f9209d21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2839f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f9209d21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2839f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46096/0 (socket says 172.21.15.28:46096) 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f9209d21640 1 -- 172.21.15.28:0/1102180802 learned_addr learned my addr 172.21.15.28:0/1102180802 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f9209d21640 1 -- 172.21.15.28:0/1102180802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc283a550 con 0x55cdc2839b60 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f9209d21640 1 --2- 172.21.15.28:0/1102180802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2839f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9204016cd0 tx=0x7f9204020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319f3992d27fa0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f90ecff9640 1 -- 172.21.15.28:0/1102180802 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9204051020 con 0x55cdc2839b60 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f90ecff9640 1 -- 172.21.15.28:0/1102180802 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9204021910 con 0x55cdc2839b60 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.948+0000 7f90ecff9640 1 -- 172.21.15.28:0/1102180802 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9204021440 con 0x55cdc2839b60 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 -- 172.21.15.28:0/1102180802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 msgr2=0x55cdc2839f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1102180802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2839f60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9204016cd0 tx=0x7f9204020460 comp rx=0 tx=0).stop 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 -- 172.21.15.28:0/1102180802 shutdown_connections 2024-09-19T11:24:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1102180802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2839f60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 -- 172.21.15.28:0/1102180802 >> 172.21.15.28:0/1102180802 conn(0x55cdc2835f60 msgr2=0x55cdc28341a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 -- 172.21.15.28:0/1102180802 shutdown_connections 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 -- 172.21.15.28:0/1102180802 wait complete. 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 Processor -- start 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 -- start start 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.949+0000 7f920ba2f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2841170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f920ba2f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cdc283ab70 con 0x55cdc2839b60 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f9209d21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2841170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f9209d21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2841170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46106/0 (socket says 172.21.15.28:46106) 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 learned_addr learned my addr 172.21.15.28:0/3127459682 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:26.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc28416d0 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f9209d21640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2841170 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9204022ff0 tx=0x7f9204054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f90deffd640 1 -- 172.21.15.28:0/3127459682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92040532c0 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f90deffd640 1 -- 172.21.15.28:0/3127459682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92040459d0 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cdc2841960 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.950+0000 7f90deffd640 1 -- 172.21.15.28:0/3127459682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9204045cf0 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.951+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc2841960 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.951+0000 7f90deffd640 1 -- 172.21.15.28:0/3127459682 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f9204054940 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.952+0000 7f90deffd640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90d403d000 0x7f90d403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.952+0000 7f90deffd640 1 -- 172.21.15.28:0/3127459682 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f9204034d50 con 0x55cdc2839b60 2024-09-19T11:24:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.952+0000 7f9209520640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90d403d000 0x7f90d403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.952+0000 7f9209520640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90d403d000 0x7f90d403f4c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91fc01a6a0 tx=0x7f91fc03ff90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.952+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.952+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc284c860 0x55cdc284ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.952+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc284f470 con 0x55cdc284c860 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f9208d1f640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc284c860 0x55cdc284ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f9208d1f640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc284c860 0x55cdc284ed20 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f920ba2f800 20 realm 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc284f8a0 con 0x55cdc284c860 2024-09-19T11:24:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f920ba2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.953+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2850170 con 0x55cdc284c860 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2850510 con 0x55cdc284c860 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc2851270 0x55cdc2853660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2853c70 con 0x55cdc2851270 2024-09-19T11:24:26.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f9209d21640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc2851270 0x55cdc2853660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.954+0000 7f9209d21640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc2851270 0x55cdc2853660 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f920ba2f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f920ba2f800 20 searching for the correct realm 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f9209d21640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f92040565e0 0x7f9204056a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.955+0000 7f9209520640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f92040565e0 0x7f9204056a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.956+0000 7f9209520640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f92040565e0 0x7f9204056a20 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.956+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.956+0000 7f9209520640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f91fc017050 0x7f91fc022750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.956+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f91fc017050 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.956+0000 7f9208d1f640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f91fc017050 0x7f91fc022750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.956+0000 7f9208d1f640 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f91fc017050 0x7f91fc022750 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.957+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.957+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.957+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.957+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.957+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.957+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f91fc017050 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.958+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.958+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f91fc017050 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.958+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.958+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.958+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.958+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.959+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.959+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.959+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.959+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f91fc017050 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.959+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.959+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.959+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.960+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.960+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.960+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.960+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.960+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.960+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.960+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.961+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.962+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.962+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f91fc017050 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.962+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.962+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.962+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.962+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.963+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.963+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.963+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.963+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f91fc017050 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.963+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.963+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f91fc017050 2024-09-19T11:24:26.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc2851270 2024-09-19T11:24:26.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.964+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x55cdc284c860 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28555a0 con 0x7f92040565e0 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.965+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2854630 con 0x55cdc284c860 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2854e80 con 0x55cdc284c860 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2855a20 con 0x55cdc2851270 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 zone default found 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 4 Realm: () 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:26.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 zonegroup default 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.966+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2855dc0 con 0x7f91fc017050 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2857bd0 con 0x55cdc2851270 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28580e0 con 0x55cdc2851270 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 20 started sync module instance, tier type = 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.967+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2857070 con 0x7f92040565e0 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.968+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.968+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94342220182624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2856cd0 con 0x7f92040565e0 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.968+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2858f00 con 0x55cdc284c860 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.969+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94342220182624] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.970+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.970+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94342220186944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28592f0 con 0x55cdc284c860 2024-09-19T11:24:26.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.970+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2859d50 con 0x55cdc2851270 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.971+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.971+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94342220191744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2859d50 con 0x55cdc2851270 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.971+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285ad50 con 0x7f92040565e0 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.971+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94342220186944] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.972+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94342220191744] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.973+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.973+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94342220195904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285a140 con 0x7f92040565e0 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.973+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285b760 con 0x55cdc284c860 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.974+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94342220195904] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.974+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.974+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94342220188000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285b760 con 0x55cdc284c860 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.974+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285c010 con 0x7f91fc017050 2024-09-19T11:24:26.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.975+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94342220188000] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.975+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.975+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94342220198576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285bfd0 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.975+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285ca50 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.976+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94342220198576] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.976+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.976+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94342220203328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285ca10 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.976+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285d460 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94342220203328] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94342220205904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285d420 con 0x7f91fc017050 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 20 add_watcher() i=0 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 20 add_watcher() i=1 2024-09-19T11:24:26.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 20 add_watcher() i=2 2024-09-19T11:24:26.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 20 add_watcher() i=3 2024-09-19T11:24:26.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 20 add_watcher() i=4 2024-09-19T11:24:26.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 20 add_watcher() i=5 2024-09-19T11:24:26.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.978+0000 7f920ba2f800 20 add_watcher() i=6 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.979+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94342220205904] v33'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.979+0000 7f920ba2f800 20 add_watcher() i=7 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.979+0000 7f920ba2f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.979+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28608e0 con 0x55cdc284c860 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.979+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.979+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94342220200384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc285de00 con 0x55cdc284c860 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.980+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94342220200384] v33'214 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.980+0000 7f90bf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.980+0000 7f920ba2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.980+0000 7f920ba2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.980+0000 7f920ba2f800 5 note: GC not initialized 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f90be7e4640 20 reqs_thread_entry: start 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 20 Filter name: none 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f902ffff640 20 reqs_thread_entry: start 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28937a0 con 0x7f91fc017050 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-19T11:24:26.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 20 rados->read ofs=0 len=0 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.981+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2891e60 con 0x7f91fc017050 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.982+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.982+0000 7f920ba2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.982+0000 7f920ba2f800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.982+0000 7f920ba2f800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.982+0000 7f920ba2f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.982+0000 7f920ba2f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.982+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2892700 con 0x7f91fc017050 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v33'13 uv13 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f920ba2f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-19T11:24:26.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f920ba2f800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f920ba2f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94342220423760 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28970e0 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== watch-notify(notify (1) cookie 94342220198576 notify 141733920803 ret 0) ==== 598+0+0 (crc 0 0 0) 0x55cdc2892aa0 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f90ecff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920803 cookie 94342220198576 notifier 4740 bl.length()=556 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f90ecff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f90ecff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f90ecff9640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f90cc003ea0 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.983+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(65 notify.5 [notify cookie 94342220423760 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.984+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.984+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== watch-notify(notify_complete (2) cookie 94342220423760 notify 141733920803 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55cdc28930f0 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.984+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc2894bf0 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v33'9 uv9 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f920ba2f800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:24:26.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f920ba2f800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f920ba2f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94342220423680 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28970c0 con 0x55cdc2851270 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== watch-notify(notify (1) cookie 94342220191744 notify 141733920774 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55cdc28416d0 con 0x55cdc2851270 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f90dffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920774 cookie 94342220191744 notifier 4740 bl.length()=177 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f90dffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f90dffff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f90dffff640 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f903c003400 con 0x55cdc2851270 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.985+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 notify.2 [notify cookie 94342220423680 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.986+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.986+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== watch-notify(notify_complete (2) cookie 94342220423680 notify 141733920774 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55cdc283a550 con 0x55cdc2851270 2024-09-19T11:24:26.995 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-09-19T11:24:26.983088Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:24:26.983070Z","tags":[],"group_ids":[]} 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.987+0000 7f90bf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.987+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94342220200384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x55cdc284c860 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.988+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94342220200384] v33'215 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.988+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94342220182624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x7f92040565e0 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.989+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94342220182624] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.989+0000 7f920ba2f800 20 remove_watcher() i=0 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.989+0000 7f920ba2f800 2 removed watcher, disabling cache 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.989+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94342220186944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x55cdc284c860 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.990+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94342220186944] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.990+0000 7f920ba2f800 20 remove_watcher() i=1 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.990+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94342220191744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x55cdc2851270 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.991+0000 7f9209d21640 1 -- 172.21.15.28:0/3127459682 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94342220191744] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f920400a4d0 con 0x55cdc2851270 2024-09-19T11:24:26.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.991+0000 7f920ba2f800 20 remove_watcher() i=2 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.991+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94342220195904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x7f92040565e0 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.992+0000 7f9209520640 1 -- 172.21.15.28:0/3127459682 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94342220195904] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91fc01ab20 con 0x7f92040565e0 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.992+0000 7f920ba2f800 20 remove_watcher() i=3 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.992+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94342220188000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x55cdc284c860 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.993+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94342220188000] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x55cdc284c860 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.993+0000 7f920ba2f800 20 remove_watcher() i=4 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.993+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94342220198576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x7f91fc017050 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.995+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94342220198576] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.995+0000 7f920ba2f800 20 remove_watcher() i=5 2024-09-19T11:24:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.995+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94342220203328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x7f91fc017050 2024-09-19T11:24:27.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.996+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94342220203328] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:27.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.996+0000 7f920ba2f800 20 remove_watcher() i=6 2024-09-19T11:24:27.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.996+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94342220205904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cdc28915d0 con 0x7f91fc017050 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94342220205904] v33'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f91f4051070 con 0x7f91fc017050 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 20 remove_watcher() i=7 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f91fc017050 msgr2=0x7f91fc022750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f91fc017050 0x7f91fc022750 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc284c860 msgr2=0x55cdc284ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc284c860 0x55cdc284ed20 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f92040565e0 msgr2=0x7f9204056a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f92040565e0 0x7f9204056a20 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc2851270 msgr2=0x55cdc2853660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc2851270 0x55cdc2853660 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90d403d000 msgr2=0x7f90d403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90d403d000 0x7f90d403f4c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91fc01a6a0 tx=0x7f91fc03ff90 comp rx=0 tx=0).stop 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 msgr2=0x55cdc2841170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f9208d1f640 1 -- 172.21.15.28:0/3127459682 reap_dead start 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.997+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2841170 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9204022ff0 tx=0x7f9204054790 comp rx=0 tx=0).stop 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 shutdown_connections 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3127459682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc2839b60 0x55cdc2841170 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 >> 172.21.15.28:0/3127459682 conn(0x55cdc2835f60 msgr2=0x55cdc2834170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 shutdown_connections 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3127459682 wait complete. 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90e003d000 msgr2=0x7f90e003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90e003d000 0x7f90e003f4c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x55cdc281eb20 tx=0x7f91fc018f00 comp rx=0 tx=0).stop 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc282d0e0 msgr2=0x55cdc282d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc282d0e0 0x55cdc282d500 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55cdc28134d0 tx=0x7f92040167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 shutdown_connections 2024-09-19T11:24:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90e003d000 0x7f90e003f4c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 --2- 172.21.15.28:0/3129502549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc282d0e0 0x55cdc282d500 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 >> 172.21.15.28:0/3129502549 conn(0x55cdc2821490 msgr2=0x55cdc2821870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 shutdown_connections 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/3129502549 wait complete. 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc271bdb0 msgr2=0x55cdc2819760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc271bdb0 0x55cdc2819760 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc281bb80 msgr2=0x55cdc281dff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.998+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc281bb80 0x55cdc281dff0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90f003cfb0 msgr2=0x7f90f003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90f003cfb0 0x7f90f003f470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f91f4057000 tx=0x7f91f4052000 comp rx=0 tx=0).stop 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 msgr2=0x55cdc2812110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc2812110 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55cdc280b550 tx=0x7f91fc000bc0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 shutdown_connections 2024-09-19T11:24:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cdc281bb80 0x55cdc281dff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f90f003cfb0 0x7f90f003f470 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cdc271bdb0 0x55cdc2819760 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 --2- 172.21.15.28:0/1983233316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cdc271aa40 0x55cdc2812110 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 >> 172.21.15.28:0/1983233316 conn(0x55cdc268ddb0 msgr2=0x55cdc280bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 shutdown_connections 2024-09-19T11:24:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:26.999+0000 7f920ba2f800 1 -- 172.21.15.28:0/1983233316 wait complete. 2024-09-19T11:24:27.009 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-09-19T11:24:26.983088Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:24:26.983070Z', 'tags': [], 'group_ids': []} 2024-09-19T11:24:27.029 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:24:27.034 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-09-19T11:24:27.035 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-09-19T11:24:27.035 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-09-19T11:24:27.035 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-09-19T11:24:27.083 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:27.083 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:27.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.082+0000 7f4d66119800 1 Processor -- start 2024-09-19T11:24:27.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.082+0000 7f4d66119800 1 -- start start 2024-09-19T11:24:27.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d66119800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5582c27c8ac0 0x5582c27caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d66119800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5582c27cb5e0 con 0x5582c27c4f50 2024-09-19T11:24:27.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d66119800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5582c264f7d0 con 0x5582c27c8ac0 2024-09-19T11:24:27.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d63409640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5582c27c8ac0 0x5582c27caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d63c0a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5582c27c4f50 0x5582c27c7340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:54826/0 (socket says 172.21.15.28:54826) 2024-09-19T11:24:27.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d63c0a640 1 -- 172.21.15.28:0/1788850978 learned_addr learned my addr 172.21.15.28:0/1788850978 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:27.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d63409640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5582c27c8ac0 0x5582c27caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46118/0 (socket says 172.21.15.28:46118) 2024-09-19T11:24:27.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d63409640 1 -- 172.21.15.28:0/1788850978 >> v1:172.21.15.22:6789/0 conn(0x5582c27c4f50 legacy=0x5582c27c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.083+0000 7f4d63409640 1 -- 172.21.15.28:0/1788850978 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c27cc810 con 0x5582c27c8ac0 2024-09-19T11:24:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.085+0000 7f4d63409640 1 --2- 172.21.15.28:0/1788850978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c8ac0 0x5582c27caf60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800a470 tx=0x7f4d58045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f87bffa94a417b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.085+0000 7f4d62c08640 1 -- 172.21.15.28:0/1788850978 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d58053570 con 0x5582c27c8ac0 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.085+0000 7f4d62c08640 1 -- 172.21.15.28:0/1788850978 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d58058070 con 0x5582c27c8ac0 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.085+0000 7f4d62c08640 1 -- 172.21.15.28:0/1788850978 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d58053880 con 0x5582c27c8ac0 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.086+0000 7f4d66119800 1 -- 172.21.15.28:0/1788850978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c8ac0 msgr2=0x5582c27caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.086+0000 7f4d66119800 1 --2- 172.21.15.28:0/1788850978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c8ac0 0x5582c27caf60 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800a470 tx=0x7f4d58045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.086+0000 7f4d66119800 1 -- 172.21.15.28:0/1788850978 shutdown_connections 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.086+0000 7f4d66119800 1 --2- 172.21.15.28:0/1788850978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c8ac0 0x5582c27caf60 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.086+0000 7f4d66119800 1 -- 172.21.15.28:0/1788850978 >> 172.21.15.28:0/1788850978 conn(0x5582c27bea70 msgr2=0x5582c27c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.086+0000 7f4d66119800 1 -- 172.21.15.28:0/1788850978 shutdown_connections 2024-09-19T11:24:27.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.086+0000 7f4d66119800 1 -- 172.21.15.28:0/1788850978 wait complete. 2024-09-19T11:24:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.101+0000 7f4d66119800 1 Processor -- start 2024-09-19T11:24:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.101+0000 7f4d66119800 1 -- start start 2024-09-19T11:24:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.101+0000 7f4d66119800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c27c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.101+0000 7f4d66119800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582c265cdd0 con 0x5582c27c4f50 2024-09-19T11:24:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.101+0000 7f4d63c0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c27c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.101+0000 7f4d63c0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c27c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46122/0 (socket says 172.21.15.28:46122) 2024-09-19T11:24:27.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.101+0000 7f4d63c0a640 1 -- 172.21.15.28:0/647150739 learned_addr learned my addr 172.21.15.28:0/647150739 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.102+0000 7f4d63c0a640 1 -- 172.21.15.28:0/647150739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c27c4cd0 con 0x5582c27c4f50 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.102+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/647150739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c27c46e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4d54001df0 tx=0x7f4d5400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d78a713625ff814 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.102+0000 7f4c51e0b640 1 -- 172.21.15.28:0/647150739 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d54012020 con 0x5582c27c4f50 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.102+0000 7f4c51e0b640 1 -- 172.21.15.28:0/647150739 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5400e470 con 0x5582c27c4f50 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.102+0000 7f4c51e0b640 1 -- 172.21.15.28:0/647150739 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5400e7f0 con 0x5582c27c4f50 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 -- 172.21.15.28:0/647150739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 msgr2=0x5582c27c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 --2- 172.21.15.28:0/647150739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c27c46e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4d54001df0 tx=0x7f4d5400b6c0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 -- 172.21.15.28:0/647150739 shutdown_connections 2024-09-19T11:24:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 --2- 172.21.15.28:0/647150739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c27c46e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 -- 172.21.15.28:0/647150739 >> 172.21.15.28:0/647150739 conn(0x5582c27bea70 msgr2=0x5582c28afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 -- 172.21.15.28:0/647150739 shutdown_connections 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 -- 172.21.15.28:0/647150739 wait complete. 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 Processor -- start 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 -- start start 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c28be030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.103+0000 7f4d66119800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582c27cbab0 con 0x5582c27c4f50 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4d63c0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c28be030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4d63c0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c28be030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46124/0 (socket says 172.21.15.28:46124) 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4d63c0a640 1 -- 172.21.15.28:0/155422730 learned_addr learned my addr 172.21.15.28:0/155422730 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4d63c0a640 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c28be590 con 0x5582c27c4f50 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c28be030 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x5582c27c4210 tx=0x7f4d54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4c4bfff640 1 -- 172.21.15.28:0/155422730 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d54012030 con 0x5582c27c4f50 2024-09-19T11:24:27.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4c4bfff640 1 -- 172.21.15.28:0/155422730 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5401a6d0 con 0x5582c27c4f50 2024-09-19T11:24:27.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5582c28be820 con 0x5582c27c4f50 2024-09-19T11:24:27.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4c4bfff640 1 -- 172.21.15.28:0/155422730 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5401a9d0 con 0x5582c27c4f50 2024-09-19T11:24:27.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.104+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c28bf220 con 0x5582c27c4f50 2024-09-19T11:24:27.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.105+0000 7f4c4bfff640 1 -- 172.21.15.28:0/155422730 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4d54015020 con 0x5582c27c4f50 2024-09-19T11:24:27.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4c4bfff640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c4c03cfb0 0x7f4c4c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4c4bfff640 1 -- 172.21.15.28:0/155422730 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f4d54020030 con 0x5582c27c4f50 2024-09-19T11:24:27.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4d63409640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c4c03cfb0 0x7f4c4c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c27c8ac0 0x5582c28c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28c5bc0 con 0x5582c27c8ac0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4d6440b640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c27c8ac0 0x5582c28c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4d63409640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c4c03cfb0 0x7f4c4c03f470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d58057000 tx=0x7f4d58052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.106+0000 7f4d6440b640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c27c8ac0 0x5582c28c5680 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.107+0000 7f4d6440b640 1 -- 172.21.15.28:0/155422730 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d5c00a4d0 con 0x5582c27c8ac0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.107+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28c5f60 con 0x5582c27c8ac0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.107+0000 7f4d6440b640 1 -- 172.21.15.28:0/155422730 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4d5c00a4d0 con 0x5582c27c8ac0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.107+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28c7aa0 0x5582c28c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.107+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28ca6a0 con 0x5582c28c7aa0 2024-09-19T11:24:27.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.107+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28c7aa0 0x5582c28c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.107+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28c7aa0 0x5582c28c9f10 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.108+0000 7f4d63c0a640 1 -- 172.21.15.28:0/155422730 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4d5401ad50 con 0x5582c28c7aa0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.108+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28c67b0 con 0x5582c27c8ac0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.108+0000 7f4d6440b640 1 -- 172.21.15.28:0/155422730 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4d5c00a4d0 con 0x5582c27c8ac0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.108+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28c6bd0 con 0x5582c28c7aa0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.108+0000 7f4d63c0a640 1 -- 172.21.15.28:0/155422730 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4d5401ad50 con 0x5582c28c7aa0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.108+0000 7f4d66119800 1 Processor -- start 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d66119800 1 -- start start 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d66119800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28d8ff0 0x5582c28d9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d66119800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582c26bb4a0 con 0x5582c28d8ff0 2024-09-19T11:24:27.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d6440b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28d8ff0 0x5582c28d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d6440b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28d8ff0 0x5582c28d9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46140/0 (socket says 172.21.15.28:46140) 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d6440b640 1 -- 172.21.15.28:0/1499485489 learned_addr learned my addr 172.21.15.28:0/1499485489 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d6440b640 1 -- 172.21.15.28:0/1499485489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c28d9a50 con 0x5582c28d8ff0 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.109+0000 7f4d6440b640 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28d8ff0 0x5582c28d9410 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x5582c28bf3f0 tx=0x7f4d5c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.110+0000 7f4c497fa640 1 -- 172.21.15.28:0/1499485489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c013640 con 0x5582c28d8ff0 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.110+0000 7f4c497fa640 1 -- 172.21.15.28:0/1499485489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c0137e0 con 0x5582c28d8ff0 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.110+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5582c28da280 con 0x5582c28d8ff0 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.110+0000 7f4c497fa640 1 -- 172.21.15.28:0/1499485489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c013ae0 con 0x5582c28d8ff0 2024-09-19T11:24:27.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.110+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c28da820 con 0x5582c28d8ff0 2024-09-19T11:24:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4c497fa640 1 -- 172.21.15.28:0/1499485489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4d5c011040 con 0x5582c28d8ff0 2024-09-19T11:24:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4c497fa640 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c3c03d000 0x7f4c3c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4c497fa640 1 -- 172.21.15.28:0/1499485489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f4d5c00e040 con 0x5582c28d8ff0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c3c03d000 0x7f4c3c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4d66119800 1 Processor -- start 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c3c03d000 0x7f4c3c03f4c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d54052a50 tx=0x7f4d54041040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4d66119800 1 -- start start 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.111+0000 7f4d66119800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4d66119800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582c27c3130 con 0x5582c28e5ab0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4d6440b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4d6440b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28e5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46150/0 (socket says 172.21.15.28:46150) 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4d6440b640 1 -- 172.21.15.28:0/1044898406 learned_addr learned my addr 172.21.15.28:0/1044898406 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4d6440b640 1 -- 172.21.15.28:0/1044898406 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c28e64a0 con 0x5582c28e5ab0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4d6440b640 1 --2- 172.21.15.28:0/1044898406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28e5eb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c054860 tx=0x7f4d5c01f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516ec2aeb7a4ec11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4c42ffd640 1 -- 172.21.15.28:0/1044898406 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c044030 con 0x5582c28e5ab0 2024-09-19T11:24:27.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4c42ffd640 1 -- 172.21.15.28:0/1044898406 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c021e20 con 0x5582c28e5ab0 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.112+0000 7f4c42ffd640 1 -- 172.21.15.28:0/1044898406 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c04de60 con 0x5582c28e5ab0 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 -- 172.21.15.28:0/1044898406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 msgr2=0x5582c28e5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 --2- 172.21.15.28:0/1044898406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28e5eb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c054860 tx=0x7f4d5c01f400 comp rx=0 tx=0).stop 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 -- 172.21.15.28:0/1044898406 shutdown_connections 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 --2- 172.21.15.28:0/1044898406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28e5eb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 -- 172.21.15.28:0/1044898406 >> 172.21.15.28:0/1044898406 conn(0x5582c28e1ed0 msgr2=0x5582c28e0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 -- 172.21.15.28:0/1044898406 shutdown_connections 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 -- 172.21.15.28:0/1044898406 wait complete. 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 Processor -- start 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 -- start start 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28ed090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d66119800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5582c28e6ac0 con 0x5582c28e5ab0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.113+0000 7f4d6440b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4d6440b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46160/0 (socket says 172.21.15.28:46160) 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 learned_addr learned my addr 172.21.15.28:0/2990643310 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c28ed5f0 con 0x5582c28e5ab0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4d6440b640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28ed090 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c016cd0 tx=0x7f4d5c0547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4c417fa640 1 -- 172.21.15.28:0/2990643310 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c020bd0 con 0x5582c28e5ab0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4c417fa640 1 -- 172.21.15.28:0/2990643310 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c020d70 con 0x5582c28e5ab0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4c417fa640 1 -- 172.21.15.28:0/2990643310 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c01fa90 con 0x5582c28e5ab0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5582c28ed880 con 0x5582c28e5ab0 2024-09-19T11:24:27.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.114+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c28ee410 con 0x5582c28e5ab0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.115+0000 7f4c417fa640 1 -- 172.21.15.28:0/2990643310 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4d5c01fc50 con 0x5582c28e5ab0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.115+0000 7f4c417fa640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c2c03cfb0 0x7f4c2c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.115+0000 7f4c417fa640 1 -- 172.21.15.28:0/2990643310 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f4d5c04c030 con 0x5582c28e5ab0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.115+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c2c03cfb0 0x7f4c2c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.116+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.116+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c28f89b0 0x5582c28fae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.116+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c2c03cfb0 0x7f4c2c03f470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4d54022a80 tx=0x7f4d54054420 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.116+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28fb5c0 con 0x5582c28f89b0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.116+0000 7f4d63409640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c28f89b0 0x5582c28fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.116+0000 7f4d63409640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c28f89b0 0x5582c28fae70 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 realm 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28fb9f0 con 0x5582c28f89b0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28fc2c0 con 0x5582c28f89b0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28fc660 con 0x5582c28f89b0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28fd3c0 0x5582c28ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c28ffe00 con 0x5582c28fd3c0 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.117+0000 7f4d6440b640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28fd3c0 0x5582c28ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d6440b640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28fd3c0 0x5582c28ff7b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d66119800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d66119800 20 searching for the correct realm 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d6440b640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4d5c056070 0x7f4d5c0539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.118+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4d5c056070 0x7f4d5c0539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.119+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4d5c056070 0x7f4d5c0539b0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.119+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.119+0000 7f4d63c0a640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4d54017050 0x7f4d5404b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.119+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d54017050 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.119+0000 7f4d63409640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4d54017050 0x7f4d5404b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.120+0000 7f4d63409640 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4d54017050 0x7f4d5404b310 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.120+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.120+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.120+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.120+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.120+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.120+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d54017050 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.121+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.121+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d54017050 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.121+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.121+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.121+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.121+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d54017050 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.122+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.123+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.124+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d54017050 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.125+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d54017050 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.126+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.126+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.126+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.126+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d54017050 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.126+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.126+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28fd3c0 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x5582c28f89b0 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.127+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901730 con 0x7f4d5c056070 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c29007c0 con 0x5582c28f89b0 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901010 con 0x5582c28f89b0 2024-09-19T11:24:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901bb0 con 0x5582c28fd3c0 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 zone default found 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 4 Realm: () 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 zonegroup default 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.128+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c2901f70 con 0x7f4d54017050 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c2903da0 con 0x5582c28fd3c0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c2904210 con 0x5582c28fd3c0 2024-09-19T11:24:27.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 20 started sync module instance, tier type = 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.129+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2903240 con 0x7f4d5c056070 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.130+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.130+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94020098340400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2902ea0 con 0x7f4d5c056070 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.130+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2905140 con 0x5582c28f89b0 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.131+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94020098340400] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.132+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.132+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94020098346128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2905560 con 0x5582c28f89b0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.132+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2905f20 con 0x5582c28fd3c0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.133+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.133+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94020098349680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c29063a0 con 0x5582c28fd3c0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.133+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2906ec0 con 0x7f4d5c056070 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.133+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94020098346128] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.134+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.134+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94020098353552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c29072b0 con 0x7f4d5c056070 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.134+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94020098349680] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.134+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2907d00 con 0x5582c28f89b0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.135+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94020098353552] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.135+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.135+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94020098357232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2907cc0 con 0x5582c28f89b0 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.135+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2908680 con 0x7f4d54017050 2024-09-19T11:24:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.136+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.136+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94020098359760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2908aa0 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.136+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2909560 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.137+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94020098357232] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.137+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94020098359760] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.138+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.138+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94020098363440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c29085f0 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.138+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2909e30 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94020098363440] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94020098347184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c29085f0 con 0x7f4d54017050 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 20 add_watcher() i=0 2024-09-19T11:24:27.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 20 add_watcher() i=1 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 20 add_watcher() i=2 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 20 add_watcher() i=3 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 20 add_watcher() i=4 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 20 add_watcher() i=5 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.139+0000 7f4d66119800 20 add_watcher() i=6 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.140+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94020098347184] v33'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.140+0000 7f4d66119800 20 add_watcher() i=7 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.140+0000 7f4d66119800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.140+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c290cde0 con 0x5582c28f89b0 2024-09-19T11:24:27.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.140+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.140+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94020098365696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c290a2e0 con 0x5582c28f89b0 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.141+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94020098365696] v33'216 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.141+0000 7f4c19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 5 note: GC not initialized 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4c18fe1640 20 reqs_thread_entry: start 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 20 Filter name: none 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4b927fc640 20 reqs_thread_entry: start 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-09-19T11:24:27.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5582c293fc10 con 0x7f4d5c056070 2024-09-19T11:24:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(62 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:24:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.142+0000 7f4d66119800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-09-19T11:24:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.143+0000 7f4d66119800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-19T11:24:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.143+0000 7f4d66119800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:24:27.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.143+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c2943e20 con 0x7f4d54017050 2024-09-19T11:24:27.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.144+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 fud [create,call,writefull 0~206] v33'14 uv13 ondisk = -17 ((17) File exists)) ==== 231+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.145+0000 7f4c19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.145+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94020098365696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x5582c28f89b0 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.146+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94020098365696] v33'217 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.146+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94020098340400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x7f4d5c056070 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.147+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94020098340400] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.147+0000 7f4d66119800 20 remove_watcher() i=0 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.147+0000 7f4d66119800 2 removed watcher, disabling cache 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.147+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94020098346128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x5582c28f89b0 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.148+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94020098346128] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.148+0000 7f4d66119800 20 remove_watcher() i=1 2024-09-19T11:24:27.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.148+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94020098349680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x5582c28fd3c0 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.149+0000 7f4d6440b640 1 -- 172.21.15.28:0/2990643310 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94020098349680] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5c053020 con 0x5582c28fd3c0 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.149+0000 7f4d66119800 20 remove_watcher() i=2 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.149+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94020098353552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x7f4d5c056070 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.150+0000 7f4d63c0a640 1 -- 172.21.15.28:0/2990643310 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94020098353552] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d5401ad50 con 0x7f4d5c056070 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.150+0000 7f4d66119800 20 remove_watcher() i=3 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.150+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94020098357232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x5582c28f89b0 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.151+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94020098357232] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x5582c28f89b0 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.151+0000 7f4d66119800 20 remove_watcher() i=4 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.151+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94020098359760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x7f4d54017050 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.153+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94020098359760] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.153+0000 7f4d66119800 20 remove_watcher() i=5 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.153+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94020098363440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x7f4d54017050 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.154+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94020098363440] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.154+0000 7f4d66119800 20 remove_watcher() i=6 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.154+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94020098347184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5582c293dab0 con 0x7f4d54017050 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94020098347184] v33'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d58051070 con 0x7f4d54017050 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 20 remove_watcher() i=7 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4d54017050 msgr2=0x7f4d5404b310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4d54017050 0x7f4d5404b310 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c28f89b0 msgr2=0x5582c28fae70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c28f89b0 0x5582c28fae70 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4d5c056070 msgr2=0x7f4d5c0539b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f4d5c056070 0x7f4d5c0539b0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28fd3c0 msgr2=0x5582c28ff7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.155+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28fd3c0 0x5582c28ff7b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c2c03cfb0 msgr2=0x7f4c2c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c2c03cfb0 0x7f4c2c03f470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4d54022a80 tx=0x7f4d54054420 comp rx=0 tx=0).stop 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 msgr2=0x5582c28ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/2990643310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28e5ab0 0x5582c28ed090 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c016cd0 tx=0x7f4d5c0547c0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 reap_dead start 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d63409640 1 -- 172.21.15.28:0/2990643310 reap_dead start 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 shutdown_connections 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 >> 172.21.15.28:0/2990643310 conn(0x5582c28e1ed0 msgr2=0x5582c28e00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 shutdown_connections 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/2990643310 wait complete. 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c3c03d000 msgr2=0x7f4c3c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c3c03d000 0x7f4c3c03f4c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d54052a50 tx=0x7f4d54041040 comp rx=0 tx=0).stop 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28d8ff0 msgr2=0x5582c28d9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28d8ff0 0x5582c28d9410 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x5582c28bf3f0 tx=0x7f4d5c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 shutdown_connections 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c3c03d000 0x7f4c3c03f4c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/1499485489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c28d8ff0 0x5582c28d9410 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 >> 172.21.15.28:0/1499485489 conn(0x5582c28cd3a0 msgr2=0x5582c28cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 shutdown_connections 2024-09-19T11:24:27.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/1499485489 wait complete. 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c27c8ac0 msgr2=0x5582c28c5680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c27c8ac0 0x5582c28c5680 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28c7aa0 msgr2=0x5582c28c9f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.156+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28c7aa0 0x5582c28c9f10 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c4c03cfb0 msgr2=0x7f4c4c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c4c03cfb0 0x7f4c4c03f470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d58057000 tx=0x7f4d58052000 comp rx=0 tx=0).stop 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 msgr2=0x5582c28be030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c28be030 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x5582c27c4210 tx=0x7f4d54000bc0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 shutdown_connections 2024-09-19T11:24:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5582c28c7aa0 0x5582c28c9f10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f4c4c03cfb0 0x7f4c4c03f470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5582c27c8ac0 0x5582c28c5680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 --2- 172.21.15.28:0/155422730 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5582c27c4f50 0x5582c28be030 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 >> 172.21.15.28:0/155422730 conn(0x5582c27bea70 msgr2=0x5582c28af810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 shutdown_connections 2024-09-19T11:24:27.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.157+0000 7f4d66119800 1 -- 172.21.15.28:0/155422730 wait complete. 2024-09-19T11:24:27.169 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-19T11:24:27.175 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:24:27.175 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-09-19T11:24:27.180 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-09-19T11:24:27.181 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-09-19T11:24:27.181 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-09-19T11:24:27.181 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-09-19T11:24:27.181 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-09-19T11:24:27.228 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:27.228 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.227+0000 7f05ca346800 1 Processor -- start 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.227+0000 7f05ca346800 1 -- start start 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.227+0000 7f05ca346800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5627f93d7ad0 0x5627f93d9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.227+0000 7f05ca346800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5627f93da5f0 con 0x5627f93d3f60 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.228+0000 7f05ca346800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5627f925e7d0 con 0x5627f93d7ad0 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.228+0000 7f05c762f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5627f93d7ad0 0x5627f93d9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.228+0000 7f05c7e30640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5627f93d3f60 0x5627f93d6350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:54842/0 (socket says 172.21.15.28:54842) 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.228+0000 7f05c762f640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5627f93d7ad0 0x5627f93d9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46162/0 (socket says 172.21.15.28:46162) 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.228+0000 7f05c7e30640 1 -- 172.21.15.28:0/543098201 learned_addr learned my addr 172.21.15.28:0/543098201 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.228+0000 7f05c762f640 1 -- 172.21.15.28:0/543098201 >> v1:172.21.15.22:6789/0 conn(0x5627f93d3f60 legacy=0x5627f93d6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.228+0000 7f05c762f640 1 -- 172.21.15.28:0/543098201 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f93db820 con 0x5627f93d7ad0 2024-09-19T11:24:27.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.230+0000 7f05c762f640 1 --2- 172.21.15.28:0/543098201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d7ad0 0x5627f93d9f70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00a470 tx=0x7f05bc045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3852712e77f7e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.230+0000 7f05c6e2e640 1 -- 172.21.15.28:0/543098201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05bc053570 con 0x5627f93d7ad0 2024-09-19T11:24:27.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.230+0000 7f05c6e2e640 1 -- 172.21.15.28:0/543098201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05bc058070 con 0x5627f93d7ad0 2024-09-19T11:24:27.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.230+0000 7f05c6e2e640 1 -- 172.21.15.28:0/543098201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05bc053880 con 0x5627f93d7ad0 2024-09-19T11:24:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.230+0000 7f05ca346800 1 -- 172.21.15.28:0/543098201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d7ad0 msgr2=0x5627f93d9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.230+0000 7f05ca346800 1 --2- 172.21.15.28:0/543098201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d7ad0 0x5627f93d9f70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00a470 tx=0x7f05bc045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.231+0000 7f05ca346800 1 -- 172.21.15.28:0/543098201 shutdown_connections 2024-09-19T11:24:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.231+0000 7f05ca346800 1 --2- 172.21.15.28:0/543098201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d7ad0 0x5627f93d9f70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.231+0000 7f05ca346800 1 -- 172.21.15.28:0/543098201 >> 172.21.15.28:0/543098201 conn(0x5627f93cdaa0 msgr2=0x5627f93cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.231+0000 7f05ca346800 1 -- 172.21.15.28:0/543098201 shutdown_connections 2024-09-19T11:24:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.231+0000 7f05ca346800 1 -- 172.21.15.28:0/543098201 wait complete. 2024-09-19T11:24:27.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05ca346800 1 Processor -- start 2024-09-19T11:24:27.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05ca346800 1 -- start start 2024-09-19T11:24:27.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05ca346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05ca346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5627f926bdd0 con 0x5627f93d3f60 2024-09-19T11:24:27.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05c7e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05c7e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46168/0 (socket says 172.21.15.28:46168) 2024-09-19T11:24:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05c7e30640 1 -- 172.21.15.28:0/1570986684 learned_addr learned my addr 172.21.15.28:0/1570986684 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.247+0000 7f05c7e30640 1 -- 172.21.15.28:0/1570986684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f94c38f0 con 0x5627f93d3f60 2024-09-19T11:24:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.248+0000 7f05c7e30640 1 --2- 172.21.15.28:0/1570986684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94c3300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f05b8001800 tx=0x7f05b8001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f34493ac696bdb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.248+0000 7f04b5e0b640 1 -- 172.21.15.28:0/1570986684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b8004520 con 0x5627f93d3f60 2024-09-19T11:24:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.248+0000 7f04b5e0b640 1 -- 172.21.15.28:0/1570986684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b80046c0 con 0x5627f93d3f60 2024-09-19T11:24:27.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.248+0000 7f04b5e0b640 1 -- 172.21.15.28:0/1570986684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b80049e0 con 0x5627f93d3f60 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 -- 172.21.15.28:0/1570986684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 msgr2=0x5627f94c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 --2- 172.21.15.28:0/1570986684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94c3300 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f05b8001800 tx=0x7f05b8001cc0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 -- 172.21.15.28:0/1570986684 shutdown_connections 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 --2- 172.21.15.28:0/1570986684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94c3300 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 -- 172.21.15.28:0/1570986684 >> 172.21.15.28:0/1570986684 conn(0x5627f93cdaa0 msgr2=0x5627f94bec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 -- 172.21.15.28:0/1570986684 shutdown_connections 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 -- 172.21.15.28:0/1570986684 wait complete. 2024-09-19T11:24:27.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 Processor -- start 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 -- start start 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05ca346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5627f93daac0 con 0x5627f93d3f60 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05c7e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05c7e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46178/0 (socket says 172.21.15.28:46178) 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.249+0000 7f05c7e30640 1 -- 172.21.15.28:0/4186417033 learned_addr learned my addr 172.21.15.28:0/4186417033 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.250+0000 7f05c7e30640 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f94cb3d0 con 0x5627f93d3f60 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.250+0000 7f05c7e30640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94cae70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f05b8000f80 tx=0x7f05b8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.250+0000 7f04affff640 1 -- 172.21.15.28:0/4186417033 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b8022020 con 0x5627f93d3f60 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.250+0000 7f04affff640 1 -- 172.21.15.28:0/4186417033 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b8014890 con 0x5627f93d3f60 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.250+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627f94cb660 con 0x5627f93d3f60 2024-09-19T11:24:27.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.250+0000 7f04affff640 1 -- 172.21.15.28:0/4186417033 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05b8014bb0 con 0x5627f93d3f60 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.250+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f94cbfe0 con 0x5627f93d3f60 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.251+0000 7f04affff640 1 -- 172.21.15.28:0/4186417033 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f05b8025020 con 0x5627f93d3f60 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.251+0000 7f04affff640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04b003d000 0x7f04b003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.251+0000 7f04affff640 1 -- 172.21.15.28:0/4186417033 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f05b8018030 con 0x5627f93d3f60 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.251+0000 7f05c762f640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04b003d000 0x7f04b003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.251+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f93d7ad0 0x5627f94d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.251+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f94d2a00 con 0x5627f93d7ad0 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.252+0000 7f05c8631640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f93d7ad0 0x5627f94d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.252+0000 7f05c762f640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04b003d000 0x7f04b003f4c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05bc057000 tx=0x7f05bc052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.252+0000 7f05c8631640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f93d7ad0 0x5627f94d24c0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.252+0000 7f05c8631640 1 -- 172.21.15.28:0/4186417033 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f93d7ad0 2024-09-19T11:24:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.252+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f94d2da0 con 0x5627f93d7ad0 2024-09-19T11:24:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.252+0000 7f05c8631640 1 -- 172.21.15.28:0/4186417033 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f93d7ad0 2024-09-19T11:24:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f94d48a0 0x5627f94d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05c7e30640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f94d48a0 0x5627f94d6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f94d74b0 con 0x5627f94d48a0 2024-09-19T11:24:27.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05c7e30640 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f94d48a0 0x5627f94d6d20 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05c7e30640 1 -- 172.21.15.28:0/4186417033 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f05b801b070 con 0x5627f94d48a0 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f94d35f0 con 0x5627f93d7ad0 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05c8631640 1 -- 172.21.15.28:0/4186417033 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f93d7ad0 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.253+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f94d3a40 con 0x5627f94d48a0 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.254+0000 7f05c7e30640 1 -- 172.21.15.28:0/4186417033 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f05b801b070 con 0x5627f94d48a0 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.254+0000 7f05ca346800 1 Processor -- start 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.254+0000 7f05ca346800 1 -- start start 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.254+0000 7f05ca346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3700 0x5627f93d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.254+0000 7f05ca346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5627f92ca4a0 con 0x5627f93d3700 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.254+0000 7f05c8631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3700 0x5627f93d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.254+0000 7f05c8631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3700 0x5627f93d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46188/0 (socket says 172.21.15.28:46188) 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f05c8631640 1 -- 172.21.15.28:0/4285252484 learned_addr learned my addr 172.21.15.28:0/4285252484 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f05c8631640 1 -- 172.21.15.28:0/4285252484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f93d3c20 con 0x5627f93d3700 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f05c8631640 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3700 0x5627f93d1f70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5627f94ccda0 tx=0x7f05c000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f04ad7fa640 1 -- 172.21.15.28:0/4285252484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c001c020 con 0x5627f93d3700 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f04ad7fa640 1 -- 172.21.15.28:0/4285252484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0007c90 con 0x5627f93d3700 2024-09-19T11:24:27.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f04ad7fa640 1 -- 172.21.15.28:0/4285252484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c0018440 con 0x5627f93d3700 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627f93d24b0 con 0x5627f93d3700 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.255+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f93d2a00 con 0x5627f93d3700 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.256+0000 7f04ad7fa640 1 -- 172.21.15.28:0/4285252484 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f05c001f020 con 0x5627f93d3700 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.256+0000 7f04ad7fa640 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04a003d050 0x7f04a003f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.256+0000 7f04ad7fa640 1 -- 172.21.15.28:0/4285252484 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f05c0055b40 con 0x5627f93d3700 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.256+0000 7f05c7e30640 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04a003d050 0x7f04a003f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05ca346800 1 Processor -- start 2024-09-19T11:24:27.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05ca346800 1 -- start start 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05c7e30640 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04a003d050 0x7f04a003f510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5627f94d3990 tx=0x7f05b8011460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05ca346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05ca346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5627f94c3d20 con 0x5627f94f4ac0 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05c8631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05c8631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46194/0 (socket says 172.21.15.28:46194) 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05c8631640 1 -- 172.21.15.28:0/3324794165 learned_addr learned my addr 172.21.15.28:0/3324794165 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.257+0000 7f05c8631640 1 -- 172.21.15.28:0/3324794165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f94f54b0 con 0x5627f94f4ac0 2024-09-19T11:24:27.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05c8631640 1 --2- 172.21.15.28:0/3324794165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94f4ec0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f05c0053df0 tx=0x7f05c0015640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec155b7bac00687f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f04a6ffd640 1 -- 172.21.15.28:0/3324794165 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c0016c50 con 0x5627f94f4ac0 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f04a6ffd640 1 -- 172.21.15.28:0/3324794165 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0016df0 con 0x5627f94f4ac0 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f04a6ffd640 1 -- 172.21.15.28:0/3324794165 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c0013450 con 0x5627f94f4ac0 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05ca346800 1 -- 172.21.15.28:0/3324794165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 msgr2=0x5627f94f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05ca346800 1 --2- 172.21.15.28:0/3324794165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94f4ec0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f05c0053df0 tx=0x7f05c0015640 comp rx=0 tx=0).stop 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05ca346800 1 -- 172.21.15.28:0/3324794165 shutdown_connections 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05ca346800 1 --2- 172.21.15.28:0/3324794165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94f4ec0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05ca346800 1 -- 172.21.15.28:0/3324794165 >> 172.21.15.28:0/3324794165 conn(0x5627f94f0ec0 msgr2=0x5627f94ef100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05ca346800 1 -- 172.21.15.28:0/3324794165 shutdown_connections 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.258+0000 7f05ca346800 1 -- 172.21.15.28:0/3324794165 wait complete. 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05ca346800 1 Processor -- start 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05ca346800 1 -- start start 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05ca346800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05ca346800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5627f94f5ad0 con 0x5627f94f4ac0 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05c8631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05c8631640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46198/0 (socket says 172.21.15.28:46198) 2024-09-19T11:24:27.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 learned_addr learned my addr 172.21.15.28:0/1701684518 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.259+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f94fc650 con 0x5627f94f4ac0 2024-09-19T11:24:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.260+0000 7f05c8631640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94fc0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f05c0053d20 tx=0x7f05c0053d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.260+0000 7f04a57fa640 1 -- 172.21.15.28:0/1701684518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c002e020 con 0x5627f94f4ac0 2024-09-19T11:24:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.260+0000 7f04a57fa640 1 -- 172.21.15.28:0/1701684518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c0020ad0 con 0x5627f94f4ac0 2024-09-19T11:24:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.260+0000 7f04a57fa640 1 -- 172.21.15.28:0/1701684518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05c0016c50 con 0x5627f94f4ac0 2024-09-19T11:24:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.260+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5627f94fc8e0 con 0x5627f94f4ac0 2024-09-19T11:24:27.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.260+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f94fd260 con 0x5627f94f4ac0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f04a57fa640 1 -- 172.21.15.28:0/1701684518 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f05c0031020 con 0x5627f94f4ac0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f04a57fa640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f049003d000 0x7f049003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f04a57fa640 1 -- 172.21.15.28:0/1701684518 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f05c0033840 con 0x5627f94f4ac0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f05c7e30640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f049003d000 0x7f049003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f05c7e30640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f049003d000 0x7f049003f4c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05b801d850 tx=0x7f05b8048000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f95079e0 0x5627f9509ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f950a5f0 con 0x5627f95079e0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.261+0000 7f05c762f640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f95079e0 0x5627f9509ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05c762f640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f95079e0 0x5627f9509ea0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 20 realm 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f950aa20 con 0x5627f95079e0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.262+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f950b2f0 con 0x5627f95079e0 2024-09-19T11:24:27.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f950b690 con 0x5627f95079e0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05ca346800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f950c3f0 0x5627f950e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f950edf0 con 0x5627f950c3f0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05c8631640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f950c3f0 0x5627f950e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.263+0000 7f05c8631640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f950c3f0 0x5627f950e7e0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05ca346800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05ca346800 20 searching for the correct realm 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x5627f950c3f0 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05c8631640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f05c003e040 0x7f05c0021aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x7f05c003e040 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05c7e30640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f05c003e040 0x7f05c0021aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.264+0000 7f05c7e30640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f05c003e040 0x7f05c0021aa0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.265+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.265+0000 7f05c7e30640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f05b8027020 0x7f05b8054050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.265+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x7f05b8027020 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.265+0000 7f05c762f640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f05b8027020 0x7f05b8054050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.265+0000 7f05c762f640 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f05b8027020 0x7f05b8054050 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x5627f95079e0 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x5627f95079e0 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x7f05b8027020 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.266+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x7f05b8027020 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x7f05c003e040 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x5627f950c3f0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x5627f95079e0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.267+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x7f05b8027020 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.268+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.268+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x5627f95079e0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.268+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.268+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510720 con 0x7f05c003e040 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.268+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.268+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f950c3f0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f950c3f0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05c003e040 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f950c3f0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f950c3f0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.269+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.270+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f95079e0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.270+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.270+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f95079e0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.270+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.270+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05b8027020 2024-09-19T11:24:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.270+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.270+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f950c3f0 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05c003e040 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f95079e0 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05b8027020 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.271+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f950c3f0 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05b8027020 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05c003e040 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f950c3f0 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.272+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05c003e040 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x5627f95079e0 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7f05b8060060 con 0x7f05c003e040 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:27.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f950f7b0 con 0x5627f95079e0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.273+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510000 con 0x5627f95079e0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510ba0 con 0x5627f950c3f0 2024-09-19T11:24:27.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 zone default found 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 4 Realm: () 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 zonegroup default 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9510fa0 con 0x7f05b8027020 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.274+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9512dd0 con 0x5627f950c3f0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9513290 con 0x5627f950c3f0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05ca346800 20 started sync module instance, tier type = 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05ca346800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.275+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9512270 con 0x7f05c003e040 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.276+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.276+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94729686556096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95136f0 con 0x7f05c003e040 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.276+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9513f40 con 0x5627f95079e0 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.277+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94729686556096] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.277+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.278+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94729686558352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9514040 con 0x5627f95079e0 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.278+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9514b60 con 0x5627f950c3f0 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.279+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94729686558352] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.279+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.279+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94729686566448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9514b20 con 0x5627f950c3f0 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.279+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9515600 con 0x7f05c003e040 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.280+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.280+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94729686569296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95155f0 con 0x7f05c003e040 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.280+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95165f0 con 0x5627f95079e0 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.280+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94729686566448] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.281+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94729686569296] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.281+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.281+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94729686571552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9513a90 con 0x5627f95079e0 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.281+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9516c10 con 0x7f05b8027020 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.282+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94729686571552] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.282+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.282+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94729686573344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9516c10 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.282+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95175d0 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.283+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.283+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94729686577440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95179f0 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.283+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9518450 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.283+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94729686573344] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94729686577440] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94729686574944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9517f30 con 0x7f05b8027020 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 20 add_watcher() i=0 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 20 add_watcher() i=1 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 20 add_watcher() i=2 2024-09-19T11:24:27.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 20 add_watcher() i=3 2024-09-19T11:24:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 20 add_watcher() i=4 2024-09-19T11:24:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 20 add_watcher() i=5 2024-09-19T11:24:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.284+0000 7f05ca346800 20 add_watcher() i=6 2024-09-19T11:24:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.285+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94729686574944] v33'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.285+0000 7f05ca346800 20 add_watcher() i=7 2024-09-19T11:24:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.285+0000 7f05ca346800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.285+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f951b480 con 0x5627f95079e0 2024-09-19T11:24:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.286+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.286+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94729686578496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9518900 con 0x5627f95079e0 2024-09-19T11:24:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.287+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94729686578496] v33'218 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.287+0000 7f047dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.287+0000 7f05ca346800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.287+0000 7f05ca346800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.287+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9523c30 con 0x5627f95079e0 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.289+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.289+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9523c30 con 0x7f05b8027020 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.290+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.290+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9524650 con 0x5627f950c3f0 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.291+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.291+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95246e0 con 0x5627f95079e0 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.292+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'219 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.292+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9524ae0 con 0x7f05c003e040 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.293+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'68 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.293+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9524ca0 con 0x5627f950c3f0 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.294+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'45 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.294+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9524ca0 con 0x5627f95079e0 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.295+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.295+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9524ca0 con 0x7f05b8027020 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.296+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'45 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.296+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525230 con 0x7f05c003e040 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.298+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'27 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.298+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525150 con 0x5627f950c3f0 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.298+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.299+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525030 con 0x7f05c003e040 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.299+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'69 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.299+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95250c0 con 0x7f05b8027020 2024-09-19T11:24:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.301+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.301+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525150 con 0x7f05c003e040 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.302+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.302+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95251e0 con 0x7f05b8027020 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.303+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'52 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.303+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95251e0 con 0x5627f95079e0 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.304+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'220 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.304+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95251e0 con 0x5627f950c3f0 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.305+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'62 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.305+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95252a0 con 0x7f05c003e040 2024-09-19T11:24:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.306+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'70 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.306+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95252a0 con 0x5627f950c3f0 2024-09-19T11:24:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.307+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.307+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525270 con 0x5627f95079e0 2024-09-19T11:24:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.309+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'34 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.309+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525440 con 0x5627f95079e0 2024-09-19T11:24:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.310+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'221 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.310+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525440 con 0x5627f950c3f0 2024-09-19T11:24:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.311+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'63 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.311+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525440 con 0x7f05c003e040 2024-09-19T11:24:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.312+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'36 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.312+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525440 con 0x5627f950c3f0 2024-09-19T11:24:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.313+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.313+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525440 con 0x5627f950c3f0 2024-09-19T11:24:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.314+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.314+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525440 con 0x5627f950c3f0 2024-09-19T11:24:27.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.315+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'22 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.315+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525530 con 0x5627f950c3f0 2024-09-19T11:24:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.316+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'46 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.316+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9525530 con 0x7f05b8027020 2024-09-19T11:24:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.317+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'30 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.317+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95254d0 con 0x5627f950c3f0 2024-09-19T11:24:27.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.318+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.318+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95254d0 con 0x7f05b8027020 2024-09-19T11:24:27.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.319+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'46 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.319+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95254d0 con 0x5627f950c3f0 2024-09-19T11:24:27.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.320+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'64 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.320+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95254d0 con 0x7f05b8027020 2024-09-19T11:24:27.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.321+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'53 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.321+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95254d0 con 0x7f05c003e040 2024-09-19T11:24:27.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f047cfe1640 20 reqs_thread_entry: start 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f05ca346800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f05ca346800 20 Filter name: none 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f03f67fc640 20 reqs_thread_entry: start 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f05ca346800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.322+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9550010 con 0x7f05c003e040 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(94 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:24:27.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:95 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9552940 con 0x5627f95079e0 2024-09-19T11:24:27.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(95 bud.buckets [call out=184b] v0'0 uv8 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 : miss 2024-09-19T11:24:27.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.323+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:96 8.19 8:99df436e:root::.bucket.meta.myfoo6%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9552810 con 0x5627f95079e0 2024-09-19T11:24:27.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(96 .bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 [call out=48b,stat out=16b,read 0~388 out=388b,getxattrs out=212b] v0'0 uv9 ondisk = 0) ==== 333+0+664 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 20 rados_obj.operate() r=0 bl.length=388 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 info.flags=0x17 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 to cache LRU end 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 4.5 4:a33d4c10:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd00 con 0x5627f950c3f0 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(97 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:98 4.2f 4:f73deb55:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd00 con 0x5627f95079e0 2024-09-19T11:24:27.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(98 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.324+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 4.2d 4:b59ac719:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd00 con 0x7f05b8027020 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.325+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(99 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.325+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:100 4.a 4:511fdb22:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd00 con 0x7f05c003e040 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.325+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(100 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3 [call out=89b] v0'0 uv7 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.325+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:101 4.32 4:4c487552:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd40 con 0x5627f95079e0 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.326+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(101 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.326+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:102 4.2a 4:572cf71d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd40 con 0x5627f95079e0 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.326+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(102 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5 [call out=89b] v0'0 uv6 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.326+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:103 4.25 4:a58dc530:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd40 con 0x5627f950c3f0 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.326+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(103 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.326+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:104 4.20 4:066e08cc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd40 con 0x5627f95079e0 2024-09-19T11:24:27.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.327+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(104 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7 [call out=89b] v0'0 uv9 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.327+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 4.22 4:44a8aa8f:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd40 con 0x5627f95079e0 2024-09-19T11:24:27.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.327+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(105 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.327+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:106 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd40 con 0x7f05b8027020 2024-09-19T11:24:27.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.328+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(106 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [call out=349b] v0'0 uv8 ondisk = 0) ==== 194+0+349 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.328+0000 7f05ca346800 10 cls_bucket_list_unordered: got thirteen 2024-09-19T11:24:27.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.328+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:107 4.c 4:3127bc01:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cfd80 con 0x5627f95079e0 2024-09-19T11:24:27.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.328+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(107 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10 [call out=89b] v0'0 uv7 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.328+0000 7f05ca346800 20 get_obj_state: octx=0x7f05b801d550 obj=myfoo6:thirteen state=0x5627f95cddc8 s->prefetch_data=0 2024-09-19T11:24:27.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.328+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:108 3.24s0 3:26bc51ef:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95cf750 con 0x7f05b8027020 2024-09-19T11:24:27.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.329+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(108 293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen [call out=24b,getxattrs out=819b,stat out=16b] v0'0 uv4 ondisk = 0) ==== 280+0+859 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.329+0000 7f05ca346800 10 manifest: total_size = 8 2024-09-19T11:24:27.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.329+0000 7f05ca346800 20 get_obj_state: setting s->obj_tag to _u5iB-d_8ZK660TiUCkN6H13vPfH5bZj 2024-09-19T11:24:27.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.329+0000 7f05ca346800 20 get_obj_state: octx=0x7f05b801d550 obj=myfoo6:thirteen state=0x5627f95cddc8 s->prefetch_data=0 2024-09-19T11:24:27.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.329+0000 7f05ca346800 20 prepare_atomic_modification: state is not atomic. state=0x5627f95cddc8 2024-09-19T11:24:27.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.329+0000 7f05ca346800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.329+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:109 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9556b50 con 0x7f05b8027020 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.330+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(109 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [stat,call,call] v33'9 uv9 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.330+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:110 3.24s0 3:26bc51ef:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9556b50 con 0x7f05b8027020 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(110 293899f1-2738-4b64-a8e1-a18ff039d626.4229.9_thirteen [call] v33'5 uv5 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:111 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95575e0 con 0x7f05b8027020 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05ca346800 20 list_objects_ordered: starting attempt 1 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05ca346800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:112 4.5 4:a33d4c10:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9558f50 con 0x5627f950c3f0 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:113 4.2f 4:f73deb55:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9558f50 con 0x5627f95079e0 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:114 4.2d 4:b59ac719:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f955dd20 con 0x7f05b8027020 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.331+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:115 4.a 4:511fdb22:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9560790 con 0x7f05c003e040 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(112 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(113 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:116 4.32 4:4c487552:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9560790 con 0x5627f95079e0 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(114 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(115 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3 [call out=89b] v0'0 uv7 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:117 4.2a 4:572cf71d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9558f50 con 0x5627f95079e0 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:118 4.25 4:a58dc530:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f955b850 con 0x5627f950c3f0 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:119 4.20 4:066e08cc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95610d0 con 0x5627f95079e0 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:120 4.22 4:44a8aa8f:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95610d0 con 0x5627f95079e0 2024-09-19T11:24:27.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:121 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9566250 con 0x7f05b8027020 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:122 4.c 4:3127bc01:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(116 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(118 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(117 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5 [call out=89b] v0'0 uv6 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(119 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7 [call out=89b] v0'0 uv9 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.332+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(122 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10 [call out=89b] v0'0 uv7 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(120 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(111 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [stat,call,call] v33'10 uv10 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f04a6ffd640 20 handle_completion(): completion ok for obj=thirteen 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(121 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [call out=128b] v0'0 uv10 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-19T11:24:27.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[293899f1-2738-4b64-a8e1-a18ff039d626.4229.9])"; uploads.size()=0; is_truncated=0 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:123 4.5 4:a33d4c10:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f950c3f0 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:124 4.2f 4:f73deb55:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95573e0 con 0x5627f95079e0 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:125 4.2d 4:b59ac719:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9559380 con 0x7f05b8027020 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:126 4.a 4:511fdb22:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x7f05c003e040 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:127 4.32 4:4c487552:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95573e0 con 0x5627f95079e0 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:128 4.2a 4:572cf71d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9559380 con 0x5627f95079e0 2024-09-19T11:24:27.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:129 4.25 4:a58dc530:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f95573e0 con 0x5627f950c3f0 2024-09-19T11:24:27.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:130 4.20 4:066e08cc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9559380 con 0x5627f95079e0 2024-09-19T11:24:27.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:131 4.22 4:44a8aa8f:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9565b00 con 0x5627f95079e0 2024-09-19T11:24:27.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:132 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9559380 con 0x7f05b8027020 2024-09-19T11:24:27.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:133 4.c 4:3127bc01:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f956fb70 con 0x5627f95079e0 2024-09-19T11:24:27.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(125 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(123 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.333+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(132 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [call out=128b] v0'0 uv10 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(129 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(126 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3 [call out=89b] v0'0 uv7 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 41 ==== osd_op_reply(124 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 42 ==== osd_op_reply(127 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 43 ==== osd_op_reply(128 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5 [call out=89b] v0'0 uv6 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 44 ==== osd_op_reply(130 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7 [call out=89b] v0'0 uv9 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 45 ==== osd_op_reply(131 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 46 ==== osd_op_reply(133 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10 [call out=89b] v0'0 uv7 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.334+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:134 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 47 ==== osd_op_reply(134 bud.buckets [call] v33'10 uv10 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05ca346800 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-09-19T11:24:27.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:135 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 48 ==== osd_op_reply(135 myfoo6 [call out=48b,read 0~166 out=166b] v0'0 uv19 ondisk = 0) ==== 192+0+214 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05ca346800 20 rados_obj.operate() r=0 bl.length=166 2024-09-19T11:24:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05ca346800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x11 2024-09-19T11:24:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05ca346800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-19T11:24:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.335+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:136 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.336+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 49 ==== osd_op_reply(136 myfoo6 [call,call,delete] v33'20 uv20 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.336+0000 7f05ca346800 10 removing default.rgw.meta+root+myfoo6 from cache 2024-09-19T11:24:27.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.336+0000 7f05ca346800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-09-19T11:24:27.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.336+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 94729686838832 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x7f05c003e040 2024-09-19T11:24:27.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.336+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== watch-notify(notify (1) cookie 94729686569296 notify 141733920794 ret 0) ==== 192+0+0 (crc 0 0 0) 0x5627f95241a0 con 0x7f05c003e040 2024-09-19T11:24:27.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.337+0000 7f04a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920794 cookie 94729686569296 notifier 4754 bl.length()=150 2024-09-19T11:24:27.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.337+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(137 notify.3 [notify cookie 94729686838832 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.337+0000 7f04a67fc640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f03fc003320 con 0x7f05c003e040 2024-09-19T11:24:27.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.337+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.337+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== watch-notify(notify_complete (2) cookie 94729686838832 notify 141733920794 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f05b8051040 con 0x7f05c003e040 2024-09-19T11:24:27.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.337+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:139 8.19 8:99df436e:root::.bucket.meta.myfoo6%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.9:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 50 ==== osd_op_reply(139 .bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 [call,call,delete] v33'11 uv11 ondisk = 0) ==== 291+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f05ca346800 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 from cache 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f05ca346800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:293899f1-2738-4b64-a8e1-a18ff039d626.4229.9, ofs0, ns] 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:140 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94729686820736 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x7f05c003e040 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== watch-notify(notify (1) cookie 94729686556096 notify 141733920795 ret 0) ==== 249+0+0 (crc 0 0 0) 0x7f05b8051040 con 0x7f05c003e040 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f04a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920795 cookie 94729686556096 notifier 4754 bl.length()=207 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f04a67fc640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:141 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f03fc003320 con 0x7f05c003e040 2024-09-19T11:24:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.338+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(140 notify.0 [notify cookie 94729686820736 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== watch-notify(notify_complete (2) cookie 94729686820736 notify 141733920795 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5627f94cbfe0 con 0x7f05c003e040 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(141 notify.0 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:142 4.5 4:a33d4c10:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f950c3f0 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:143 4.2f 4:f73deb55:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95572e0 con 0x5627f95079e0 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:144 4.2d 4:b59ac719:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9557680 con 0x7f05b8027020 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:145 4.a 4:511fdb22:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9551510 con 0x7f05c003e040 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:146 4.32 4:4c487552:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:147 4.2a 4:572cf71d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f95572e0 con 0x5627f95079e0 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:148 4.25 4:a58dc530:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9557680 con 0x5627f950c3f0 2024-09-19T11:24:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:149 4.20 4:066e08cc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9551510 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:150 4.22 4:44a8aa8f:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.339+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:151 4.37 4:eddad0bd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9551510 con 0x7f05b8027020 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.340+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:152 4.c 4:3127bc01:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c40 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 51 ==== osd_op_reply(149 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.7 [delete] v33'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(148 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.6 [delete] v33'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(142 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.0 [delete] v33'3 uv3 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 52 ==== osd_op_reply(150 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.8 [delete] v33'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(144 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.2 [delete] v33'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 53 ==== osd_op_reply(143 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.1 [delete] v33'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 54 ==== osd_op_reply(152 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.10 [delete] v33'11 uv11 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 55 ==== osd_op_reply(146 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.4 [delete] v33'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 56 ==== osd_op_reply(147 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.5 [delete] v33'7 uv7 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(145 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.3 [delete] v33'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.341+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(151 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.9.9 [delete] v33'11 uv11 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05ca346800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo6/293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 : miss 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05ca346800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:153 6.18 6:1d9967d7:::pubsub..bucket.myfoo6%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x7f05b8027020 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(153 pubsub..bucket.myfoo6/293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05ca346800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05ca346800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo6/293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 info.flags=0x0 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05ca346800 10 adding default.rgw.log++pubsub..bucket.myfoo6/293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 to cache LRU end 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.342+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:154 6.18 6:1d9967d7:::pubsub..bucket.myfoo6%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x7f05b8027020 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.343+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(154 pubsub..bucket.myfoo6/293899f1-2738-4b64-a8e1-a18ff039d626.4229.9 [delete] v33'27 uv21 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.343+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:155 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x5627f95079e0 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.344+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 57 ==== osd_op_reply(155 bud.buckets [call] v33'12 uv12 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.344+0000 7f05ca346800 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.344+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:156 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x5627f95079e0 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.345+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 58 ==== osd_op_reply(156 NX5QOQKC6BH2IDN8HC7A [delete] v33'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.345+0000 7f05ca346800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-09-19T11:24:27.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.345+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:157 7.3 7:c609908c:::notify.5:head [notify cookie 94729686820560 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x7f05b8027020 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.345+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== watch-notify(notify (1) cookie 94729686573344 notify 141733920804 ret 0) ==== 212+0+0 (crc 0 0 0) 0x5627f9551d80 con 0x7f05b8027020 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.345+0000 7f04a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920804 cookie 94729686573344 notifier 4754 bl.length()=170 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.345+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(157 notify.5 [notify cookie 94729686820560 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.345+0000 7f04a67fc640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:158 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f03fc003320 con 0x7f05b8027020 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.346+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== watch-notify(notify_complete (2) cookie 94729686820560 notify 141733920804 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5627f951d540 con 0x7f05b8027020 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.346+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(158 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.346+0000 7f05ca346800 10 removing email index: 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.346+0000 7f05ca346800 10 removing user buckets index 2024-09-19T11:24:27.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.346+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:159 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x5627f95079e0 2024-09-19T11:24:27.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 59 ==== osd_op_reply(159 bud.buckets [delete] v33'13 uv13 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05ca346800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-09-19T11:24:27.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94729686820560 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x7f05b8027020 2024-09-19T11:24:27.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== watch-notify(notify (1) cookie 94729686577440 notify 141733920805 ret 0) ==== 202+0+0 (crc 0 0 0) 0x5627f951d540 con 0x7f05b8027020 2024-09-19T11:24:27.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f04a6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920805 cookie 94729686577440 notifier 4754 bl.length()=160 2024-09-19T11:24:27.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(160 notify.6 [notify cookie 94729686820560 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f04a6ffd640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f048c003d60 con 0x7f05b8027020 2024-09-19T11:24:27.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== watch-notify(notify_complete (2) cookie 94729686820560 notify 141733920805 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5627f951d370 con 0x7f05b8027020 2024-09-19T11:24:27.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05ca346800 10 removing user index: bud 2024-09-19T11:24:27.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.347+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:162 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x7f05c003e040 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.348+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== osd_op_reply(162 bud [call,call,delete] v33'3 uv3 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.348+0000 7f05ca346800 10 removing default.rgw.meta+users.uid+bud from cache 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.348+0000 7f05ca346800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.348+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94729686820560 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x7f05b8027020 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.349+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== watch-notify(notify (1) cookie 94729686573344 notify 141733920806 ret 0) ==== 194+0+0 (crc 0 0 0) 0x5627f951d370 con 0x7f05b8027020 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.349+0000 7f04a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920806 cookie 94729686573344 notifier 4754 bl.length()=152 2024-09-19T11:24:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.349+0000 7f04a67fc640 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f03fc0032c0 con 0x7f05b8027020 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.349+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== osd_op_reply(163 notify.5 [notify cookie 94729686820560 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.349+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 46 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.349+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 47 ==== watch-notify(notify_complete (2) cookie 94729686820560 notify 141733920806 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5627f9523fd0 con 0x7f05b8027020 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.350+0000 7f047dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.350+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94729686578496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x5627f95079e0 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.351+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 60 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 94729686578496] v33'222 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.351+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94729686556096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x7f05c003e040 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.352+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 94729686556096] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.352+0000 7f05ca346800 20 remove_watcher() i=0 2024-09-19T11:24:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.352+0000 7f05ca346800 2 removed watcher, disabling cache 2024-09-19T11:24:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.352+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94729686558352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x5627f95079e0 2024-09-19T11:24:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.353+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 61 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 94729686558352] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.353+0000 7f05ca346800 20 remove_watcher() i=1 2024-09-19T11:24:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.353+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94729686566448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9568c00 con 0x5627f950c3f0 2024-09-19T11:24:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.354+0000 7f05c8631640 1 -- 172.21.15.28:0/1701684518 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 94729686566448] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05c00083e0 con 0x5627f950c3f0 2024-09-19T11:24:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.354+0000 7f05ca346800 20 remove_watcher() i=2 2024-09-19T11:24:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.354+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94729686569296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9514630 con 0x7f05c003e040 2024-09-19T11:24:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.355+0000 7f05c7e30640 1 -- 172.21.15.28:0/1701684518 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 94729686569296] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05b801b070 con 0x7f05c003e040 2024-09-19T11:24:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.355+0000 7f05ca346800 20 remove_watcher() i=3 2024-09-19T11:24:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.355+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94729686571552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9515150 con 0x5627f95079e0 2024-09-19T11:24:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.356+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.1 v2:172.21.15.28:6816/656119574 62 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 94729686571552] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x5627f95079e0 2024-09-19T11:24:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.356+0000 7f05ca346800 20 remove_watcher() i=4 2024-09-19T11:24:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.356+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94729686573344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9514630 con 0x7f05b8027020 2024-09-19T11:24:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.358+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 48 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 94729686573344] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.358+0000 7f05ca346800 20 remove_watcher() i=5 2024-09-19T11:24:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.358+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94729686577440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9516120 con 0x7f05b8027020 2024-09-19T11:24:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.359+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 49 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 94729686577440] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.359+0000 7f05ca346800 20 remove_watcher() i=6 2024-09-19T11:24:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.359+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94729686574944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5627f9514630 con 0x7f05b8027020 2024-09-19T11:24:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.359+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 <== osd.0 v2:172.21.15.22:6800/3144434255 50 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 94729686574944] v33'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f05bc051070 con 0x7f05b8027020 2024-09-19T11:24:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 20 remove_watcher() i=7 2024-09-19T11:24:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f05b8027020 msgr2=0x7f05b8054050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f05b8027020 0x7f05b8054050 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f95079e0 msgr2=0x5627f9509ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f95079e0 0x5627f9509ea0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f05c003e040 msgr2=0x7f05c0021aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f05c003e040 0x7f05c0021aa0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f950c3f0 msgr2=0x5627f950e7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f950c3f0 0x5627f950e7e0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f049003d000 msgr2=0x7f049003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f049003d000 0x7f049003f4c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f05b801d850 tx=0x7f05b8048000 comp rx=0 tx=0).stop 2024-09-19T11:24:27.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 msgr2=0x5627f94fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94fc0f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f05c0053d20 tx=0x7f05c0053d50 comp rx=0 tx=0).stop 2024-09-19T11:24:27.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05c762f640 1 -- 172.21.15.28:0/1701684518 reap_dead start 2024-09-19T11:24:27.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 shutdown_connections 2024-09-19T11:24:27.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/1701684518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f94f4ac0 0x5627f94fc0f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 >> 172.21.15.28:0/1701684518 conn(0x5627f94f0ec0 msgr2=0x5627f94ef0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 shutdown_connections 2024-09-19T11:24:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/1701684518 wait complete. 2024-09-19T11:24:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04a003d050 msgr2=0x7f04a003f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04a003d050 0x7f04a003f510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5627f94d3990 tx=0x7f05b8011460 comp rx=0 tx=0).stop 2024-09-19T11:24:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3700 msgr2=0x5627f93d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3700 0x5627f93d1f70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5627f94ccda0 tx=0x7f05c000bf30 comp rx=0 tx=0).stop 2024-09-19T11:24:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 shutdown_connections 2024-09-19T11:24:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04a003d050 0x7f04a003f510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 --2- 172.21.15.28:0/4285252484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3700 0x5627f93d1f70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.360+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 >> 172.21.15.28:0/4285252484 conn(0x5627f94da220 msgr2=0x5627f94da600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 shutdown_connections 2024-09-19T11:24:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4285252484 wait complete. 2024-09-19T11:24:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f93d7ad0 msgr2=0x5627f94d24c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f93d7ad0 0x5627f94d24c0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f94d48a0 msgr2=0x5627f94d6d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f94d48a0 0x5627f94d6d20 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04b003d000 msgr2=0x7f04b003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04b003d000 0x7f04b003f4c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05bc057000 tx=0x7f05bc052000 comp rx=0 tx=0).stop 2024-09-19T11:24:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 msgr2=0x5627f94cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94cae70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f05b8000f80 tx=0x7f05b8001250 comp rx=0 tx=0).stop 2024-09-19T11:24:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 shutdown_connections 2024-09-19T11:24:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5627f94d48a0 0x5627f94d6d20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f04b003d000 0x7f04b003f4c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5627f93d7ad0 0x5627f94d24c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 --2- 172.21.15.28:0/4186417033 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5627f93d3f60 0x5627f94cae70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 >> 172.21.15.28:0/4186417033 conn(0x5627f93cdaa0 msgr2=0x5627f94c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 shutdown_connections 2024-09-19T11:24:27.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.361+0000 7f05ca346800 1 -- 172.21.15.28:0/4186417033 wait complete. 2024-09-19T11:24:27.390 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-09-19T11:24:27.390 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-09-19T11:24:27.390 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-09-19T11:24:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:24:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.436+0000 7f6d9d775800 1 Processor -- start 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.436+0000 7f6d9d775800 1 -- start start 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.436+0000 7f6d9d775800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5638fb17aab0 0x5638fb180650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.436+0000 7f6d9d775800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5638fb179d50 con 0x5638fb1796a0 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.436+0000 7f6d9d775800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5638fb0057d0 con 0x5638fb17aab0 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.437+0000 7f6d9aa65640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5638fb17aab0 0x5638fb180650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.437+0000 7f6d9b266640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5638fb1796a0 0x5638fb17a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:54850/0 (socket says 172.21.15.28:54850) 2024-09-19T11:24:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.437+0000 7f6d9b266640 1 -- 172.21.15.28:0/3886427375 learned_addr learned my addr 172.21.15.28:0/3886427375 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:24:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.437+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3886427375 >> v1:172.21.15.22:6789/0 conn(0x5638fb1796a0 legacy=0x5638fb17a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.437+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3886427375 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fb17af60 con 0x5638fb17aab0 2024-09-19T11:24:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.439+0000 7f6d9aa65640 1 --2- 172.21.15.28:0/3886427375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb17aab0 0x5638fb180650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000a470 tx=0x7f6d90045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5feeed817e9b54ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.439+0000 7f6d9a264640 1 -- 172.21.15.28:0/3886427375 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d90053570 con 0x5638fb17aab0 2024-09-19T11:24:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.439+0000 7f6d9a264640 1 -- 172.21.15.28:0/3886427375 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d90058070 con 0x5638fb17aab0 2024-09-19T11:24:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.439+0000 7f6d9a264640 1 -- 172.21.15.28:0/3886427375 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d90053880 con 0x5638fb17aab0 2024-09-19T11:24:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.439+0000 7f6d9d775800 1 -- 172.21.15.28:0/3886427375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb17aab0 msgr2=0x5638fb180650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.439+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3886427375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb17aab0 0x5638fb180650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000a470 tx=0x7f6d90045a20 comp rx=0 tx=0).stop 2024-09-19T11:24:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.440+0000 7f6d9d775800 1 -- 172.21.15.28:0/3886427375 shutdown_connections 2024-09-19T11:24:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.440+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3886427375 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb17aab0 0x5638fb180650 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.440+0000 7f6d9d775800 1 -- 172.21.15.28:0/3886427375 >> 172.21.15.28:0/3886427375 conn(0x5638fb174a70 msgr2=0x5638fb176ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.440+0000 7f6d9d775800 1 -- 172.21.15.28:0/3886427375 shutdown_connections 2024-09-19T11:24:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.440+0000 7f6d9d775800 1 -- 172.21.15.28:0/3886427375 wait complete. 2024-09-19T11:24:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9d775800 1 Processor -- start 2024-09-19T11:24:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9d775800 1 -- start start 2024-09-19T11:24:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9d775800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb0ed7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9d775800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5638fb012dd0 con 0x5638fb1796a0 2024-09-19T11:24:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9b266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb0ed7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9b266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb0ed7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46214/0 (socket says 172.21.15.28:46214) 2024-09-19T11:24:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9b266640 1 -- 172.21.15.28:0/3356116822 learned_addr learned my addr 172.21.15.28:0/3356116822 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.455+0000 7f6d9b266640 1 -- 172.21.15.28:0/3356116822 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fb0edd10 con 0x5638fb1796a0 2024-09-19T11:24:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6d9b266640 1 --2- 172.21.15.28:0/3356116822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb0ed7d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c001e40 tx=0x7f6d8c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffb106097f67ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6c8960a640 1 -- 172.21.15.28:0/3356116822 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d8c012020 con 0x5638fb1796a0 2024-09-19T11:24:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6c8960a640 1 -- 172.21.15.28:0/3356116822 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8c00e5a0 con 0x5638fb1796a0 2024-09-19T11:24:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6c8960a640 1 -- 172.21.15.28:0/3356116822 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d8c00e8a0 con 0x5638fb1796a0 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6d9d775800 1 -- 172.21.15.28:0/3356116822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 msgr2=0x5638fb0ed7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3356116822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb0ed7d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c001e40 tx=0x7f6d8c001e70 comp rx=0 tx=0).stop 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6d9d775800 1 -- 172.21.15.28:0/3356116822 shutdown_connections 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3356116822 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb0ed7d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.456+0000 7f6d9d775800 1 -- 172.21.15.28:0/3356116822 >> 172.21.15.28:0/3356116822 conn(0x5638fb174a70 msgr2=0x5638fb264bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9d775800 1 -- 172.21.15.28:0/3356116822 shutdown_connections 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9d775800 1 -- 172.21.15.28:0/3356116822 wait complete. 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9d775800 1 Processor -- start 2024-09-19T11:24:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9d775800 1 -- start start 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9d775800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb274090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9d775800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5638fb180da0 con 0x5638fb1796a0 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9b266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb274090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9b266640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb274090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46230/0 (socket says 172.21.15.28:46230) 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9b266640 1 -- 172.21.15.28:0/2887646132 learned_addr learned my addr 172.21.15.28:0/2887646132 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.457+0000 7f6d9b266640 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fb2745f0 con 0x5638fb1796a0 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.458+0000 7f6d9b266640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb274090 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c010 tx=0x7f6d8c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.458+0000 7f6c87e07640 1 -- 172.21.15.28:0/2887646132 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d8c012030 con 0x5638fb1796a0 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.458+0000 7f6c87e07640 1 -- 172.21.15.28:0/2887646132 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8c0014a0 con 0x5638fb1796a0 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.458+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5638fb274880 con 0x5638fb1796a0 2024-09-19T11:24:27.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.458+0000 7f6c87e07640 1 -- 172.21.15.28:0/2887646132 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d8c01a7a0 con 0x5638fb1796a0 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.458+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fb275280 con 0x5638fb1796a0 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.459+0000 7f6c87e07640 1 -- 172.21.15.28:0/2887646132 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6d8c015020 con 0x5638fb1796a0 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.459+0000 7f6c87e07640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c8003d000 0x7f6c8003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.459+0000 7f6c87e07640 1 -- 172.21.15.28:0/2887646132 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f6d8c020030 con 0x5638fb1796a0 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.459+0000 7f6d9aa65640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c8003d000 0x7f6c8003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.459+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb17aab0 0x5638fb27b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.459+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb275e10 con 0x5638fb17aab0 2024-09-19T11:24:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.459+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb17aab0 0x5638fb27b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9aa65640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c8003d000 0x7f6c8003f4c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d90057000 tx=0x7f6d90052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb17aab0 0x5638fb27b6e0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9ba67640 1 -- 172.21.15.28:0/2887646132 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb17aab0 2024-09-19T11:24:27.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb275710 con 0x5638fb17aab0 2024-09-19T11:24:27.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9ba67640 1 -- 172.21.15.28:0/2887646132 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb17aab0 2024-09-19T11:24:27.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb27db50 0x5638fb27ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb280610 con 0x5638fb27db50 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.460+0000 7f6d9b266640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb27db50 0x5638fb27ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.461+0000 7f6d9b266640 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb27db50 0x5638fb27ff40 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.461+0000 7f6d9b266640 1 -- 172.21.15.28:0/2887646132 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6d8c0187b0 con 0x5638fb27db50 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.461+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb27c9f0 con 0x5638fb17aab0 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.461+0000 7f6d9ba67640 1 -- 172.21.15.28:0/2887646132 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb17aab0 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.461+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb27ce50 con 0x5638fb27db50 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9b266640 1 -- 172.21.15.28:0/2887646132 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6d8c0187b0 con 0x5638fb27db50 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9d775800 1 Processor -- start 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9d775800 1 -- start start 2024-09-19T11:24:27.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9d775800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb28ef80 0x5638fb28f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9d775800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5638fb0714a0 con 0x5638fb28ef80 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9ba67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb28ef80 0x5638fb28f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9ba67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb28ef80 0x5638fb28f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46232/0 (socket says 172.21.15.28:46232) 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.462+0000 7f6d9ba67640 1 -- 172.21.15.28:0/775722903 learned_addr learned my addr 172.21.15.28:0/775722903 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.463+0000 7f6d9ba67640 1 -- 172.21.15.28:0/775722903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fb28f9e0 con 0x5638fb28ef80 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.463+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb28ef80 0x5638fb28f3a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5638fb274f20 tx=0x7f6d940167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.463+0000 7f6c85602640 1 -- 172.21.15.28:0/775722903 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d94013640 con 0x5638fb28ef80 2024-09-19T11:24:27.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.463+0000 7f6c85602640 1 -- 172.21.15.28:0/775722903 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d940137e0 con 0x5638fb28ef80 2024-09-19T11:24:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.463+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5638fb290210 con 0x5638fb28ef80 2024-09-19T11:24:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.463+0000 7f6c85602640 1 -- 172.21.15.28:0/775722903 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d94013ae0 con 0x5638fb28ef80 2024-09-19T11:24:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.463+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fb2907b0 con 0x5638fb28ef80 2024-09-19T11:24:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.464+0000 7f6c85602640 1 -- 172.21.15.28:0/775722903 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6d94011040 con 0x5638fb28ef80 2024-09-19T11:24:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.464+0000 7f6c85602640 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c7403d050 0x7f6c7403f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.464+0000 7f6c85602640 1 -- 172.21.15.28:0/775722903 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f6d9400e040 con 0x5638fb28ef80 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.464+0000 7f6d9b266640 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c7403d050 0x7f6c7403f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.464+0000 7f6d9d775800 1 Processor -- start 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9d775800 1 -- start start 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9b266640 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c7403d050 0x7f6c7403f510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c720 tx=0x7f6d8c052990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9d775800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb29be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9d775800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5638fb26f400 con 0x5638fb29ba00 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9ba67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb29be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9ba67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb29be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46246/0 (socket says 172.21.15.28:46246) 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9ba67640 1 -- 172.21.15.28:0/4215775922 learned_addr learned my addr 172.21.15.28:0/4215775922 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9ba67640 1 -- 172.21.15.28:0/4215775922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fb29c3f0 con 0x5638fb29ba00 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/4215775922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb29be00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6d940547f0 tx=0x7f6d9401fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fb833b2edcd0e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6c7effd640 1 -- 172.21.15.28:0/4215775922 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d94051020 con 0x5638fb29ba00 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6c7effd640 1 -- 172.21.15.28:0/4215775922 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d940143c0 con 0x5638fb29ba00 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.465+0000 7f6c7effd640 1 -- 172.21.15.28:0/4215775922 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d9401f4b0 con 0x5638fb29ba00 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 -- 172.21.15.28:0/4215775922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 msgr2=0x5638fb29be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 --2- 172.21.15.28:0/4215775922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb29be00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6d940547f0 tx=0x7f6d9401fe70 comp rx=0 tx=0).stop 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 -- 172.21.15.28:0/4215775922 shutdown_connections 2024-09-19T11:24:27.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 --2- 172.21.15.28:0/4215775922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb29be00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 -- 172.21.15.28:0/4215775922 >> 172.21.15.28:0/4215775922 conn(0x5638fb297e00 msgr2=0x5638fb296040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 -- 172.21.15.28:0/4215775922 shutdown_connections 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 -- 172.21.15.28:0/4215775922 wait complete. 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 Processor -- start 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 -- start start 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb2a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.466+0000 7f6d9d775800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5638fb29ca10 con 0x5638fb29ba00 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6d9ba67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb2a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6d9ba67640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb2a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:46252/0 (socket says 172.21.15.28:46252) 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 learned_addr learned my addr 172.21.15.28:0/3340702911 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638fb2a3570 con 0x5638fb29ba00 2024-09-19T11:24:27.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb2a3010 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6d94054750 tx=0x7f6d94054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6c7d7fa640 1 -- 172.21.15.28:0/3340702911 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d94021570 con 0x5638fb29ba00 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6c7d7fa640 1 -- 172.21.15.28:0/3340702911 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d940459c0 con 0x5638fb29ba00 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6c7d7fa640 1 -- 172.21.15.28:0/3340702911 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d94045ce0 con 0x5638fb29ba00 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5638fb2a3800 con 0x5638fb29ba00 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.467+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638fb2a4180 con 0x5638fb29ba00 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.468+0000 7f6c7d7fa640 1 -- 172.21.15.28:0/3340702911 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6d94054930 con 0x5638fb29ba00 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.468+0000 7f6c7d7fa640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c6403d000 0x7f6c6403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.468+0000 7f6c7d7fa640 1 -- 172.21.15.28:0/3340702911 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f6d94017570 con 0x5638fb29ba00 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.468+0000 7f6d9b266640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c6403d000 0x7f6c6403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.469+0000 7f6d9b266640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c6403d000 0x7f6c6403f4c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c018ac0 tx=0x7f6d8c047000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.469+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.469+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb2ae820 0x5638fb2b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.469+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b1430 con 0x5638fb2ae820 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.469+0000 7f6d9aa65640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb2ae820 0x5638fb2b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9aa65640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb2ae820 0x5638fb2b0ce0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 realm 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b1860 con 0x5638fb2ae820 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b2130 con 0x5638fb2ae820 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b24d0 con 0x5638fb2ae820 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb2b3230 0x5638fb2b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.470+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b5c30 con 0x5638fb2b3230 2024-09-19T11:24:27.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb2b3230 0x5638fb2b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb2b3230 0x5638fb2b5620 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9d775800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9d775800 20 searching for the correct realm 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9ba67640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6d9404d2d0 0x7f6d94057650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.471+0000 7f6d9b266640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6d9404d2d0 0x7f6d94057650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.472+0000 7f6d9b266640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6d9404d2d0 0x7f6d94057650 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.472+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.472+0000 7f6d9b266640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6d8c044c20 0x7f6d8c04b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:24:27.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.472+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d8c044c20 2024-09-19T11:24:27.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.472+0000 7f6d9aa65640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6d8c044c20 0x7f6d8c04b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:24:27.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.473+0000 7f6d9aa65640 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6d8c044c20 0x7f6d8c04b860 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:24:27.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.473+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.473+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.473+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.473+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.473+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.473+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d8c044c20 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d8c044c20 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.474+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d8c044c20 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.475+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.476+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d8c044c20 2024-09-19T11:24:27.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.477+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d8c044c20 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.478+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.479+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.479+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d8c044c20 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.479+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.479+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.479+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.479+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2b3230 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x5638fb2ae820 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7560 con 0x7f6d9404d2d0 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:24:27.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.480+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b6620 con 0x5638fb2ae820 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b6e70 con 0x5638fb2ae820 2024-09-19T11:24:27.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b79e0 con 0x5638fb2b3230 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 zone default found 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 4 Realm: () 2024-09-19T11:24:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 zonegroup default 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.481+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b7da0 con 0x7f6d8c044c20 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b9c00 con 0x5638fb2b3230 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ba060 con 0x5638fb2b3230 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 20 started sync module instance, tier type = 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.482+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b9070 con 0x7f6d9404d2d0 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.483+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.483+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94802732094560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2b8cd0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.483+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2baf90 con 0x5638fb2ae820 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.484+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.484+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94802732100800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bb770 con 0x5638fb2ae820 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.484+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bc290 con 0x5638fb2b3230 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.485+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94802732094560] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.486+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.486+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94802732105056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bc250 con 0x5638fb2b3230 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.486+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bcd10 con 0x7f6d9404d2d0 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.486+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94802732100800] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.487+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94802732105056] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.487+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.487+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94802732098752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bb1c0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.487+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bd720 con 0x5638fb2ae820 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.488+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.488+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94802732110448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bdb40 con 0x5638fb2ae820 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.488+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2be610 con 0x7f6d8c044c20 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.488+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94802732098752] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.489+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.489+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94802732114176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2be5d0 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.489+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf090 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.489+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94802732110448] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.490+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94802732114176] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.490+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.490+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94802732107808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2beba0 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.490+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf560 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94802732107808] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94802732111504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf560 con 0x7f6d8c044c20 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 20 add_watcher() i=0 2024-09-19T11:24:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 20 add_watcher() i=1 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 20 add_watcher() i=2 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 20 add_watcher() i=3 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 20 add_watcher() i=4 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 20 add_watcher() i=5 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.491+0000 7f6d9d775800 20 add_watcher() i=6 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.492+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94802732111504] v33'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.492+0000 7f6d9d775800 20 add_watcher() i=7 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.492+0000 7f6d9d775800 2 all 8 watchers are set, enabling cache 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.492+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2c2860 con 0x5638fb2ae820 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.492+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.492+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94802732118560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bfaf0 con 0x5638fb2ae820 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.494+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94802732118560] v33'223 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.494+0000 7f6c51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.494+0000 7f6d9d775800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.494+0000 7f6d9d775800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.494+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cb0d0 con 0x5638fb2ae820 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.495+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.495+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cb760 con 0x7f6d8c044c20 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.496+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.496+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cbcd0 con 0x5638fb2b3230 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.497+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.497+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cbd60 con 0x5638fb2ae820 2024-09-19T11:24:27.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.498+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'224 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.498+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cbd60 con 0x7f6d9404d2d0 2024-09-19T11:24:27.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.499+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'71 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.499+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cc350 con 0x5638fb2b3230 2024-09-19T11:24:27.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.500+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'47 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.500+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cc3c0 con 0x5638fb2ae820 2024-09-19T11:24:27.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.502+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.502+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cc450 con 0x7f6d8c044c20 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.503+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'47 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.503+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cc4e0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.504+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'28 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.504+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccad0 con 0x5638fb2b3230 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.505+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.505+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cca50 con 0x7f6d9404d2d0 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.506+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'72 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.506+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cca20 con 0x7f6d8c044c20 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.507+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.507+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cca20 con 0x7f6d9404d2d0 2024-09-19T11:24:27.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.508+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.508+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cc990 con 0x7f6d8c044c20 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.509+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'54 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.509+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cc990 con 0x5638fb2ae820 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.510+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'225 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.510+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cc990 con 0x5638fb2b3230 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.511+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'65 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.512+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cca80 con 0x7f6d9404d2d0 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.513+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'73 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.513+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cca80 con 0x5638fb2b3230 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.514+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.514+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cca60 con 0x5638fb2ae820 2024-09-19T11:24:27.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.515+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'35 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.515+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2cca60 con 0x5638fb2ae820 2024-09-19T11:24:27.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.516+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'226 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.516+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccaa0 con 0x5638fb2b3230 2024-09-19T11:24:27.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.517+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'66 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.517+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccaa0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.518+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'37 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.518+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccae0 con 0x5638fb2b3230 2024-09-19T11:24:27.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.519+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.519+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccb20 con 0x5638fb2b3230 2024-09-19T11:24:27.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.520+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.520+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccb60 con 0x5638fb2b3230 2024-09-19T11:24:27.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.521+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'23 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.521+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccb60 con 0x5638fb2b3230 2024-09-19T11:24:27.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.522+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'48 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.522+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccba0 con 0x7f6d8c044c20 2024-09-19T11:24:27.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.523+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'31 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.523+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccbe0 con 0x5638fb2b3230 2024-09-19T11:24:27.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.524+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.524+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccbe0 con 0x7f6d8c044c20 2024-09-19T11:24:27.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.525+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'48 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.525+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccbe0 con 0x5638fb2b3230 2024-09-19T11:24:27.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.526+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'67 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.527+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccbe0 con 0x7f6d8c044c20 2024-09-19T11:24:27.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.528+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'55 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.528+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2ccc20 con 0x7f6d9404d2d0 2024-09-19T11:24:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.529+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.529+0000 7f6c50fe1640 20 reqs_thread_entry: start 2024-09-19T11:24:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.529+0000 7f6d9d775800 20 init_complete bucket index max shards: 11 2024-09-19T11:24:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 20 Filter name: none 2024-09-19T11:24:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6bca7fc640 20 reqs_thread_entry: start 2024-09-19T11:24:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2f7b40 con 0x7f6d8c044c20 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 20 rados_obj.operate() r=0 bl.length=380 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.530+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2f9fd0 con 0x7f6d8c044c20 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(95 fud.buckets [call out=186b] v0'0 uv5 ondisk = 0) ==== 155+0+186 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 : miss 2024-09-19T11:24:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:96 8.6 8:6207d7ce:root::.bucket.meta.myfoo7%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.10:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2f9b70 con 0x7f6d8c044c20 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(96 .bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 [call out=48b,stat out=16b,read 0~390 out=390b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 334+0+672 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 20 rados_obj.operate() r=0 bl.length=390 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 info.flags=0x17 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 to cache LRU end 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:97 4.3e 4:7e1d3efd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(97 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.531+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:98 4.1a 4:586bf992:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.532+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(98 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.532+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 4.23 4:c485236c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x7f6d8c044c20 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.532+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(99 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.532+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:100 4.10 4:0a135a43:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x5638fb2b3230 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.532+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(100 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3 [call out=89b] v0'0 uv4 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.532+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:101 4.3a 4:5e7fe3ab:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x7f6d8c044c20 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.533+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(101 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.533+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:102 4.c 4:31a9e5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x5638fb2ae820 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.533+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(102 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5 [call out=382b] v0'0 uv10 ondisk = 0) ==== 195+0+382 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.533+0000 7f6d9d775800 10 cls_bucket_list_unordered: got fourteen 2024-09-19T11:24:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.533+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:103 4.6 4:61eb98f0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x5638fb2b3230 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.534+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(103 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.534+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:104 4.5 4:a36a4c67:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x5638fb2b3230 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.534+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(104 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7 [call out=89b] v0'0 uv2 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.534+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:105 4.3a 4:5fca9668:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3777f0 con 0x7f6d8c044c20 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.534+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(105 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8 [call out=89b] v0'0 uv2 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.534+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:106 4.1 4:80ac83d5:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb377810 con 0x7f6d9404d2d0 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.535+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(106 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.535+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:107 4.27 4:e7dfbb3c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb377810 con 0x5638fb2ae820 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.535+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(107 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10 [call out=89b] v0'0 uv3 ondisk = 0) ==== 196+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.535+0000 7f6d9d775800 20 get_obj_state: octx=0x7f6d8c03c040 obj=myfoo7:fourteen state=0x5638fb2fb2a8 s->prefetch_data=0 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.535+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:108 3.12s0 3:4940d673:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.10_fourteen:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fd160 con 0x7f6d9404d2d0 2024-09-19T11:24:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.536+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(108 293899f1-2738-4b64-a8e1-a18ff039d626.4229.10_fourteen [call out=24b,getxattrs out=858b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 281+0+898 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.536+0000 7f6d9d775800 10 manifest: total_size = 8 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.536+0000 7f6d9d775800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.13816444449074814271 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.536+0000 7f6d9d775800 20 get_obj_state: octx=0x7f6d8c03c040 obj=myfoo7:fourteen state=0x5638fb2fb2a8 s->prefetch_data=0 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.536+0000 7f6d9d775800 20 prepare_atomic_modification: state is not atomic. state=0x5638fb2fb2a8 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.536+0000 7f6d9d775800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.536+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:109 4.c 4:31a9e5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=188b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fc920 con 0x5638fb2ae820 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.537+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(109 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5 [stat,call,call] v33'12 uv12 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.537+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:110 3.12s0 3:4940d673:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.10_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fc920 con 0x7f6d9404d2d0 2024-09-19T11:24:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.538+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(110 293899f1-2738-4b64-a8e1-a18ff039d626.4229.10_fourteen [call] v33'4 uv4 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.538+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:111 4.c 4:31a9e5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=265b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb301730 con 0x5638fb2ae820 2024-09-19T11:24:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.538+0000 7f6d9d775800 20 list_objects_ordered: starting attempt 1 2024-09-19T11:24:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.538+0000 7f6d9d775800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-19T11:24:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.538+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:112 4.3e 4:7e1d3efd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fd0a0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.538+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:113 4.1a 4:586bf992:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3054e0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.538+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:114 4.23 4:c485236c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb307fc0 con 0x7f6d8c044c20 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:115 4.10 4:0a135a43:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb307fc0 con 0x5638fb2b3230 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:116 4.3a 4:5e7fe3ab:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30cec0 con 0x7f6d8c044c20 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:117 4.c 4:31a9e5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30f8a0 con 0x5638fb2ae820 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(112 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(113 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:118 4.6 4:61eb98f0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb303010 con 0x5638fb2b3230 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:119 4.5 4:a36a4c67:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3101e0 con 0x5638fb2b3230 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:120 4.3a 4:5fca9668:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb312e30 con 0x7f6d8c044c20 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:121 4.1 4:80ac83d5:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3157f0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(114 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(115 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3 [call out=89b] v0'0 uv4 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(116 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.539+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(120 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8 [call out=89b] v0'0 uv2 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:122 4.27 4:e7dfbb3c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3101e0 con 0x5638fb2ae820 2024-09-19T11:24:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(111 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5 [stat,call,call] v33'13 uv13 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6c7effd640 20 handle_completion(): completion ok for obj=fourteen 2024-09-19T11:24:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(121 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(117 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5 [call out=128b] v0'0 uv13 ondisk = 0) ==== 195+0+128 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(118 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(119 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7 [call out=89b] v0'0 uv2 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(122 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10 [call out=89b] v0'0 uv3 ondisk = 0) ==== 196+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-19T11:24:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-19T11:24:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-19T11:24:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-19T11:24:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[293899f1-2738-4b64-a8e1-a18ff039d626.4229.10])"; uploads.size()=0; is_truncated=0 2024-09-19T11:24:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:123 4.3e 4:7e1d3efd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30cf20 con 0x7f6d9404d2d0 2024-09-19T11:24:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:124 4.1a 4:586bf992:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb307970 con 0x7f6d9404d2d0 2024-09-19T11:24:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:125 4.23 4:c485236c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3101c0 con 0x7f6d8c044c20 2024-09-19T11:24:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:126 4.10 4:0a135a43:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30cf20 con 0x5638fb2b3230 2024-09-19T11:24:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:127 4.3a 4:5e7fe3ab:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb313730 con 0x7f6d8c044c20 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:128 4.c 4:31a9e5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb307970 con 0x5638fb2ae820 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:129 4.6 4:61eb98f0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30cf20 con 0x5638fb2b3230 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:130 4.5 4:a36a4c67:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3101c0 con 0x5638fb2b3230 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:131 4.3a 4:5fca9668:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3019e0 con 0x7f6d8c044c20 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.540+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:132 4.1 4:80ac83d5:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30cac0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(124 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(123 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(128 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5 [call out=128b] v0'0 uv13 ondisk = 0) ==== 195+0+128 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(126 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3 [call out=89b] v0'0 uv4 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:133 4.27 4:e7dfbb3c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30cac0 con 0x5638fb2ae820 2024-09-19T11:24:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(125 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(127 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(131 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8 [call out=89b] v0'0 uv2 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== osd_op_reply(132 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(130 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7 [call out=89b] v0'0 uv2 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(129 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(133 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10 [call out=89b] v0'0 uv3 ondisk = 0) ==== 196+0+89 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.541+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:134 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30ca40 con 0x7f6d8c044c20 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.542+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(134 fud.buckets [call] v33'6 uv6 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.542+0000 7f6d9d775800 10 cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.542+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.542+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:135 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30ca40 con 0x7f6d9404d2d0 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.543+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(135 myfoo7 [call out=48b,read 0~168 out=168b] v0'0 uv44 ondisk = 0) ==== 192+0+216 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.543+0000 7f6d9d775800 20 rados_obj.operate() r=0 bl.length=168 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.543+0000 7f6d9d775800 10 cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x11 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.543+0000 7f6d9d775800 10 adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-09-19T11:24:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.543+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:136 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb313260 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(136 myfoo7 [call,call,delete] v33'45 uv45 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6d9d775800 10 removing default.rgw.meta+root+myfoo7 from cache 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6d9d775800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 94802732368624 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3127d0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== watch-notify(notify (1) cookie 94802732098752 notify 141733920796 ret 0) ==== 192+0+0 (crc 0 0 0) 0x5638fb2cbdb0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6c7e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920796 cookie 94802732098752 notifier 4761 bl.length()=150 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(137 notify.3 [notify cookie 94802732368624 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6c7e7fc640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f6bd00032c0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.544+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.545+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 36 ==== watch-notify(notify_complete (2) cookie 94802732368624 notify 141733920796 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5638fb2a3800 con 0x7f6d9404d2d0 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.545+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:139 8.6 8:6207d7ce:root::.bucket.meta.myfoo7%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.10:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb313260 con 0x7f6d8c044c20 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(139 .bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 [call,call,delete] v33'4 uv4 ondisk = 0) ==== 292+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6d9d775800 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 from cache 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6d9d775800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:293899f1-2738-4b64-a8e1-a18ff039d626.4229.10, ofs0, ns] 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:140 7.0 7:05bf5b68:::notify.1:head [notify cookie 94802732368624 in=220b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3127d0 con 0x5638fb2ae820 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== watch-notify(notify (1) cookie 94802732100800 notify 141733920789 ret 0) ==== 250+0+0 (crc 0 0 0) 0x5638fb2f9110 con 0x5638fb2ae820 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6c7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920789 cookie 94802732100800 notifier 4761 bl.length()=208 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(140 notify.1 [notify cookie 94802732368624 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.546+0000 7f6c7effd640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:141 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f6c60003d60 con 0x5638fb2ae820 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== watch-notify(notify_complete (2) cookie 94802732368624 notify 141733920789 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5638fb2cbb00 con 0x5638fb2ae820 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(141 notify.1 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:142 4.3e 4:7e1d3efd:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb313260 con 0x7f6d9404d2d0 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:143 4.1a 4:586bf992:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30fd10 con 0x7f6d9404d2d0 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:144 4.23 4:c485236c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30c4b0 con 0x7f6d8c044c20 2024-09-19T11:24:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:145 4.10 4:0a135a43:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30fd10 con 0x5638fb2b3230 2024-09-19T11:24:27.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:146 4.3a 4:5e7fe3ab:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30c4b0 con 0x7f6d8c044c20 2024-09-19T11:24:27.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:147 4.c 4:31a9e5de:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30fd10 con 0x5638fb2ae820 2024-09-19T11:24:27.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:148 4.6 4:61eb98f0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30c4b0 con 0x5638fb2b3230 2024-09-19T11:24:27.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:149 4.5 4:a36a4c67:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30c4b0 con 0x5638fb2b3230 2024-09-19T11:24:27.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:150 4.3a 4:5fca9668:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb3024b0 con 0x7f6d8c044c20 2024-09-19T11:24:27.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:151 4.1 4:80ac83d5:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb315c70 con 0x7f6d9404d2d0 2024-09-19T11:24:27.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.547+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:152 4.27 4:e7dfbb3c:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb30c4b0 con 0x5638fb2ae820 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 37 ==== osd_op_reply(143 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.1 [delete] v33'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== osd_op_reply(146 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.4 [delete] v33'3 uv3 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(145 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.3 [delete] v33'5 uv5 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(150 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.8 [delete] v33'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 38 ==== osd_op_reply(142 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.0 [delete] v33'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 38 ==== osd_op_reply(148 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.6 [delete] v33'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== osd_op_reply(144 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.2 [delete] v33'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 39 ==== osd_op_reply(149 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.7 [delete] v33'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 39 ==== osd_op_reply(151 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.9 [delete] v33'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(152 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.10 [delete] v33'4 uv4 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(147 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.10.5 [delete] v33'14 uv14 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9d775800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 : miss 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9d775800 20 rados->read ofs=0 len=0 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:153 6.17 6:efbff7c5:::pubsub..bucket.myfoo7%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.10:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb315c70 con 0x5638fb2ae820 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 41 ==== osd_op_reply(153 pubsub..bucket.myfoo7/293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9d775800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9d775800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 info.flags=0x0 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.549+0000 7f6d9d775800 10 adding default.rgw.log++pubsub..bucket.myfoo7/293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 to cache LRU end 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.550+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:154 6.17 6:efbff7c5:::pubsub..bucket.myfoo7%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb315c70 con 0x5638fb2ae820 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.550+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 42 ==== osd_op_reply(154 pubsub..bucket.myfoo7/293899f1-2738-4b64-a8e1-a18ff039d626.4229.10 [delete] v33'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.550+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:155 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=147b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb315c70 con 0x7f6d8c044c20 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.551+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(155 fud.buckets [call] v33'7 uv7 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.552+0000 7f6d9d775800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-09-19T11:24:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.552+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:156 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fc2d0 con 0x7f6d8c044c20 2024-09-19T11:24:27.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.552+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== osd_op_reply(156 p5YnriCv1nAtykxBrupQ [delete] v33'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.552+0000 7f6d9d775800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-19T11:24:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:157 7.15 7:a93a5511:::notify.2:head [notify cookie 94802732459984 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fb680 con 0x5638fb2b3230 2024-09-19T11:24:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 40 ==== watch-notify(notify (1) cookie 94802732105056 notify 141733920775 ret 0) ==== 212+0+0 (crc 0 0 0) 0x5638fb2caae0 con 0x5638fb2b3230 2024-09-19T11:24:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6c7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920775 cookie 94802732105056 notifier 4761 bl.length()=170 2024-09-19T11:24:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 41 ==== osd_op_reply(157 notify.2 [notify cookie 94802732459984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6c7effd640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:158 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f6c60003d60 con 0x5638fb2b3230 2024-09-19T11:24:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 42 ==== watch-notify(notify_complete (2) cookie 94802732459984 notify 141733920775 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5638fb2a3570 con 0x5638fb2b3230 2024-09-19T11:24:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 43 ==== osd_op_reply(158 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9d775800 10 removing email index: 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9d775800 10 removing user buckets index 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.553+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:159 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fc2d0 con 0x7f6d8c044c20 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.554+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(159 fud.buckets [delete] v33'8 uv8 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.554+0000 7f6d9d775800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.554+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94802732459984 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fc2d0 con 0x7f6d8c044c20 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== watch-notify(notify (1) cookie 94802732107808 notify 141733920807 ret 0) ==== 202+0+0 (crc 0 0 0) 0x5638fb2cbb00 con 0x7f6d8c044c20 2024-09-19T11:24:27.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6c7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920807 cookie 94802732107808 notifier 4761 bl.length()=160 2024-09-19T11:24:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(160 notify.6 [notify cookie 94802732459984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6c7effd640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f6c60003d60 con 0x7f6d8c044c20 2024-09-19T11:24:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== watch-notify(notify_complete (2) cookie 94802732459984 notify 141733920807 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5638fb2cb090 con 0x7f6d8c044c20 2024-09-19T11:24:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6d9d775800 10 removing user index: fud 2024-09-19T11:24:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.555+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:162 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fb660 con 0x7f6d8c044c20 2024-09-19T11:24:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 46 ==== osd_op_reply(162 fud [call,call,delete] v33'15 uv15 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9d775800 10 removing default.rgw.meta+users.uid+fud from cache 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9d775800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94802732459984 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fb660 con 0x7f6d8c044c20 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 47 ==== watch-notify(notify (1) cookie 94802732114176 notify 141733920808 ret 0) ==== 194+0+0 (crc 0 0 0) 0x5638fb2cb090 con 0x7f6d8c044c20 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6c7e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920808 cookie 94802732114176 notifier 4761 bl.length()=152 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 48 ==== osd_op_reply(163 notify.5 [notify cookie 94802732459984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6c7e7fc640 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f6bd00032c0 con 0x7f6d8c044c20 2024-09-19T11:24:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 49 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.557+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 50 ==== watch-notify(notify_complete (2) cookie 94802732459984 notify 141733920808 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5638fb2cb470 con 0x7f6d8c044c20 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.558+0000 7f6c51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.558+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94802732118560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2fc2d0 con 0x5638fb2ae820 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.559+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 43 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 94802732118560] v33'227 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.559+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94802732094560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf220 con 0x7f6d9404d2d0 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.560+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 40 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 94802732094560] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.560+0000 7f6d9d775800 20 remove_watcher() i=0 2024-09-19T11:24:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.560+0000 7f6d9d775800 2 removed watcher, disabling cache 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.560+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94802732100800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf220 con 0x5638fb2ae820 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.562+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 44 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 94802732100800] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.562+0000 7f6d9d775800 20 remove_watcher() i=1 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.562+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94802732105056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf220 con 0x5638fb2b3230 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.563+0000 7f6d9ba67640 1 -- 172.21.15.28:0/3340702911 <== osd.3 v2:172.21.15.28:6808/3027134945 44 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 94802732105056] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d9400a4d0 con 0x5638fb2b3230 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.563+0000 7f6d9d775800 20 remove_watcher() i=2 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.563+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94802732098752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bbd60 con 0x7f6d9404d2d0 2024-09-19T11:24:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.564+0000 7f6d9b266640 1 -- 172.21.15.28:0/3340702911 <== osd.2 v2:172.21.15.28:6800/2071226075 41 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 94802732098752] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d8c0187b0 con 0x7f6d9404d2d0 2024-09-19T11:24:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.564+0000 7f6d9d775800 20 remove_watcher() i=3 2024-09-19T11:24:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.564+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94802732110448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf220 con 0x5638fb2ae820 2024-09-19T11:24:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.565+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.1 v2:172.21.15.28:6816/656119574 45 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 94802732110448] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x5638fb2ae820 2024-09-19T11:24:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.565+0000 7f6d9d775800 20 remove_watcher() i=4 2024-09-19T11:24:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.565+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94802732114176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bbd60 con 0x7f6d8c044c20 2024-09-19T11:24:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.566+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 51 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 94802732114176] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.566+0000 7f6d9d775800 20 remove_watcher() i=5 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.566+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94802732107808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bd270 con 0x7f6d8c044c20 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.567+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 52 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 94802732107808] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.567+0000 7f6d9d775800 20 remove_watcher() i=6 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.567+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94802732111504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5638fb2bf220 con 0x7f6d8c044c20 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 <== osd.0 v2:172.21.15.22:6800/3144434255 53 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 94802732111504] v33'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6d90051070 con 0x7f6d8c044c20 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 20 remove_watcher() i=7 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6d8c044c20 msgr2=0x7f6d8c04b860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f6d8c044c20 0x7f6d8c04b860 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb2ae820 msgr2=0x5638fb2b0ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb2ae820 0x5638fb2b0ce0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6d9404d2d0 msgr2=0x7f6d94057650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6d9404d2d0 0x7f6d94057650 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb2b3230 msgr2=0x5638fb2b5620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb2b3230 0x5638fb2b5620 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c6403d000 msgr2=0x7f6c6403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c6403d000 0x7f6c6403f4c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c018ac0 tx=0x7f6d8c047000 comp rx=0 tx=0).stop 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 msgr2=0x5638fb2a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 --2- 172.21.15.28:0/3340702911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb29ba00 0x5638fb2a3010 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6d94054750 tx=0x7f6d94054780 comp rx=0 tx=0).stop 2024-09-19T11:24:27.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 reap_dead start 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9aa65640 1 -- 172.21.15.28:0/3340702911 reap_dead start 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 shutdown_connections 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 >> 172.21.15.28:0/3340702911 conn(0x5638fb297e00 msgr2=0x5638fb296010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 shutdown_connections 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.568+0000 7f6d9d775800 1 -- 172.21.15.28:0/3340702911 wait complete. 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c7403d050 msgr2=0x7f6c7403f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c7403d050 0x7f6c7403f510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c720 tx=0x7f6d8c052990 comp rx=0 tx=0).stop 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb28ef80 msgr2=0x5638fb28f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb28ef80 0x5638fb28f3a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x5638fb274f20 tx=0x7f6d940167b0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 shutdown_connections 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c7403d050 0x7f6c7403f510 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/775722903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb28ef80 0x5638fb28f3a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 >> 172.21.15.28:0/775722903 conn(0x5638fb283330 msgr2=0x5638fb283710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 shutdown_connections 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/775722903 wait complete. 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb17aab0 msgr2=0x5638fb27b6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb17aab0 0x5638fb27b6e0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb27db50 msgr2=0x5638fb27ff40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb27db50 0x5638fb27ff40 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c8003d000 msgr2=0x7f6c8003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c8003d000 0x7f6c8003f4c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6d90057000 tx=0x7f6d90052000 comp rx=0 tx=0).stop 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 msgr2=0x5638fb274090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb274090 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00c010 tx=0x7f6d8c000ed0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 shutdown_connections 2024-09-19T11:24:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5638fb27db50 0x5638fb27ff40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6c8003d000 0x7f6c8003f4c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5638fb17aab0 0x5638fb27b6e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 --2- 172.21.15.28:0/2887646132 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5638fb1796a0 0x5638fb274090 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:24:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 >> 172.21.15.28:0/2887646132 conn(0x5638fb174a70 msgr2=0x5638fb2633a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:24:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 shutdown_connections 2024-09-19T11:24:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:24:27.569+0000 7f6d9d775800 1 -- 172.21.15.28:0/2887646132 wait complete. 2024-09-19T11:25:07.595 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-09-19T11:25:07.595 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-09-19T11:25:07.595 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-09-19T11:25:07.649 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:07.649 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:07.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.649+0000 7fd4b860f800 1 Processor -- start 2024-09-19T11:25:07.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.649+0000 7fd4b860f800 1 -- start start 2024-09-19T11:25:07.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.649+0000 7fd4b860f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563e70411840 0x563e7041a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.649+0000 7fd4b860f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563e7041a8a0 con 0x563e7040fd20 2024-09-19T11:25:07.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.649+0000 7fd4b860f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x563e7029c7d0 con 0x563e70411840 2024-09-19T11:25:07.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.650+0000 7fd4b3591640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563e70411840 0x563e7041a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.650+0000 7fd4b3591640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x563e70411840 0x563e7041a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44326/0 (socket says 172.21.15.28:44326) 2024-09-19T11:25:07.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.650+0000 7fd4b3591640 1 -- 172.21.15.28:0/3414790763 learned_addr learned my addr 172.21.15.28:0/3414790763 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.650+0000 7fd4b3d92640 1 --1- 172.21.15.28:0/3414790763 >> v1:172.21.15.22:6789/0 conn(0x563e7040fd20 0x563e70410120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48206/0 (socket says 172.21.15.28:48206) 2024-09-19T11:25:07.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.650+0000 7fd4b3591640 1 -- 172.21.15.28:0/3414790763 >> v1:172.21.15.22:6789/0 conn(0x563e7040fd20 legacy=0x563e70410120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.650+0000 7fd4b3591640 1 -- 172.21.15.28:0/3414790763 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7041ba80 con 0x563e70411840 2024-09-19T11:25:07.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.650+0000 7fd4b2d90640 1 -- 172.21.15.28:0/3414790763 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 482865394 0 0) 0x563e7041a8a0 con 0x563e7040fd20 2024-09-19T11:25:07.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.652+0000 7fd4b3591640 1 --2- 172.21.15.28:0/3414790763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70411840 0x563e7041a360 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800a520 tx=0x7fd4a8045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c03e10175452a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.652+0000 7fd4b2d90640 1 -- 172.21.15.28:0/3414790763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8055070 con 0x563e70411840 2024-09-19T11:25:07.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.652+0000 7fd4b2d90640 1 -- 172.21.15.28:0/3414790763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a80515b0 con 0x563e70411840 2024-09-19T11:25:07.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.652+0000 7fd4b2d90640 1 -- 172.21.15.28:0/3414790763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a80518d0 con 0x563e70411840 2024-09-19T11:25:07.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.653+0000 7fd4b860f800 1 -- 172.21.15.28:0/3414790763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70411840 msgr2=0x563e7041a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.653+0000 7fd4b860f800 1 --2- 172.21.15.28:0/3414790763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70411840 0x563e7041a360 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800a520 tx=0x7fd4a8045ba0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.653+0000 7fd4b860f800 1 -- 172.21.15.28:0/3414790763 shutdown_connections 2024-09-19T11:25:07.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.653+0000 7fd4b860f800 1 --2- 172.21.15.28:0/3414790763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70411840 0x563e7041a360 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.653+0000 7fd4b860f800 1 -- 172.21.15.28:0/3414790763 >> 172.21.15.28:0/3414790763 conn(0x563e7040ba50 msgr2=0x563e7040deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.653+0000 7fd4b860f800 1 -- 172.21.15.28:0/3414790763 shutdown_connections 2024-09-19T11:25:07.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.653+0000 7fd4b860f800 1 -- 172.21.15.28:0/3414790763 wait complete. 2024-09-19T11:25:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.670+0000 7fd4b860f800 1 Processor -- start 2024-09-19T11:25:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.671+0000 7fd4b860f800 1 -- start start 2024-09-19T11:25:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.671+0000 7fd4b860f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e70503390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.671+0000 7fd4b860f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563e702a9dd0 con 0x563e7040fd20 2024-09-19T11:25:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.671+0000 7fd4b3d92640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e70503390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.671+0000 7fd4b3d92640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e70503390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44338/0 (socket says 172.21.15.28:44338) 2024-09-19T11:25:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.671+0000 7fd4b3d92640 1 -- 172.21.15.28:0/3517523431 learned_addr learned my addr 172.21.15.28:0/3517523431 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.671+0000 7fd4b3d92640 1 -- 172.21.15.28:0/3517523431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e70503980 con 0x563e7040fd20 2024-09-19T11:25:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.672+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/3517523431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e70503390 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4001ea0 tx=0x7fd4a400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87046a214997e693 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.672+0000 7fd3a1e0b640 1 -- 172.21.15.28:0/3517523431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a4008070 con 0x563e7040fd20 2024-09-19T11:25:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.672+0000 7fd3a1e0b640 1 -- 172.21.15.28:0/3517523431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a400d500 con 0x563e7040fd20 2024-09-19T11:25:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.672+0000 7fd3a1e0b640 1 -- 172.21.15.28:0/3517523431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a400d880 con 0x563e7040fd20 2024-09-19T11:25:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.672+0000 7fd4b860f800 1 -- 172.21.15.28:0/3517523431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 msgr2=0x563e70503390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.672+0000 7fd4b860f800 1 --2- 172.21.15.28:0/3517523431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e70503390 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4001ea0 tx=0x7fd4a400b6a0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 -- 172.21.15.28:0/3517523431 shutdown_connections 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 --2- 172.21.15.28:0/3517523431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e70503390 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 -- 172.21.15.28:0/3517523431 >> 172.21.15.28:0/3517523431 conn(0x563e7040ba50 msgr2=0x563e704fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 -- 172.21.15.28:0/3517523431 shutdown_connections 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 -- 172.21.15.28:0/3517523431 wait complete. 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 Processor -- start 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 -- start start 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e7050ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b860f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563e7041ad40 con 0x563e7040fd20 2024-09-19T11:25:07.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b3d92640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e7050ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b3d92640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e7050ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44342/0 (socket says 172.21.15.28:44342) 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.673+0000 7fd4b3d92640 1 -- 172.21.15.28:0/2882286501 learned_addr learned my addr 172.21.15.28:0/2882286501 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.674+0000 7fd4b3d92640 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7050b3e0 con 0x563e7040fd20 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.674+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e7050ae80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40012f0 tx=0x7fd4a4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.674+0000 7fd39bfff640 1 -- 172.21.15.28:0/2882286501 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a400f9d0 con 0x563e7040fd20 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.674+0000 7fd39bfff640 1 -- 172.21.15.28:0/2882286501 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a400fb70 con 0x563e7040fd20 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.674+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e7050b670 con 0x563e7040fd20 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.674+0000 7fd39bfff640 1 -- 172.21.15.28:0/2882286501 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a400fe50 con 0x563e7040fd20 2024-09-19T11:25:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.674+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7050c070 con 0x563e7040fd20 2024-09-19T11:25:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.675+0000 7fd39bfff640 1 -- 172.21.15.28:0/2882286501 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd4a4017040 con 0x563e7040fd20 2024-09-19T11:25:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.675+0000 7fd39bfff640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd39c03d1f0 0x7fd39c03f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.675+0000 7fd39bfff640 1 -- 172.21.15.28:0/2882286501 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fd4a4015020 con 0x563e7040fd20 2024-09-19T11:25:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.676+0000 7fd4b3591640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd39c03d1f0 0x7fd39c03f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.676+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e70411840 0x563e705124d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.676+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e70512a10 con 0x563e70411840 2024-09-19T11:25:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.676+0000 7fd4b4593640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e70411840 0x563e705124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.676+0000 7fd4b3591640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd39c03d1f0 0x7fd39c03f6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8057000 tx=0x7fd4a8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.676+0000 7fd4b4593640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e70411840 0x563e705124d0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.677+0000 7fd4b4593640 1 -- 172.21.15.28:0/2882286501 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4ac00a4d0 con 0x563e70411840 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.677+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e70512db0 con 0x563e70411840 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.677+0000 7fd4b4593640 1 -- 172.21.15.28:0/2882286501 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd4ac00a4d0 con 0x563e70411840 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.677+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e705148b0 0x563e70516d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.677+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e705174c0 con 0x563e705148b0 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.677+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e705148b0 0x563e70516d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.677+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e705148b0 0x563e70516d30 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.678+0000 7fd4b3d92640 1 -- 172.21.15.28:0/2882286501 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd4a4054040 con 0x563e705148b0 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.678+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e70513600 con 0x563e70411840 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.678+0000 7fd4b4593640 1 -- 172.21.15.28:0/2882286501 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd4ac00a4d0 con 0x563e70411840 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.678+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e70513a50 con 0x563e705148b0 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.678+0000 7fd4b3d92640 1 -- 172.21.15.28:0/2882286501 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd4a4054040 con 0x563e705148b0 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.679+0000 7fd4b860f800 1 Processor -- start 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.679+0000 7fd4b860f800 1 -- start start 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.679+0000 7fd4b860f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70525e50 0x563e70526270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.679+0000 7fd4b860f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563e703084a0 con 0x563e70525e50 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.679+0000 7fd4b4593640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70525e50 0x563e70526270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.679+0000 7fd4b4593640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70525e50 0x563e70526270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44350/0 (socket says 172.21.15.28:44350) 2024-09-19T11:25:07.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.679+0000 7fd4b4593640 1 -- 172.21.15.28:0/2804655788 learned_addr learned my addr 172.21.15.28:0/2804655788 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.680+0000 7fd4b4593640 1 -- 172.21.15.28:0/2804655788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e705268b0 con 0x563e70525e50 2024-09-19T11:25:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.680+0000 7fd4b4593640 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70525e50 0x563e70526270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x563e7050be80 tx=0x7fd4ac0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.680+0000 7fd3997fa640 1 -- 172.21.15.28:0/2804655788 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ac013640 con 0x563e70525e50 2024-09-19T11:25:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.680+0000 7fd3997fa640 1 -- 172.21.15.28:0/2804655788 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ac0137e0 con 0x563e70525e50 2024-09-19T11:25:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.680+0000 7fd3997fa640 1 -- 172.21.15.28:0/2804655788 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ac013ac0 con 0x563e70525e50 2024-09-19T11:25:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.680+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e705270e0 con 0x563e70525e50 2024-09-19T11:25:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.680+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e70527680 con 0x563e70525e50 2024-09-19T11:25:07.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.681+0000 7fd3997fa640 1 -- 172.21.15.28:0/2804655788 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd4ac011040 con 0x563e70525e50 2024-09-19T11:25:07.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd3997fa640 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd38c03d000 0x7fd38c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd3997fa640 1 -- 172.21.15.28:0/2804655788 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fd4ac00e040 con 0x563e70525e50 2024-09-19T11:25:07.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd38c03d000 0x7fd38c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd4b860f800 1 Processor -- start 2024-09-19T11:25:07.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd38c03d000 0x7fd38c03f4c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400ecb0 tx=0x7fd4a4001460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd4b860f800 1 -- start start 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd4b860f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70532cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd4b860f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563e70503db0 con 0x563e705328d0 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.682+0000 7fd4b4593640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70532cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.683+0000 7fd4b4593640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70532cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44354/0 (socket says 172.21.15.28:44354) 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.683+0000 7fd4b4593640 1 -- 172.21.15.28:0/8214079 learned_addr learned my addr 172.21.15.28:0/8214079 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.683+0000 7fd4b4593640 1 -- 172.21.15.28:0/8214079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e705332c0 con 0x563e705328d0 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.683+0000 7fd4b4593640 1 --2- 172.21.15.28:0/8214079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70532cd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00afe0 tx=0x7fd4ac0204b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05aed11019c0298 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.683+0000 7fd392ffd640 1 -- 172.21.15.28:0/8214079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ac044030 con 0x563e705328d0 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.683+0000 7fd392ffd640 1 -- 172.21.15.28:0/8214079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ac021a90 con 0x563e705328d0 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.683+0000 7fd392ffd640 1 -- 172.21.15.28:0/8214079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ac021570 con 0x563e705328d0 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 -- 172.21.15.28:0/8214079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 msgr2=0x563e70532cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 --2- 172.21.15.28:0/8214079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70532cd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00afe0 tx=0x7fd4ac0204b0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 -- 172.21.15.28:0/8214079 shutdown_connections 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 --2- 172.21.15.28:0/8214079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70532cd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 -- 172.21.15.28:0/8214079 >> 172.21.15.28:0/8214079 conn(0x563e7052ecd0 msgr2=0x563e7052cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 -- 172.21.15.28:0/8214079 shutdown_connections 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 -- 172.21.15.28:0/8214079 wait complete. 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 Processor -- start 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 -- start start 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70539eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.684+0000 7fd4b860f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563e705338e0 con 0x563e705328d0 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.685+0000 7fd4b4593640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70539eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.685+0000 7fd4b4593640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70539eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44366/0 (socket says 172.21.15.28:44366) 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.685+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 learned_addr learned my addr 172.21.15.28:0/4265893099 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.685+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e7053a410 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.685+0000 7fd4b4593640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70539eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac020b60 tx=0x7fd4ac0547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.686+0000 7fd3917fa640 1 -- 172.21.15.28:0/4265893099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ac01fa90 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.686+0000 7fd3917fa640 1 -- 172.21.15.28:0/4265893099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ac01fc30 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.686+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563e7053a6a0 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.686+0000 7fd3917fa640 1 -- 172.21.15.28:0/4265893099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ac056070 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.686+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7053b230 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.686+0000 7fd3917fa640 1 -- 172.21.15.28:0/4265893099 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd4ac056210 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.687+0000 7fd3917fa640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd37c03d000 0x7fd37c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.687+0000 7fd3917fa640 1 -- 172.21.15.28:0/4265893099 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fd4ac03d060 con 0x563e705328d0 2024-09-19T11:25:07.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.687+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd37c03d000 0x7fd37c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.687+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd37c03d000 0x7fd37c03f4c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400bbf0 tx=0x7fd4a404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.687+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e705457a0 0x563e70547c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e705483b0 con 0x563e705457a0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b3591640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e705457a0 0x563e70547c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b3591640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e705457a0 0x563e70547c60 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b860f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b860f800 20 realm 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.688+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e705487e0 con 0x563e705457a0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e705490b0 con 0x563e705457a0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e70549450 con 0x563e705457a0 2024-09-19T11:25:07.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e7054a1b0 0x563e7054c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e7054cbb0 con 0x563e7054a1b0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.689+0000 7fd4b4593640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e7054a1b0 0x563e7054c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b4593640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e7054a1b0 0x563e7054c5a0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b860f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b860f800 20 searching for the correct realm 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b4593640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd4ac03cab0 0x7fd4ac019000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.690+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd4ac03cab0 0x7fd4ac019000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.691+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd4ac03cab0 0x7fd4ac019000 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.691+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.691+0000 7fd4b3d92640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd4a403c070 0x7fd4a40542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.691+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4a403c070 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.691+0000 7fd4b3591640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd4a403c070 0x7fd4a40542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.692+0000 7fd4b3591640 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd4a403c070 0x7fd4a40542d0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.692+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.692+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4a403c070 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4a403c070 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.693+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.694+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.694+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.694+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.694+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.694+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.694+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4a403c070 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.695+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.696+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.697+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.697+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.697+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.697+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4a403c070 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.697+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.697+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.698+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.698+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.698+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.698+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.698+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.698+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4a403c070 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4a403c070 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.699+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e7054a1b0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.700+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.700+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.700+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.700+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x563e705457a0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.700+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.700+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e4e0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:07.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e7054d5a0 con 0x563e705457a0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e7054ddf0 con 0x563e705457a0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e7054e960 con 0x563e7054a1b0 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 zone default found 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 4 Realm: () 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.701+0000 7fd4b860f800 20 zonegroup default 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e7054ed60 con 0x7fd4a403c070 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e70551040 con 0x563e7054a1b0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e705513e0 con 0x563e7054a1b0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.702+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.703+0000 7fd4b860f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.703+0000 7fd4b860f800 20 started sync module instance, tier type = 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.703+0000 7fd4b860f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.703+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70550480 con 0x7fd4ac03cab0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.704+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.704+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94826172579792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705517f0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.704+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70552040 con 0x563e705457a0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.705+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94826172579792] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.705+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.705+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94826172586896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70552460 con 0x563e705457a0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.705+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70552cf0 con 0x563e7054a1b0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.706+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94826172586896] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.707+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.707+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94826172587952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70552880 con 0x563e7054a1b0 2024-09-19T11:25:07.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.707+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70553740 con 0x7fd4ac03cab0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.708+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94826172587952] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.708+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.708+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94826172589008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705527f0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.708+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70554600 con 0x563e705457a0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.709+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94826172589008] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.709+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.709+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94826172596560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70554570 con 0x563e705457a0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.709+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705554c0 con 0x7fd4a403c070 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.710+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94826172596560] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.711+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.711+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94826172598688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705554c0 con 0x7fd4a403c070 2024-09-19T11:25:07.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.711+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70555e80 con 0x7fd4a403c070 2024-09-19T11:25:07.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.712+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94826172598688] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.712+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.712+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94826172600848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70555e80 con 0x7fd4a403c070 2024-09-19T11:25:07.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.712+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70556840 con 0x7fd4a403c070 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94826172600848] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94826172602832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70556a20 con 0x7fd4a403c070 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 20 add_watcher() i=0 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 20 add_watcher() i=1 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 20 add_watcher() i=2 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 20 add_watcher() i=3 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 20 add_watcher() i=4 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 20 add_watcher() i=5 2024-09-19T11:25:07.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.713+0000 7fd4b860f800 20 add_watcher() i=6 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.714+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94826172602832] v33'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.714+0000 7fd4b860f800 20 add_watcher() i=7 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.714+0000 7fd4b860f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.714+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e70559c10 con 0x563e705457a0 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.715+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.715+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94826172605904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70556ea0 con 0x563e705457a0 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94826172605904] v33'228 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd369fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd4b860f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:07.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd4b860f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd4b860f800 5 note: GC not initialized 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd368fe1640 20 reqs_thread_entry: start 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd4b860f800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.716+0000 7fd4b860f800 20 Filter name: none 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd2e27fc640 20 reqs_thread_entry: start 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x563e7058cad0 con 0x563e705457a0 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.717+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e7058f400 con 0x7fd4a403c070 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.719+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 usage.10 [call] v33'32 uv32 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.719+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e7058f400 con 0x7fd4a403c070 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.720+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(64 usage.10 [call] v33'33 uv32 ondisk = -61 ((61) No data available)) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.720+0000 7fd369fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.720+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94826172605904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705589d0 con 0x563e705457a0 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.721+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94826172605904] v33'229 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.721+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94826172579792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705565d0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.723+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94826172579792] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.723+0000 7fd4b860f800 20 remove_watcher() i=0 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.723+0000 7fd4b860f800 2 removed watcher, disabling cache 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.723+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94826172586896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705565d0 con 0x563e705457a0 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.724+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94826172586896] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.724+0000 7fd4b860f800 20 remove_watcher() i=1 2024-09-19T11:25:07.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.724+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94826172587952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70551b90 con 0x563e7054a1b0 2024-09-19T11:25:07.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.725+0000 7fd4b4593640 1 -- 172.21.15.28:0/4265893099 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94826172587952] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4ac053020 con 0x563e7054a1b0 2024-09-19T11:25:07.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.725+0000 7fd4b860f800 20 remove_watcher() i=2 2024-09-19T11:25:07.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.725+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94826172589008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705565d0 con 0x7fd4ac03cab0 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.726+0000 7fd4b3d92640 1 -- 172.21.15.28:0/4265893099 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94826172589008] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a4054040 con 0x7fd4ac03cab0 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.726+0000 7fd4b860f800 20 remove_watcher() i=3 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.726+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94826172596560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705565d0 con 0x563e705457a0 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.727+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94826172596560] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x563e705457a0 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.728+0000 7fd4b860f800 20 remove_watcher() i=4 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.728+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94826172598688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e70554150 con 0x7fd4a403c070 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.729+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94826172598688] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.729+0000 7fd4b860f800 20 remove_watcher() i=5 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.729+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94826172600848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705565d0 con 0x7fd4a403c070 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.730+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94826172600848] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.730+0000 7fd4b860f800 20 remove_watcher() i=6 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.730+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94826172602832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x563e705565d0 con 0x7fd4a403c070 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94826172602832] v33'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8057030 con 0x7fd4a403c070 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 20 remove_watcher() i=7 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd4a403c070 msgr2=0x7fd4a40542d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd4a403c070 0x7fd4a40542d0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e705457a0 msgr2=0x563e70547c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e705457a0 0x563e70547c60 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd4ac03cab0 msgr2=0x7fd4ac019000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd4ac03cab0 0x7fd4ac019000 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e7054a1b0 msgr2=0x563e7054c5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e7054a1b0 0x563e7054c5a0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd37c03d000 msgr2=0x7fd37c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd37c03d000 0x7fd37c03f4c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400bbf0 tx=0x7fd4a404f000 comp rx=0 tx=0).stop 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b3591640 1 -- 172.21.15.28:0/4265893099 reap_dead start 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 msgr2=0x563e70539eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70539eb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac020b60 tx=0x7fd4ac0547f0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 shutdown_connections 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 --2- 172.21.15.28:0/4265893099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e705328d0 0x563e70539eb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 >> 172.21.15.28:0/4265893099 conn(0x563e7052ecd0 msgr2=0x563e7052cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.731+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 shutdown_connections 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/4265893099 wait complete. 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd38c03d000 msgr2=0x7fd38c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd38c03d000 0x7fd38c03f4c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400ecb0 tx=0x7fd4a4001460 comp rx=0 tx=0).stop 2024-09-19T11:25:07.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70525e50 msgr2=0x563e70526270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70525e50 0x563e70526270 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x563e7050be80 tx=0x7fd4ac0167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 shutdown_connections 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd38c03d000 0x7fd38c03f4c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2804655788 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e70525e50 0x563e70526270 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 >> 172.21.15.28:0/2804655788 conn(0x563e7051a200 msgr2=0x563e7051a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 shutdown_connections 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2804655788 wait complete. 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e70411840 msgr2=0x563e705124d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e70411840 0x563e705124d0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e705148b0 msgr2=0x563e70516d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e705148b0 0x563e70516d30 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd39c03d1f0 msgr2=0x7fd39c03f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd39c03d1f0 0x7fd39c03f6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8057000 tx=0x7fd4a8052000 comp rx=0 tx=0).stop 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 msgr2=0x563e7050ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e7050ae80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40012f0 tx=0x7fd4a4000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 shutdown_connections 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x563e705148b0 0x563e70516d30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd39c03d1f0 0x7fd39c03f6b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x563e70411840 0x563e705124d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 --2- 172.21.15.28:0/2882286501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563e7040fd20 0x563e7050ae80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.732+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 >> 172.21.15.28:0/2882286501 conn(0x563e7040ba50 msgr2=0x563e70504880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.733+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 shutdown_connections 2024-09-19T11:25:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.733+0000 7fd4b860f800 1 -- 172.21.15.28:0/2882286501 wait complete. 2024-09-19T11:25:07.743 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-09-19T11:25:07.743 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-09-19T11:25:07.743 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-09-19T11:25:07.789 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:07.789 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96d6a0e800 1 Processor -- start 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96d6a0e800 1 -- start start 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96d6a0e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55fd8005b870 0x55fd80064390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96d6a0e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55fd800648d0 con 0x55fd80059d50 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96d6a0e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55fd7fee67d0 con 0x55fd8005b870 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96cbfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55fd8005b870 0x55fd80064390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96d20cb640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55fd80059d50 0x55fd8005a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48208/0 (socket says 172.21.15.28:48208) 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96d20cb640 1 -- 172.21.15.28:0/3578646751 learned_addr learned my addr 172.21.15.28:0/3578646751 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96cbfff640 1 -- 172.21.15.28:0/3578646751 >> v1:172.21.15.22:6789/0 conn(0x55fd80059d50 legacy=0x55fd8005a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.789+0000 7f96cbfff640 1 -- 172.21.15.28:0/3578646751 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd80065ab0 con 0x55fd8005b870 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.791+0000 7f96cbfff640 1 --2- 172.21.15.28:0/3578646751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8005b870 0x55fd80064390 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00a470 tx=0x7f96bc045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa10622f8afd6bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d18ca640 1 -- 172.21.15.28:0/3578646751 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96bc053570 con 0x55fd8005b870 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d18ca640 1 -- 172.21.15.28:0/3578646751 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc058070 con 0x55fd8005b870 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d18ca640 1 -- 172.21.15.28:0/3578646751 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96bc053880 con 0x55fd8005b870 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3578646751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8005b870 msgr2=0x55fd80064390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3578646751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8005b870 0x55fd80064390 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00a470 tx=0x7f96bc045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3578646751 shutdown_connections 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3578646751 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8005b870 0x55fd80064390 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3578646751 >> 172.21.15.28:0/3578646751 conn(0x55fd80055a60 msgr2=0x55fd80057ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3578646751 shutdown_connections 2024-09-19T11:25:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.792+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3578646751 wait complete. 2024-09-19T11:25:07.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.807+0000 7f96d6a0e800 1 Processor -- start 2024-09-19T11:25:07.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.807+0000 7f96d6a0e800 1 -- start start 2024-09-19T11:25:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.807+0000 7f96d6a0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd8014d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.808+0000 7f96d6a0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fd7fef3dd0 con 0x55fd80059d50 2024-09-19T11:25:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.808+0000 7f96d20cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd8014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.808+0000 7f96d20cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd8014d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44382/0 (socket says 172.21.15.28:44382) 2024-09-19T11:25:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.808+0000 7f96d20cb640 1 -- 172.21.15.28:0/2623964516 learned_addr learned my addr 172.21.15.28:0/2623964516 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.808+0000 7f96d20cb640 1 -- 172.21.15.28:0/2623964516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8014db40 con 0x55fd80059d50 2024-09-19T11:25:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.808+0000 7f96d20cb640 1 --2- 172.21.15.28:0/2623964516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd8014d550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f96c4001e40 tx=0x7f96c4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b456b07b7d60fc5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.808+0000 7f95c0608640 1 -- 172.21.15.28:0/2623964516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c4012020 con 0x55fd80059d50 2024-09-19T11:25:07.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f95c0608640 1 -- 172.21.15.28:0/2623964516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c400e5a0 con 0x55fd80059d50 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f95c0608640 1 -- 172.21.15.28:0/2623964516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c400e8a0 con 0x55fd80059d50 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2623964516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 msgr2=0x55fd8014d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2623964516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd8014d550 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f96c4001e40 tx=0x7f96c4001e70 comp rx=0 tx=0).stop 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2623964516 shutdown_connections 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2623964516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd8014d550 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2623964516 >> 172.21.15.28:0/2623964516 conn(0x55fd80055a60 msgr2=0x55fd80148ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2623964516 shutdown_connections 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.809+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2623964516 wait complete. 2024-09-19T11:25:07.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d6a0e800 1 Processor -- start 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d6a0e800 1 -- start start 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d6a0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd80155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d6a0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fd80064da0 con 0x55fd80059d50 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d20cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd80155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d20cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd80155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44388/0 (socket says 172.21.15.28:44388) 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d20cb640 1 -- 172.21.15.28:0/2687097255 learned_addr learned my addr 172.21.15.28:0/2687097255 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.810+0000 7f96d20cb640 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd801555c0 con 0x55fd80059d50 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.811+0000 7f96d20cb640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd80155060 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f96c40012b0 tx=0x7f96c4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.811+0000 7f95bee05640 1 -- 172.21.15.28:0/2687097255 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c4012030 con 0x55fd80059d50 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.811+0000 7f95bee05640 1 -- 172.21.15.28:0/2687097255 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c401a750 con 0x55fd80059d50 2024-09-19T11:25:07.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.811+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fd80155850 con 0x55fd80059d50 2024-09-19T11:25:07.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.811+0000 7f95bee05640 1 -- 172.21.15.28:0/2687097255 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96c401aa70 con 0x55fd80059d50 2024-09-19T11:25:07.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.811+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd80156250 con 0x55fd80059d50 2024-09-19T11:25:07.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.812+0000 7f95bee05640 1 -- 172.21.15.28:0/2687097255 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f96c4015020 con 0x55fd80059d50 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.812+0000 7f95bee05640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95b803cfb0 0x7f95b803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.812+0000 7f95bee05640 1 -- 172.21.15.28:0/2687097255 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f96c4020030 con 0x55fd80059d50 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.812+0000 7f96cbfff640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95b803cfb0 0x7f95b803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.812+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8005b870 0x55fd8015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.812+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd8015cbf0 con 0x55fd8005b870 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.812+0000 7f96d28cc640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8005b870 0x55fd8015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96cbfff640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95b803cfb0 0x7f95b803f470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f96bc057000 tx=0x7f96bc052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96d28cc640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8005b870 0x55fd8015c6b0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96d28cc640 1 -- 172.21.15.28:0/2687097255 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96cc00a4d0 con 0x55fd8005b870 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd8015cf90 con 0x55fd8005b870 2024-09-19T11:25:07.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96d28cc640 1 -- 172.21.15.28:0/2687097255 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f96cc00a4d0 con 0x55fd8005b870 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd8015ea50 0x55fd80160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80161680 con 0x55fd8015ea50 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.813+0000 7f96d20cb640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd8015ea50 0x55fd80160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.814+0000 7f96d20cb640 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd8015ea50 0x55fd80160ef0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.814+0000 7f96d20cb640 1 -- 172.21.15.28:0/2687097255 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f96c4018ce0 con 0x55fd8015ea50 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.814+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd8015d7e0 con 0x55fd8005b870 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.814+0000 7f96d28cc640 1 -- 172.21.15.28:0/2687097255 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f96cc00a4d0 con 0x55fd8005b870 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.814+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd8015db80 con 0x55fd8015ea50 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d20cb640 1 -- 172.21.15.28:0/2687097255 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f96c4018ce0 con 0x55fd8015ea50 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d6a0e800 1 Processor -- start 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d6a0e800 1 -- start start 2024-09-19T11:25:07.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d6a0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8016ffe0 0x55fd80170400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d6a0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fd7ff524a0 con 0x55fd8016ffe0 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d28cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8016ffe0 0x55fd80170400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d28cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8016ffe0 0x55fd80170400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44400/0 (socket says 172.21.15.28:44400) 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.815+0000 7f96d28cc640 1 -- 172.21.15.28:0/3960877055 learned_addr learned my addr 172.21.15.28:0/3960877055 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.816+0000 7f96d28cc640 1 -- 172.21.15.28:0/3960877055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd80170a40 con 0x55fd8016ffe0 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.816+0000 7f96d28cc640 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8016ffe0 0x55fd80170400 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55fd80155ef0 tx=0x7f96cc0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.816+0000 7f95b7fff640 1 -- 172.21.15.28:0/3960877055 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc013640 con 0x55fd8016ffe0 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.816+0000 7f95b7fff640 1 -- 172.21.15.28:0/3960877055 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc0137e0 con 0x55fd8016ffe0 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.816+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fd80171270 con 0x55fd8016ffe0 2024-09-19T11:25:07.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.816+0000 7f95b7fff640 1 -- 172.21.15.28:0/3960877055 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc013b00 con 0x55fd8016ffe0 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.816+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd80171810 con 0x55fd8016ffe0 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.817+0000 7f95b7fff640 1 -- 172.21.15.28:0/3960877055 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f96cc011040 con 0x55fd8016ffe0 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.817+0000 7f95b7fff640 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95ac03d050 0x7f95ac03f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.817+0000 7f95b7fff640 1 -- 172.21.15.28:0/3960877055 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f96cc00e040 con 0x55fd8016ffe0 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.817+0000 7f96d20cb640 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95ac03d050 0x7f95ac03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d6a0e800 1 Processor -- start 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d20cb640 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95ac03d050 0x7f95ac03f510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55fd80157160 tx=0x7f96c4052a40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d6a0e800 1 -- start start 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d6a0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd8017ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d6a0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fd8014df70 con 0x55fd8017ca60 2024-09-19T11:25:07.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d28cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd8017ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d28cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd8017ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44404/0 (socket says 172.21.15.28:44404) 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.818+0000 7f96d28cc640 1 -- 172.21.15.28:0/14024797 learned_addr learned my addr 172.21.15.28:0/14024797 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d28cc640 1 -- 172.21.15.28:0/14024797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8017d450 con 0x55fd8017ca60 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d28cc640 1 --2- 172.21.15.28:0/14024797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd8017ce60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00afe0 tx=0x7f96cc01ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed28ed107f28c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f95b5ffb640 1 -- 172.21.15.28:0/14024797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc02a910 con 0x55fd8017ca60 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f95b5ffb640 1 -- 172.21.15.28:0/14024797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc04de50 con 0x55fd8017ca60 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f95b5ffb640 1 -- 172.21.15.28:0/14024797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc013ca0 con 0x55fd8017ca60 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d6a0e800 1 -- 172.21.15.28:0/14024797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 msgr2=0x55fd8017ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/14024797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd8017ce60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00afe0 tx=0x7f96cc01ff20 comp rx=0 tx=0).stop 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d6a0e800 1 -- 172.21.15.28:0/14024797 shutdown_connections 2024-09-19T11:25:07.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/14024797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd8017ce60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d6a0e800 1 -- 172.21.15.28:0/14024797 >> 172.21.15.28:0/14024797 conn(0x55fd80178e60 msgr2=0x55fd801770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d6a0e800 1 -- 172.21.15.28:0/14024797 shutdown_connections 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.819+0000 7f96d6a0e800 1 -- 172.21.15.28:0/14024797 wait complete. 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.820+0000 7f96d6a0e800 1 Processor -- start 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.820+0000 7f96d6a0e800 1 -- start start 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.820+0000 7f96d6a0e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd80184020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.820+0000 7f96d6a0e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fd8017da70 con 0x55fd8017ca60 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.820+0000 7f96d28cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd80184020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.820+0000 7f96d28cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd80184020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44420/0 (socket says 172.21.15.28:44420) 2024-09-19T11:25:07.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.820+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 learned_addr learned my addr 172.21.15.28:0/3650403761 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.821+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd80184580 con 0x55fd8017ca60 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.821+0000 7f96d28cc640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd80184020 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f96cc054780 tx=0x7f96cc0547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.821+0000 7f95a3fff640 1 -- 172.21.15.28:0/3650403761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc0459f0 con 0x55fd8017ca60 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.821+0000 7f95a3fff640 1 -- 172.21.15.28:0/3650403761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc045b90 con 0x55fd8017ca60 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.821+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fd80184810 con 0x55fd8017ca60 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.821+0000 7f95a3fff640 1 -- 172.21.15.28:0/3650403761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96cc054960 con 0x55fd8017ca60 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.821+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd801853a0 con 0x55fd8017ca60 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.822+0000 7f95a3fff640 1 -- 172.21.15.28:0/3650403761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f96cc054b00 con 0x55fd8017ca60 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.822+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.822+0000 7f95a3fff640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f959c03d000 0x7f959c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.823+0000 7f95a3fff640 1 -- 172.21.15.28:0/3650403761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f96cc017450 con 0x55fd8017ca60 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.823+0000 7f96d20cb640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f959c03d000 0x7f959c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.823+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8018f980 0x55fd80191dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.823+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80192520 con 0x55fd8018f980 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.823+0000 7f96cbfff640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8018f980 0x55fd80191dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.823+0000 7f96d20cb640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f959c03d000 0x7f959c03f4c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f96c403f010 tx=0x7f96c4046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.823+0000 7f96cbfff640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8018f980 0x55fd80191dd0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 realm 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80192950 con 0x55fd8018f980 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80193220 con 0x55fd8018f980 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd801935c0 con 0x55fd8018f980 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd80194320 0x55fd80196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80196c50 con 0x55fd80194320 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.824+0000 7f96d28cc640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd80194320 0x55fd80196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d28cc640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd80194320 0x55fd80196710 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d6a0e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d6a0e800 20 searching for the correct realm 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d28cc640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f96cc04d990 0x7f96cc0203a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.825+0000 7f96d20cb640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f96cc04d990 0x7f96cc0203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.826+0000 7f96d20cb640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f96cc04d990 0x7f96cc0203a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.826+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.826+0000 7f96d20cb640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f96c4017050 0x7f96c400d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.826+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96c4017050 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.826+0000 7f96cbfff640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f96c4017050 0x7f96c400d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.827+0000 7f96cbfff640 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f96c4017050 0x7f96c400d3d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.827+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.827+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.827+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.827+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.827+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.827+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96c4017050 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96c4017050 2024-09-19T11:25:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.828+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.829+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.829+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96c4017050 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.829+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.829+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.829+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.829+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.830+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.831+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96c4017050 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.832+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96c4017050 2024-09-19T11:25:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.833+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.833+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.833+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.833+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96c4017050 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.833+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.833+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd80194320 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x55fd8018f980 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.834+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55fd801985c0 con 0x7f96cc04d990 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd801976c0 con 0x55fd8018f980 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80197f60 con 0x55fd8018f980 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80197300 con 0x55fd80194320 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 zone default found 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 4 Realm: () 2024-09-19T11:25:07.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 zonegroup default 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.835+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd80198f40 con 0x7f96c4017050 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96d6a0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019aa60 con 0x55fd80194320 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96d6a0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.836+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019aef0 con 0x55fd80194320 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.837+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.837+0000 7f96d6a0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.837+0000 7f96d6a0e800 20 started sync module instance, tier type = 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.837+0000 7f96d6a0e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.837+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd80199d70 con 0x7f96cc04d990 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.838+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.838+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94547264248000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019b350 con 0x7f96cc04d990 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.838+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019bba0 con 0x55fd8018f980 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.839+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94547264248000] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.839+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.839+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94547264255728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019bfc0 con 0x55fd8018f980 2024-09-19T11:25:07.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.839+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019c870 con 0x55fd80194320 2024-09-19T11:25:07.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.840+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.840+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94547264257984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019c420 con 0x55fd80194320 2024-09-19T11:25:07.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.840+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019d840 con 0x7f96cc04d990 2024-09-19T11:25:07.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.840+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94547264255728] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.841+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94547264257984] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.842+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.842+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94547264261216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019bb10 con 0x7f96cc04d990 2024-09-19T11:25:07.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.842+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019e140 con 0x55fd8018f980 2024-09-19T11:25:07.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.843+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94547264261216] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.843+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.843+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94547264265264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019e100 con 0x55fd8018f980 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.843+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019eb80 con 0x7f96c4017050 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.844+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.844+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94547264265264] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.844+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94547264267888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019eb40 con 0x7f96c4017050 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.844+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019f620 con 0x7f96c4017050 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.845+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.845+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94547264262960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019f620 con 0x7f96c4017050 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.845+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94547264267888] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.845+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019efa0 con 0x7f96c4017050 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94547264262960] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94547264274480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019fa10 con 0x7f96c4017050 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 20 add_watcher() i=0 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 20 add_watcher() i=1 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 20 add_watcher() i=2 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 20 add_watcher() i=3 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 20 add_watcher() i=4 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 20 add_watcher() i=5 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.846+0000 7f96d6a0e800 20 add_watcher() i=6 2024-09-19T11:25:07.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.847+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94547264274480] v33'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.847+0000 7f96d6a0e800 20 add_watcher() i=7 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.847+0000 7f96d6a0e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.847+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd801a3530 con 0x55fd8018f980 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.848+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.848+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94547264270240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd801a09b0 con 0x55fd8018f980 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94547264270240] v33'230 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f9588fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96d6a0e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96d6a0e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96d6a0e800 5 note: GC not initialized 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f95037fe640 20 reqs_thread_entry: start 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96d6a0e800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:07.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96d6a0e800 20 Filter name: none 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f95017fa640 20 reqs_thread_entry: start 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96d6a0e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.849+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd801d5e10 con 0x55fd8018f980 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.850+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.850+0000 7f96d6a0e800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.850+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55fd801d7230 con 0x7f96c4017050 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.850+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv32 ondisk = 0) ==== 152+0+15 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.851+0000 7f9588fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.851+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94547264270240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd801a0500 con 0x55fd8018f980 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.852+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94547264270240] v33'231 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.852+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94547264248000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019efa0 con 0x7f96cc04d990 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.853+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94547264248000] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.853+0000 7f96d6a0e800 20 remove_watcher() i=0 2024-09-19T11:25:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.853+0000 7f96d6a0e800 2 removed watcher, disabling cache 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.853+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94547264255728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd801998c0 con 0x55fd8018f980 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.854+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94547264255728] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.854+0000 7f96d6a0e800 20 remove_watcher() i=1 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.854+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94547264257984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd801998c0 con 0x55fd80194320 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.855+0000 7f96d28cc640 1 -- 172.21.15.28:0/3650403761 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94547264257984] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96cc053020 con 0x55fd80194320 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.855+0000 7f96d6a0e800 20 remove_watcher() i=2 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.855+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94547264261216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019bfc0 con 0x7f96cc04d990 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.856+0000 7f96d20cb640 1 -- 172.21.15.28:0/3650403761 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94547264261216] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96c4018ce0 con 0x7f96cc04d990 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.856+0000 7f96d6a0e800 20 remove_watcher() i=3 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.856+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94547264265264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd801998c0 con 0x55fd8018f980 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.857+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94547264265264] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x55fd8018f980 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.857+0000 7f96d6a0e800 20 remove_watcher() i=4 2024-09-19T11:25:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.857+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94547264267888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019bfc0 con 0x7f96c4017050 2024-09-19T11:25:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.858+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94547264267888] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.858+0000 7f96d6a0e800 20 remove_watcher() i=5 2024-09-19T11:25:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.858+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94547264262960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd8019dc30 con 0x7f96c4017050 2024-09-19T11:25:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.859+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94547264262960] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.860+0000 7f96d6a0e800 20 remove_watcher() i=6 2024-09-19T11:25:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.860+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94547264274480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55fd801998c0 con 0x7f96c4017050 2024-09-19T11:25:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94547264274480] v33'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96bc051070 con 0x7f96c4017050 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 20 remove_watcher() i=7 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f96c4017050 msgr2=0x7f96c400d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f96c4017050 0x7f96c400d3d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8018f980 msgr2=0x55fd80191dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8018f980 0x55fd80191dd0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f96cc04d990 msgr2=0x7f96cc0203a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f96cc04d990 0x7f96cc0203a0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd80194320 msgr2=0x55fd80196710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd80194320 0x55fd80196710 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f959c03d000 msgr2=0x7f959c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f959c03d000 0x7f959c03f4c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f96c403f010 tx=0x7f96c4046000 comp rx=0 tx=0).stop 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 msgr2=0x55fd80184020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96cbfff640 1 -- 172.21.15.28:0/3650403761 reap_dead start 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.861+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd80184020 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f96cc054780 tx=0x7f96cc0547b0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 shutdown_connections 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3650403761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8017ca60 0x55fd80184020 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 >> 172.21.15.28:0/3650403761 conn(0x55fd80178e60 msgr2=0x55fd80177070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 shutdown_connections 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3650403761 wait complete. 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95ac03d050 msgr2=0x7f95ac03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95ac03d050 0x7f95ac03f510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55fd80157160 tx=0x7f96c4052a40 comp rx=0 tx=0).stop 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8016ffe0 msgr2=0x55fd80170400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8016ffe0 0x55fd80170400 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x55fd80155ef0 tx=0x7f96cc0167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 shutdown_connections 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95ac03d050 0x7f95ac03f510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/3960877055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd8016ffe0 0x55fd80170400 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 >> 172.21.15.28:0/3960877055 conn(0x55fd80164390 msgr2=0x55fd80164770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 shutdown_connections 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/3960877055 wait complete. 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8005b870 msgr2=0x55fd8015c6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8005b870 0x55fd8015c6b0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd8015ea50 msgr2=0x55fd80160ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.862+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd8015ea50 0x55fd80160ef0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95b803cfb0 msgr2=0x7f95b803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95b803cfb0 0x7f95b803f470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f96bc057000 tx=0x7f96bc052000 comp rx=0 tx=0).stop 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 msgr2=0x55fd80155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd80155060 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f96c40012b0 tx=0x7f96c4000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 shutdown_connections 2024-09-19T11:25:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55fd8015ea50 0x55fd80160ef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f95b803cfb0 0x7f95b803f470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55fd8005b870 0x55fd8015c6b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 --2- 172.21.15.28:0/2687097255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fd80059d50 0x55fd80155060 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 >> 172.21.15.28:0/2687097255 conn(0x55fd80055a60 msgr2=0x55fd8014ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 shutdown_connections 2024-09-19T11:25:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.863+0000 7f96d6a0e800 1 -- 172.21.15.28:0/2687097255 wait complete. 2024-09-19T11:25:07.874 INFO:teuthology.orchestra.run.smithi028.stdout:{"entries":[],"summary":[]} 2024-09-19T11:25:07.874 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-09-19T11:25:07.874 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-09-19T11:25:07.874 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-09-19T11:25:07.875 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-09-19T11:25:07.921 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:07.921 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.920+0000 7f3c57a10800 1 Processor -- start 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.920+0000 7f3c57a10800 1 -- start start 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c57a10800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562177e1fd30 0x562177e20130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c57a10800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562177e21230 con 0x562177e20670 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c57a10800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x562177cac7d0 con 0x562177e1fd30 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c530cb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562177e1fd30 0x562177e20130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c528ca640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x562177e20670 0x562177e20b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48212/0 (socket says 172.21.15.28:48212) 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c530cb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x562177e1fd30 0x562177e20130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44426/0 (socket says 172.21.15.28:44426) 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c528ca640 1 -- 172.21.15.28:0/1182136959 learned_addr learned my addr 172.21.15.28:0/1182136959 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c530cb640 1 -- 172.21.15.28:0/1182136959 >> v1:172.21.15.22:6789/0 conn(0x562177e20670 legacy=0x562177e20b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.921+0000 7f3c530cb640 1 -- 172.21.15.28:0/1182136959 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562177e21400 con 0x562177e1fd30 2024-09-19T11:25:07.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.923+0000 7f3c530cb640 1 --2- 172.21.15.28:0/1182136959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177e20130 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400a470 tx=0x7f3c44045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f3959be7f9c4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.923+0000 7f3c520c9640 1 -- 172.21.15.28:0/1182136959 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c44053570 con 0x562177e1fd30 2024-09-19T11:25:07.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.923+0000 7f3c520c9640 1 -- 172.21.15.28:0/1182136959 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c44058070 con 0x562177e1fd30 2024-09-19T11:25:07.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.923+0000 7f3c520c9640 1 -- 172.21.15.28:0/1182136959 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c44053880 con 0x562177e1fd30 2024-09-19T11:25:07.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.924+0000 7f3c57a10800 1 -- 172.21.15.28:0/1182136959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 msgr2=0x562177e20130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.924+0000 7f3c57a10800 1 --2- 172.21.15.28:0/1182136959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177e20130 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400a470 tx=0x7f3c44045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:07.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.924+0000 7f3c57a10800 1 -- 172.21.15.28:0/1182136959 shutdown_connections 2024-09-19T11:25:07.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.924+0000 7f3c57a10800 1 --2- 172.21.15.28:0/1182136959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177e20130 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.924+0000 7f3c57a10800 1 -- 172.21.15.28:0/1182136959 >> 172.21.15.28:0/1182136959 conn(0x562177e1ba60 msgr2=0x562177e1dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.924+0000 7f3c57a10800 1 -- 172.21.15.28:0/1182136959 shutdown_connections 2024-09-19T11:25:07.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.924+0000 7f3c57a10800 1 -- 172.21.15.28:0/1182136959 wait complete. 2024-09-19T11:25:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.939+0000 7f3c57a10800 1 Processor -- start 2024-09-19T11:25:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.939+0000 7f3c57a10800 1 -- start start 2024-09-19T11:25:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.939+0000 7f3c57a10800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f13540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.939+0000 7f3c57a10800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562177cb9dd0 con 0x562177e1fd30 2024-09-19T11:25:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.939+0000 7f3c530cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f13540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.939+0000 7f3c530cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f13540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44442/0 (socket says 172.21.15.28:44442) 2024-09-19T11:25:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.939+0000 7f3c530cb640 1 -- 172.21.15.28:0/537343039 learned_addr learned my addr 172.21.15.28:0/537343039 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.940+0000 7f3c530cb640 1 -- 172.21.15.28:0/537343039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562177f13b30 con 0x562177e1fd30 2024-09-19T11:25:07.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.940+0000 7f3c530cb640 1 --2- 172.21.15.28:0/537343039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f13540 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3c44025c20 tx=0x7f3c44051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7a8ed1d0a7492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.940+0000 7f3b4160a640 1 -- 172.21.15.28:0/537343039 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4405c580 con 0x562177e1fd30 2024-09-19T11:25:07.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.940+0000 7f3b4160a640 1 -- 172.21.15.28:0/537343039 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c44063030 con 0x562177e1fd30 2024-09-19T11:25:07.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.940+0000 7f3b4160a640 1 -- 172.21.15.28:0/537343039 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4405c850 con 0x562177e1fd30 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 -- 172.21.15.28:0/537343039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 msgr2=0x562177f13540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 --2- 172.21.15.28:0/537343039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f13540 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3c44025c20 tx=0x7f3c44051000 comp rx=0 tx=0).stop 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 -- 172.21.15.28:0/537343039 shutdown_connections 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 --2- 172.21.15.28:0/537343039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f13540 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 -- 172.21.15.28:0/537343039 >> 172.21.15.28:0/537343039 conn(0x562177e1ba60 msgr2=0x562177f0eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 -- 172.21.15.28:0/537343039 shutdown_connections 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 -- 172.21.15.28:0/537343039 wait complete. 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 Processor -- start 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 -- start start 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f1af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.941+0000 7f3c57a10800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562177e21990 con 0x562177e1fd30 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3c530cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f1af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3c530cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f1af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44454/0 (socket says 172.21.15.28:44454) 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3c530cb640 1 -- 172.21.15.28:0/767184206 learned_addr learned my addr 172.21.15.28:0/767184206 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3c530cb640 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562177f1b4e0 con 0x562177e1fd30 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3c530cb640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f1af80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3c44045f50 tx=0x7f3c44002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3b3fe07640 1 -- 172.21.15.28:0/767184206 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c44054ad0 con 0x562177e1fd30 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3b3fe07640 1 -- 172.21.15.28:0/767184206 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c44054c70 con 0x562177e1fd30 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562177f1b770 con 0x562177e1fd30 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3b3fe07640 1 -- 172.21.15.28:0/767184206 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c44068420 con 0x562177e1fd30 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.942+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562177f1c170 con 0x562177e1fd30 2024-09-19T11:25:07.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.943+0000 7f3b3fe07640 1 -- 172.21.15.28:0/767184206 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3c440685c0 con 0x562177e1fd30 2024-09-19T11:25:07.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.943+0000 7f3b3fe07640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b3803d000 0x7f3b3803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.943+0000 7f3b3fe07640 1 -- 172.21.15.28:0/767184206 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3c44070030 con 0x562177e1fd30 2024-09-19T11:25:07.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c528ca640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b3803d000 0x7f3b3803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177e20670 0x562177f225d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f22b10 con 0x562177e20670 2024-09-19T11:25:07.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c538cc640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177e20670 0x562177f225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c528ca640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b3803d000 0x7f3b3803f4c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3c48001d70 tx=0x7f3c480012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c538cc640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177e20670 0x562177f225d0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c538cc640 1 -- 172.21.15.28:0/767184206 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177e20670 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.944+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f22eb0 con 0x562177e20670 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.945+0000 7f3c538cc640 1 -- 172.21.15.28:0/767184206 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177e20670 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.945+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f249f0 0x562177f26e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.945+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f275f0 con 0x562177f249f0 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.945+0000 7f3c530cb640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f249f0 0x562177f26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.945+0000 7f3c530cb640 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f249f0 0x562177f26e60 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.945+0000 7f3c530cb640 1 -- 172.21.15.28:0/767184206 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3c44051070 con 0x562177f249f0 2024-09-19T11:25:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.945+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f23700 con 0x562177e20670 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.946+0000 7f3c538cc640 1 -- 172.21.15.28:0/767184206 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177e20670 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.946+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f23b20 con 0x562177f249f0 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.946+0000 7f3c530cb640 1 -- 172.21.15.28:0/767184206 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3c44051070 con 0x562177f249f0 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.946+0000 7f3c57a10800 1 Processor -- start 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.946+0000 7f3c57a10800 1 -- start start 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.947+0000 7f3c57a10800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f35f50 0x562177f36370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.947+0000 7f3c57a10800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562177d184a0 con 0x562177f35f50 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.947+0000 7f3c538cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f35f50 0x562177f36370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.947+0000 7f3c538cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f35f50 0x562177f36370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44458/0 (socket says 172.21.15.28:44458) 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.947+0000 7f3c538cc640 1 -- 172.21.15.28:0/2905614858 learned_addr learned my addr 172.21.15.28:0/2905614858 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.947+0000 7f3c538cc640 1 -- 172.21.15.28:0/2905614858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562177f369b0 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.947+0000 7f3c538cc640 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f35f50 0x562177f36370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x562177f1c340 tx=0x7f3c4c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.948+0000 7f3b3d602640 1 -- 172.21.15.28:0/2905614858 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c013640 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.948+0000 7f3b3d602640 1 -- 172.21.15.28:0/2905614858 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c0137e0 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.948+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562177f371e0 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.948+0000 7f3b3d602640 1 -- 172.21.15.28:0/2905614858 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c013b00 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.948+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562177f37780 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3b3d602640 1 -- 172.21.15.28:0/2905614858 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3c4c011040 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3b3d602640 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b2c03d000 0x7f3b2c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3b3d602640 1 -- 172.21.15.28:0/2905614858 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3c4c00e040 con 0x562177f35f50 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3c530cb640 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b2c03d000 0x7f3b2c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3c57a10800 1 Processor -- start 2024-09-19T11:25:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3c57a10800 1 -- start start 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3c530cb640 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b2c03d000 0x7f3b2c03f4c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x562177f27990 tx=0x7f3c44074f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3c57a10800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f42dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.949+0000 7f3c57a10800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562177f13f00 con 0x562177f429d0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3c538cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f42dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3c538cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f42dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44460/0 (socket says 172.21.15.28:44460) 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3c538cc640 1 -- 172.21.15.28:0/3001564745 learned_addr learned my addr 172.21.15.28:0/3001564745 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3c538cc640 1 -- 172.21.15.28:0/3001564745 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562177f433c0 con 0x562177f429d0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3c538cc640 1 --2- 172.21.15.28:0/3001564745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f42dd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0548b0 tx=0x7f3c4c054a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd9503b08e32a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3b36ffd640 1 -- 172.21.15.28:0/3001564745 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c044070 con 0x562177f429d0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3b36ffd640 1 -- 172.21.15.28:0/3001564745 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c013ca0 con 0x562177f429d0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.950+0000 7f3b36ffd640 1 -- 172.21.15.28:0/3001564745 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c021440 con 0x562177f429d0 2024-09-19T11:25:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 -- 172.21.15.28:0/3001564745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 msgr2=0x562177f42dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 --2- 172.21.15.28:0/3001564745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f42dd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0548b0 tx=0x7f3c4c054a80 comp rx=0 tx=0).stop 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 -- 172.21.15.28:0/3001564745 shutdown_connections 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 --2- 172.21.15.28:0/3001564745 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f42dd0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 -- 172.21.15.28:0/3001564745 >> 172.21.15.28:0/3001564745 conn(0x562177f3edd0 msgr2=0x562177f3d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 -- 172.21.15.28:0/3001564745 shutdown_connections 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 -- 172.21.15.28:0/3001564745 wait complete. 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 Processor -- start 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 -- start start 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f4a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c57a10800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562177f439b0 con 0x562177f429d0 2024-09-19T11:25:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c538cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c538cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f4a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44476/0 (socket says 172.21.15.28:44476) 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.951+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 learned_addr learned my addr 172.21.15.28:0/4082147205 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.952+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562177f4a560 con 0x562177f429d0 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.952+0000 7f3c538cc640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f4a000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c020930 tx=0x7f3c4c054810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.952+0000 7f3b357fa640 1 -- 172.21.15.28:0/4082147205 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c01fca0 con 0x562177f429d0 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.952+0000 7f3b357fa640 1 -- 172.21.15.28:0/4082147205 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4c056070 con 0x562177f429d0 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.952+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562177f4a7f0 con 0x562177f429d0 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.952+0000 7f3b357fa640 1 -- 172.21.15.28:0/4082147205 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4c056370 con 0x562177f429d0 2024-09-19T11:25:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.952+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562177f4b380 con 0x562177f429d0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.953+0000 7f3b357fa640 1 -- 172.21.15.28:0/4082147205 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3c4c01f500 con 0x562177f429d0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.953+0000 7f3b357fa640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b1c03d000 0x7f3b1c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.953+0000 7f3b357fa640 1 -- 172.21.15.28:0/4082147205 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3c4c004600 con 0x562177f429d0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.953+0000 7f3c530cb640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b1c03d000 0x7f3b1c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.954+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.954+0000 7f3c530cb640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b1c03d000 0x7f3b1c03f4c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009800 tx=0x7f3c44092000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.954+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177f558a0 0x562177f57d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.954+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f584b0 con 0x562177f558a0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.954+0000 7f3c528ca640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177f558a0 0x562177f57d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.954+0000 7f3c528ca640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177f558a0 0x562177f57d60 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c48009050 con 0x562177f558a0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 realm 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f54bf0 con 0x562177f558a0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c48009050 con 0x562177f558a0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f58fa0 con 0x562177f558a0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c48009050 con 0x562177f558a0 2024-09-19T11:25:07.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f59870 con 0x562177f558a0 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3c48009050 con 0x562177f558a0 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f5a290 0x562177f5c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.955+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f5cdc0 con 0x562177f5a290 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c538cc640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f5a290 0x562177f5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c538cc640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f5a290 0x562177f5c680 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c57a10800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c57a10800 20 searching for the correct realm 2024-09-19T11:25:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c538cc640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3c4c045a50 0x7f3c4c033940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.956+0000 7f3c530cb640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3c4c045a50 0x7f3c4c033940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.957+0000 7f3c530cb640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3c4c045a50 0x7f3c4c033940 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.957+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.957+0000 7f3c530cb640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3c440a6040 0x7f3c44095190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.957+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c440a6040 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.957+0000 7f3c528ca640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3c440a6040 0x7f3c44095190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.958+0000 7f3c528ca640 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3c440a6040 0x7f3c44095190 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.958+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48009050 con 0x7f3c440a6040 2024-09-19T11:25:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.958+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.958+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48009050 con 0x562177f558a0 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.958+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c440a6040 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c440a6040 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.959+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c440a6040 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.960+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.961+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.962+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c440a6040 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.963+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c440a6040 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.964+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.964+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.964+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.964+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c440a6040 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.964+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.964+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f5a290 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x562177f558a0 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.965+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x562177f5e6f0 con 0x7f3c4c045a50 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f5d780 con 0x562177f558a0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f5dfd0 con 0x562177f558a0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f5eb70 con 0x562177f5a290 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 zone default found 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 4 Realm: () 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 zonegroup default 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.966+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f5ef10 con 0x7f3c440a6040 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f611f0 con 0x562177f5a290 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f615b0 con 0x562177f5a290 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 20 started sync module instance, tier type = 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.967+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f60680 con 0x7f3c4c045a50 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.968+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.968+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94701746522736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f602e0 con 0x7f3c4c045a50 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.968+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f623b0 con 0x562177f558a0 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.969+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94701746522736] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.970+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.970+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94701746526736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f627a0 con 0x562177f558a0 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.970+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f631f0 con 0x562177f5a290 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.971+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94701746526736] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.971+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.971+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94701746531552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f631b0 con 0x562177f5a290 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.971+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f63ca0 con 0x7f3c4c045a50 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.972+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94701746531552] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.972+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.973+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94701746534320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f64090 con 0x7f3c4c045a50 2024-09-19T11:25:07.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.973+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f64ad0 con 0x562177f558a0 2024-09-19T11:25:07.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.974+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94701746534320] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.974+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.974+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94701746537888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f64ec0 con 0x562177f558a0 2024-09-19T11:25:07.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.974+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f658e0 con 0x7f3c440a6040 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.975+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94701746537888] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.975+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.975+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94701746541520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f658a0 con 0x7f3c440a6040 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.975+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f66310 con 0x7f3c440a6040 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.976+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94701746541520] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.976+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.976+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94701746544192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f66310 con 0x7f3c440a6040 2024-09-19T11:25:07.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.976+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f66d20 con 0x7f3c440a6040 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94701746544192] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94701746527792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f66d20 con 0x7f3c440a6040 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 20 add_watcher() i=0 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 20 add_watcher() i=1 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 20 add_watcher() i=2 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 20 add_watcher() i=3 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 20 add_watcher() i=4 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 20 add_watcher() i=5 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.977+0000 7f3c57a10800 20 add_watcher() i=6 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.978+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94701746527792] v33'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.978+0000 7f3c57a10800 20 add_watcher() i=7 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.978+0000 7f3c57a10800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.978+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f6a120 con 0x562177f558a0 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.978+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.978+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94701746547264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f67640 con 0x562177f558a0 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.979+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94701746547264] v33'232 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.980+0000 7f3b09fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.980+0000 7f3c57a10800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.980+0000 7f3c57a10800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.980+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f72960 con 0x562177f558a0 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.981+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.981+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f72ff0 con 0x7f3c440a6040 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.982+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.982+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f73430 con 0x562177f5a290 2024-09-19T11:25:07.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.983+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.983+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f73390 con 0x562177f558a0 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.984+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'233 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.984+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f73920 con 0x7f3c4c045a50 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.985+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'74 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.985+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f73db0 con 0x562177f5a290 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.986+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'49 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.986+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f73db0 con 0x562177f558a0 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.987+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.987+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f73d10 con 0x7f3c440a6040 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.988+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'49 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.988+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f74150 con 0x7f3c4c045a50 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.989+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'29 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.989+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f740f0 con 0x562177f5a290 2024-09-19T11:25:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.990+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.990+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f740a0 con 0x7f3c4c045a50 2024-09-19T11:25:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.991+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'75 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.991+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f74550 con 0x7f3c440a6040 2024-09-19T11:25:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.992+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'34 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.992+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f74550 con 0x7f3c4c045a50 2024-09-19T11:25:07.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.993+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:07.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.993+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f74550 con 0x7f3c440a6040 2024-09-19T11:25:07.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.994+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'56 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:07.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.994+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f707a0 con 0x562177f558a0 2024-09-19T11:25:08.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.995+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'234 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:08.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.995+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f707a0 con 0x562177f5a290 2024-09-19T11:25:08.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.996+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'68 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.996+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f707a0 con 0x7f3c4c045a50 2024-09-19T11:25:08.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.997+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'76 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.997+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f707a0 con 0x562177f5a290 2024-09-19T11:25:08.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.998+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.998+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f707a0 con 0x562177f558a0 2024-09-19T11:25:08.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:07.999+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'36 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:08.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.000+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f558a0 2024-09-19T11:25:08.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.001+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'235 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:08.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.001+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f5a290 2024-09-19T11:25:08.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.002+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'69 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.002+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x7f3c4c045a50 2024-09-19T11:25:08.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.002+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'38 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.003+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f5a290 2024-09-19T11:25:08.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.003+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.003+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f5a290 2024-09-19T11:25:08.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.004+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.004+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f5a290 2024-09-19T11:25:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.005+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'24 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.005+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f5a290 2024-09-19T11:25:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.006+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'50 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.006+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x7f3c440a6040 2024-09-19T11:25:08.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.007+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'32 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:08.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.007+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f5a290 2024-09-19T11:25:08.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.008+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.008+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x7f3c440a6040 2024-09-19T11:25:08.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.010+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'50 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:08.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.010+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x562177f5a290 2024-09-19T11:25:08.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.011+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'70 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.011+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x7f3c440a6040 2024-09-19T11:25:08.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.012+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'57 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:08.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.012+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177f705d0 con 0x7f3c4c045a50 2024-09-19T11:25:08.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3b08fe1640 20 reqs_thread_entry: start 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3c57a10800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3c57a10800 20 Filter name: none 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3a827fc640 20 reqs_thread_entry: start 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3c57a10800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.013+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177f9f1c0 con 0x562177f558a0 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv17 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 20 rados_obj.operate() r=0 bl.length=391 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-19T11:25:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fa1af0 con 0x7f3c4c045a50 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv43 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 : miss 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:96 8.2 8:45c8e818:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fa1800 con 0x7f3c4c045a50 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(96 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 [call out=48b,stat out=16b,read 0~387 out=387b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+669 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 20 rados_obj.operate() r=0 bl.length=387 2024-09-19T11:25:08.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 info.flags=0x17 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 to cache LRU end 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.014+0000 7f3c57a10800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.015+0000 7f3c57a10800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.015+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:97 4.b 4:d09f13bc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801eda0 con 0x7f3c4c045a50 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.015+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(97 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.015+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:98 4.1c 4:39c28f0e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801eda0 con 0x7f3c440a6040 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.015+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(98 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.015+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 4.4 4:22cffa8d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801eda0 con 0x7f3c440a6040 2024-09-19T11:25:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.016+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(99 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.016+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:100 4.b 4:d1264dc0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801eda0 con 0x7f3c4c045a50 2024-09-19T11:25:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.016+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(100 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3 [call out=89b] v0'0 uv4 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.016+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:101 4.10 4:0806c477:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801ede0 con 0x562177f5a290 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.017+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(101 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4 [call out=429b] v0'0 uv3 ondisk = 0) ==== 194+0+429 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.017+0000 7f3c57a10800 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.017+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:102 4.b 4:d0c5dc53:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801ee10 con 0x7f3c4c045a50 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.017+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(102 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.017+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:103 4.8 4:10e9943a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801ee90 con 0x562177f5a290 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.017+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(103 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.017+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:104 4.37 4:ef660143:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801ee90 con 0x7f3c440a6040 2024-09-19T11:25:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.018+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(104 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.018+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:105 4.18 4:1923d0aa:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801ee90 con 0x7f3c4c045a50 2024-09-19T11:25:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.018+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(105 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.018+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:106 4.2a 4:558ed01e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801ee90 con 0x562177f558a0 2024-09-19T11:25:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.018+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(106 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48029040 con 0x562177f558a0 2024-09-19T11:25:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.018+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:107 4.11 4:8a5a6a4b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56217801ee90 con 0x7f3c440a6040 2024-09-19T11:25:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(107 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f3c48029040 con 0x7f3c440a6040 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 20 get_obj_state: octx=0x7f3c4405ec20 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x56217801ce48 s->prefetch_data=0 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:108 3.3ds0 3:bfd14998:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.7_15f6f8dc036519d7fe15b39338f6e5db:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fa2370 con 0x562177f5a290 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(108 293899f1-2738-4b64-a8e1-a18ff039d626.4229.7_15f6f8dc036519d7fe15b39338f6e5db [call out=24b,getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 304+0+917 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 10 manifest: total_size = 6 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 20 get_obj_state: setting s->obj_tag to 293899f1-2738-4b64-a8e1-a18ff039d626.4229.2546178584306785787 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 20 get_obj_state: octx=0x7f3c4405ec20 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x56217801ce48 s->prefetch_data=0 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 20 prepare_atomic_modification: state is not atomic. state=0x56217801ce48 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 20 bucket index object: :.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.019+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:109 4.10 4:0806c477:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fa6040 con 0x562177f5a290 2024-09-19T11:25:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.020+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(109 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4 [stat,call,call] v33'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.020+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:110 3.3ds0 3:bfd14998:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.7_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fa6040 con 0x562177f5a290 2024-09-19T11:25:08.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(110 293899f1-2738-4b64-a8e1-a18ff039d626.4229.7_15f6f8dc036519d7fe15b39338f6e5db [call] v33'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:111 4.10 4:0806c477:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fa6ad0 con 0x562177f5a290 2024-09-19T11:25:08.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 20 list_objects_ordered: starting attempt 1 2024-09-19T11:25:08.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:112 4.b 4:d09f13bc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f3c4c074000 con 0x7f3c4c045a50 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:113 4.1c 4:39c28f0e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fa6ad0 con 0x7f3c440a6040 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:114 4.4 4:22cffa8d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fad270 con 0x7f3c440a6040 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:115 4.b 4:d1264dc0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fafce0 con 0x7f3c4c045a50 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:116 4.10 4:0806c477:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb2590 con 0x562177f5a290 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:117 4.b 4:d0c5dc53:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5370 con 0x7f3c4c045a50 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:118 4.8 4:10e9943a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb81f0 con 0x562177f5a290 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:119 4.37 4:ef660143:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fbb020 con 0x7f3c440a6040 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:120 4.18 4:1923d0aa:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fbdb30 con 0x7f3c4c045a50 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:121 4.2a 4:558ed01e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fc0900 con 0x562177f558a0 2024-09-19T11:25:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:122 4.11 4:8a5a6a4b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fc32c0 con 0x7f3c440a6040 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.022+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(112 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(115 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3 [call out=89b] v0'0 uv4 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(120 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(113 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(114 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(119 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(122 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(118 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(121 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(117 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(111 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4 [stat,call,call] v33'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3b367fc640 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(116 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4 [call out=128b] v0'0 uv7 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[293899f1-2738-4b64-a8e1-a18ff039d626.4229.7])"; uploads.size()=0; is_truncated=0 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:123 4.b 4:d09f13bc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fc0880 con 0x7f3c4c045a50 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:124 4.1c 4:39c28f0e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb4d20 con 0x7f3c440a6040 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.023+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:125 4.4 4:22cffa8d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fc0880 con 0x7f3c440a6040 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:126 4.b 4:d1264dc0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb4d20 con 0x7f3c4c045a50 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:127 4.10 4:0806c477:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fbdb30 con 0x562177f5a290 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:128 4.b 4:d0c5dc53:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb96d0 con 0x7f3c4c045a50 2024-09-19T11:25:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:129 4.8 4:10e9943a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb4d20 con 0x562177f5a290 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:130 4.37 4:ef660143:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb96d0 con 0x7f3c440a6040 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:131 4.18 4:1923d0aa:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb7d60 con 0x7f3c4c045a50 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:132 4.2a 4:558ed01e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb1740 con 0x562177f558a0 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:133 4.11 4:8a5a6a4b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb1ff0 con 0x7f3c440a6040 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== osd_op_reply(123 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(124 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(127 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4 [call out=128b] v0'0 uv7 ondisk = 0) ==== 194+0+128 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(126 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3 [call out=89b] v0'0 uv4 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(133 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10 [call out=89b] v0'0 uv1 ondisk = 0) ==== 195+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(131 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(125 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2 [call out=89b] v0'0 uv1 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(130 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(129 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(132 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9 [call out=89b] v0'0 uv3 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== osd_op_reply(128 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5 [call out=89b] v0'0 uv5 ondisk = 0) ==== 194+0+89 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.024+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:134 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb4ef0 con 0x7f3c4c045a50 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.025+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(134 foo.buckets [call] v33'46 uv46 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.025+0000 7f3c57a10800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.025+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.025+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:135 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fa65e0 con 0x7f3c4c045a50 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.026+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(135 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv40 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.026+0000 7f3c57a10800 20 rados_obj.operate() r=0 bl.length=165 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.026+0000 7f3c57a10800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.026+0000 7f3c57a10800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-19T11:25:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.026+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:136 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 36 ==== osd_op_reply(136 myfoo [call,call,delete] v33'47 uv47 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3c57a10800 10 removing default.rgw.meta+root+myfoo from cache 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3c57a10800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:137 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94701746808288 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c440a6040 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== watch-notify(notify (1) cookie 94701746544192 notify 141733920809 ret 0) ==== 191+0+0 (crc 0 0 0) 0x562177f70c20 con 0x7f3c440a6040 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3b36ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920809 cookie 94701746544192 notifier 4782 bl.length()=149 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(137 notify.6 [notify cookie 94701746808288 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.027+0000 7f3b36ffd640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:138 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f3b18003d60 con 0x7f3c440a6040 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.028+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== osd_op_reply(138 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.028+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== watch-notify(notify_complete (2) cookie 94701746808288 notify 141733920809 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562177f4a7f0 con 0x7f3c440a6040 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.028+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:139 8.2 8:45c8e818:root::.bucket.meta.myfoo%3a293899f1-2738-4b64-a8e1-a18ff039d626.4229.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 37 ==== osd_op_reply(139 .bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 [call,call,delete] v33'4 uv4 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3c57a10800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 from cache 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3c57a10800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:293899f1-2738-4b64-a8e1-a18ff039d626.4229.7, ofs0, ns] 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:140 7.3 7:c609908c:::notify.5:head [notify cookie 94701746808288 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c440a6040 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== watch-notify(notify (1) cookie 94701746541520 notify 141733920810 ret 0) ==== 248+0+0 (crc 0 0 0) 0x562177f4a7f0 con 0x7f3c440a6040 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3b36ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920810 cookie 94701746541520 notifier 4782 bl.length()=206 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3b36ffd640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:141 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f3b18003d60 con 0x7f3c440a6040 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(140 notify.5 [notify cookie 94701746808288 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.029+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== osd_op_reply(141 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== watch-notify(notify_complete (2) cookie 94701746808288 notify 141733920810 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562177f72af0 con 0x7f3c440a6040 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:142 4.b 4:d09f13bc:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:143 4.1c 4:39c28f0e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb8660 con 0x7f3c440a6040 2024-09-19T11:25:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:144 4.4 4:22cffa8d:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb8eb0 con 0x7f3c440a6040 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:145 4.b 4:d1264dc0:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb92d0 con 0x7f3c4c045a50 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:146 4.10 4:0806c477:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fbbe50 con 0x562177f5a290 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:147 4.b 4:d0c5dc53:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb8660 con 0x7f3c4c045a50 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:148 4.8 4:10e9943a:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f5a290 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:149 4.37 4:ef660143:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb8eb0 con 0x7f3c440a6040 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:150 4.18 4:1923d0aa:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb92d0 con 0x7f3c4c045a50 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:151 4.2a 4:558ed01e:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb8660 con 0x562177f558a0 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.030+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:152 4.11 4:8a5a6a4b:::.dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb92d0 con 0x7f3c440a6040 2024-09-19T11:25:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.031+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== osd_op_reply(149 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.7 [delete] v33'12 uv12 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.031+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 38 ==== osd_op_reply(150 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.8 [delete] v33'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.031+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 39 ==== osd_op_reply(142 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.0 [delete] v33'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 38 ==== osd_op_reply(146 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.4 [delete] v33'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 39 ==== osd_op_reply(148 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.6 [delete] v33'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(144 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.2 [delete] v33'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== osd_op_reply(143 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.1 [delete] v33'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== osd_op_reply(152 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.10 [delete] v33'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 40 ==== osd_op_reply(145 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.3 [delete] v33'9 uv9 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 41 ==== osd_op_reply(147 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.5 [delete] v33'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(151 .dir.293899f1-2738-4b64-a8e1-a18ff039d626.4229.7.9 [delete] v33'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c57a10800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 : miss 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c57a10800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:153 6.1a 6:5a23dcfd:::pubsub..bucket.myfoo%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f5a290 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 40 ==== osd_op_reply(153 pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c57a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c57a10800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 info.flags=0x0 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c57a10800 10 adding default.rgw.log++pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 to cache LRU end 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.032+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:154 6.1a 6:5a23dcfd:::pubsub..bucket.myfoo%2f293899f1-2738-4b64-a8e1-a18ff039d626.4229.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f5a290 2024-09-19T11:25:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.033+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 41 ==== osd_op_reply(154 pubsub..bucket.myfoo/293899f1-2738-4b64-a8e1-a18ff039d626.4229.7 [delete] v33'30 uv28 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.033+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:155 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.034+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 42 ==== osd_op_reply(155 foo.buckets [call] v33'48 uv48 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.034+0000 7f3c57a10800 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.035+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:156 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c440a6040 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 46 ==== osd_op_reply(156 9te6NH5mcdcq0Tc5i8i1 [delete] v33'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c57a10800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:157 7.e 7:7759931f:::notify.3:head [notify cookie 94701746808288 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 43 ==== watch-notify(notify (1) cookie 94701746534320 notify 141733920797 ret 0) ==== 212+0+0 (crc 0 0 0) 0x562177fa0e00 con 0x7f3c4c045a50 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3b367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920797 cookie 94701746534320 notifier 4782 bl.length()=170 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 44 ==== osd_op_reply(157 notify.3 [notify cookie 94701746808288 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3b367fc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:158 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f3a880032c0 con 0x7f3c4c045a50 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 45 ==== watch-notify(notify_complete (2) cookie 94701746808288 notify 141733920797 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f3c440b0060 con 0x7f3c4c045a50 2024-09-19T11:25:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c57a10800 10 removing email index: foo@foo.com 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 46 ==== osd_op_reply(158 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.036+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:159 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f558a0 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.037+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(159 foo@foo.com [delete] v33'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.037+0000 7f3c57a10800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.037+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:160 7.e 7:7759931f:::notify.3:head [notify cookie 94701746808288 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.037+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 47 ==== watch-notify(notify (1) cookie 94701746534320 notify 141733920798 ret 0) ==== 204+0+0 (crc 0 0 0) 0x7f3c440b0060 con 0x7f3c4c045a50 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.037+0000 7f3b367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920798 cookie 94701746534320 notifier 4782 bl.length()=162 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.037+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 48 ==== osd_op_reply(160 notify.3 [notify cookie 94701746808288 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.037+0000 7f3b367fc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:161 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f3a880032c0 con 0x7f3c4c045a50 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.038+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 49 ==== watch-notify(notify_complete (2) cookie 94701746808288 notify 141733920798 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562177f1c170 con 0x7f3c4c045a50 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.038+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 50 ==== osd_op_reply(161 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.038+0000 7f3c57a10800 10 removing user buckets index 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.038+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:162 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.039+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 51 ==== osd_op_reply(162 foo.buckets [delete] v33'49 uv49 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.039+0000 7f3c57a10800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.039+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94701746808288 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c440a6040 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.039+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 47 ==== watch-notify(notify (1) cookie 94701746541520 notify 141733920811 ret 0) ==== 202+0+0 (crc 0 0 0) 0x562177f72af0 con 0x7f3c440a6040 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.039+0000 7f3b36ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920811 cookie 94701746541520 notifier 4782 bl.length()=160 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.039+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 48 ==== osd_op_reply(163 notify.5 [notify cookie 94701746808288 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.039+0000 7f3b36ffd640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f3b18003d60 con 0x7f3c440a6040 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.040+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 49 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.040+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 50 ==== watch-notify(notify_complete (2) cookie 94701746808288 notify 141733920811 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562177f72920 con 0x7f3c440a6040 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.040+0000 7f3c57a10800 10 removing user index: foo 2024-09-19T11:25:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.040+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:165 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(165 foo [call,call,delete] v33'21 uv21 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c57a10800 10 removing default.rgw.meta+users.uid+foo from cache 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c57a10800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:166 7.d 7:b4812045:::notify.4:head [notify cookie 94701746788032 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== watch-notify(notify (1) cookie 94701746537888 notify 141733920790 ret 0) ==== 194+0+0 (crc 0 0 0) 0x562177f72920 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3b367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920790 cookie 94701746537888 notifier 4782 bl.length()=152 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3b367fc640 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:167 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f3a880032c0 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(166 notify.4 [notify cookie 94701746788032 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== watch-notify(notify_complete (2) cookie 94701746788032 notify 141733920790 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562177f72d00 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.041+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(167 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.042+0000 7f3b09fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.042+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:168 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94701746547264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f558a0 2024-09-19T11:25:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.043+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(168 data_loggenerations_metadata [watch unwatch cookie 94701746547264] v33'236 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.043+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:169 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94701746522736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.044+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 52 ==== osd_op_reply(169 notify.0 [watch unwatch cookie 94701746522736] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.044+0000 7f3c57a10800 20 remove_watcher() i=0 2024-09-19T11:25:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.044+0000 7f3c57a10800 2 removed watcher, disabling cache 2024-09-19T11:25:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.044+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:170 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94701746526736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f558a0 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.045+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(170 notify.1 [watch unwatch cookie 94701746526736] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.045+0000 7f3c57a10800 20 remove_watcher() i=1 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.045+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:171 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94701746531552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f5a290 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.047+0000 7f3c538cc640 1 -- 172.21.15.28:0/4082147205 <== osd.3 v2:172.21.15.28:6808/3027134945 42 ==== osd_op_reply(171 notify.2 [watch unwatch cookie 94701746531552] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4c00a4d0 con 0x562177f5a290 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.047+0000 7f3c57a10800 20 remove_watcher() i=2 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.047+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:172 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94701746534320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c4c045a50 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.048+0000 7f3c530cb640 1 -- 172.21.15.28:0/4082147205 <== osd.2 v2:172.21.15.28:6800/2071226075 53 ==== osd_op_reply(172 notify.3 [watch unwatch cookie 94701746534320] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c4408a050 con 0x7f3c4c045a50 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.048+0000 7f3c57a10800 20 remove_watcher() i=3 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.048+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:173 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94701746537888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x562177f558a0 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.049+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(173 notify.4 [watch unwatch cookie 94701746537888] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x562177f558a0 2024-09-19T11:25:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.049+0000 7f3c57a10800 20 remove_watcher() i=4 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.049+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:174 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94701746541520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c440a6040 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.050+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 51 ==== osd_op_reply(174 notify.5 [watch unwatch cookie 94701746541520] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.050+0000 7f3c57a10800 20 remove_watcher() i=5 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.050+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:175 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94701746544192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c440a6040 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.051+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 52 ==== osd_op_reply(175 notify.6 [watch unwatch cookie 94701746544192] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.051+0000 7f3c57a10800 20 remove_watcher() i=6 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.051+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:176 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94701746527792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x562177fb5060 con 0x7f3c440a6040 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 <== osd.0 v2:172.21.15.22:6800/3144434255 53 ==== osd_op_reply(176 notify.7 [watch unwatch cookie 94701746527792] v33'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3c48046040 con 0x7f3c440a6040 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 20 remove_watcher() i=7 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3c440a6040 msgr2=0x7f3c44095190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f3c440a6040 0x7f3c44095190 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177f558a0 msgr2=0x562177f57d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177f558a0 0x562177f57d60 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3c4c045a50 msgr2=0x7f3c4c033940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f3c4c045a50 0x7f3c4c033940 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f5a290 msgr2=0x562177f5c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f5a290 0x562177f5c680 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b1c03d000 msgr2=0x7f3b1c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b1c03d000 0x7f3b1c03f4c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009800 tx=0x7f3c44092000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 msgr2=0x562177f4a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c528ca640 1 -- 172.21.15.28:0/4082147205 reap_dead start 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f4a000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c020930 tx=0x7f3c4c054810 comp rx=0 tx=0).stop 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 shutdown_connections 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 --2- 172.21.15.28:0/4082147205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f429d0 0x562177f4a000 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 >> 172.21.15.28:0/4082147205 conn(0x562177f3edd0 msgr2=0x562177f3cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 shutdown_connections 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.052+0000 7f3c57a10800 1 -- 172.21.15.28:0/4082147205 wait complete. 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b2c03d000 msgr2=0x7f3b2c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b2c03d000 0x7f3b2c03f4c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x562177f27990 tx=0x7f3c44074f50 comp rx=0 tx=0).stop 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f35f50 msgr2=0x562177f36370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f35f50 0x562177f36370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x562177f1c340 tx=0x7f3c4c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 shutdown_connections 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b2c03d000 0x7f3b2c03f4c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/2905614858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177f35f50 0x562177f36370 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 >> 172.21.15.28:0/2905614858 conn(0x562177f2a300 msgr2=0x562177f2a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 shutdown_connections 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/2905614858 wait complete. 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177e20670 msgr2=0x562177f225d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177e20670 0x562177f225d0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f249f0 msgr2=0x562177f26e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f249f0 0x562177f26e60 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b3803d000 msgr2=0x7f3b3803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b3803d000 0x7f3b3803f4c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3c48001d70 tx=0x7f3c480012d0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 msgr2=0x562177f1af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f1af80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3c44045f50 tx=0x7f3c44002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 shutdown_connections 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x562177f249f0 0x562177f26e60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3b3803d000 0x7f3b3803f4c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x562177e20670 0x562177f225d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 --2- 172.21.15.28:0/767184206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562177e1fd30 0x562177f1af80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 >> 172.21.15.28:0/767184206 conn(0x562177e1ba60 msgr2=0x562177f0d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 shutdown_connections 2024-09-19T11:25:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.053+0000 7f3c57a10800 1 -- 172.21.15.28:0/767184206 wait complete. 2024-09-19T11:25:08.063 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-19T11:25:08.063 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-19T11:25:08.064 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-19T11:25:08.110 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:08.110 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:08.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.108+0000 7fceb1b28800 1 Processor -- start 2024-09-19T11:25:08.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.108+0000 7fceb1b28800 1 -- start start 2024-09-19T11:25:08.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.108+0000 7fceb1b28800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55cfb8150e00 0x55cfb8156750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.108+0000 7fceb1b28800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55cfb81512b0 con 0x55cfb814dce0 2024-09-19T11:25:08.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.108+0000 7fceb1b28800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55cfb7fda7d0 con 0x55cfb8150e00 2024-09-19T11:25:08.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.109+0000 7fcea50d5640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55cfb8150e00 0x55cfb8156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.109+0000 7fcead8d6640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55cfb814dce0 0x55cfb81506f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48216/0 (socket says 172.21.15.28:48216) 2024-09-19T11:25:08.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.109+0000 7fcead8d6640 1 -- 172.21.15.28:0/495761786 learned_addr learned my addr 172.21.15.28:0/495761786 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:08.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.109+0000 7fcea50d5640 1 -- 172.21.15.28:0/495761786 >> v1:172.21.15.22:6789/0 conn(0x55cfb814dce0 legacy=0x55cfb81506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.109+0000 7fcea50d5640 1 -- 172.21.15.28:0/495761786 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb8151480 con 0x55cfb8150e00 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fcea50d5640 1 --2- 172.21.15.28:0/495761786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8150e00 0x55cfb8156750 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fce9800a470 tx=0x7fce98045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7733c20df347b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fcead0d5640 1 -- 172.21.15.28:0/495761786 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce98053570 con 0x55cfb8150e00 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fcead0d5640 1 -- 172.21.15.28:0/495761786 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce98058070 con 0x55cfb8150e00 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fcead0d5640 1 -- 172.21.15.28:0/495761786 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce98053880 con 0x55cfb8150e00 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fceb1b28800 1 -- 172.21.15.28:0/495761786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8150e00 msgr2=0x55cfb8156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fceb1b28800 1 --2- 172.21.15.28:0/495761786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8150e00 0x55cfb8156750 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fce9800a470 tx=0x7fce98045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fceb1b28800 1 -- 172.21.15.28:0/495761786 shutdown_connections 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fceb1b28800 1 --2- 172.21.15.28:0/495761786 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8150e00 0x55cfb8156750 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fceb1b28800 1 -- 172.21.15.28:0/495761786 >> 172.21.15.28:0/495761786 conn(0x55cfb81491c0 msgr2=0x55cfb814b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fceb1b28800 1 -- 172.21.15.28:0/495761786 shutdown_connections 2024-09-19T11:25:08.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.111+0000 7fceb1b28800 1 -- 172.21.15.28:0/495761786 wait complete. 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.126+0000 7fceb1b28800 1 Processor -- start 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.126+0000 7fceb1b28800 1 -- start start 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fceb1b28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb814fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fceb1b28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfb7fe7dd0 con 0x55cfb814dce0 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcead8d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb814fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcead8d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb814fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44494/0 (socket says 172.21.15.28:44494) 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcead8d6640 1 -- 172.21.15.28:0/1026297141 learned_addr learned my addr 172.21.15.28:0/1026297141 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcead8d6640 1 -- 172.21.15.28:0/1026297141 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb8150430 con 0x55cfb814dce0 2024-09-19T11:25:08.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcead8d6640 1 --2- 172.21.15.28:0/1026297141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb814fe40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcea0001e40 tx=0x7fcea0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b990f21ecfd75d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcd9be07640 1 -- 172.21.15.28:0/1026297141 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea0012020 con 0x55cfb814dce0 2024-09-19T11:25:08.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcd9be07640 1 -- 172.21.15.28:0/1026297141 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea000e5a0 con 0x55cfb814dce0 2024-09-19T11:25:08.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.127+0000 7fcd9be07640 1 -- 172.21.15.28:0/1026297141 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea000e8a0 con 0x55cfb814dce0 2024-09-19T11:25:08.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 -- 172.21.15.28:0/1026297141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 msgr2=0x55cfb814fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 --2- 172.21.15.28:0/1026297141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb814fe40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fcea0001e40 tx=0x7fcea0001e70 comp rx=0 tx=0).stop 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 -- 172.21.15.28:0/1026297141 shutdown_connections 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 --2- 172.21.15.28:0/1026297141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb814fe40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 -- 172.21.15.28:0/1026297141 >> 172.21.15.28:0/1026297141 conn(0x55cfb81491c0 msgr2=0x55cfb823aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 -- 172.21.15.28:0/1026297141 shutdown_connections 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 -- 172.21.15.28:0/1026297141 wait complete. 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 Processor -- start 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.128+0000 7fceb1b28800 1 -- start start 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fceb1b28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb82492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fceb1b28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8156f90 con 0x55cfb814dce0 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcead8d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb82492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcead8d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb82492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44508/0 (socket says 172.21.15.28:44508) 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcead8d6640 1 -- 172.21.15.28:0/2346552272 learned_addr learned my addr 172.21.15.28:0/2346552272 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcead8d6640 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb8249810 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcead8d6640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb82492b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c010 tx=0x7fcea0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcd9a604640 1 -- 172.21.15.28:0/2346552272 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea0012030 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcd9a604640 1 -- 172.21.15.28:0/2346552272 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea00014a0 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfb8249aa0 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.129+0000 7fcd9a604640 1 -- 172.21.15.28:0/2346552272 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea001a7a0 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.130+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb824a4a0 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.130+0000 7fcd9a604640 1 -- 172.21.15.28:0/2346552272 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fcea0015020 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fcd9a604640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd9403cfb0 0x7fcd9403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fcd9a604640 1 -- 172.21.15.28:0/2346552272 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fcea0020030 con 0x55cfb814dce0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fcea50d5640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd9403cfb0 0x7fcd9403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8150e00 0x55cfb8250900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8250e40 con 0x55cfb8150e00 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fceae0d7640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8150e00 0x55cfb8250900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fcea50d5640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd9403cfb0 0x7fcd9403f470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce98057000 tx=0x7fce98052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.131+0000 7fceae0d7640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8150e00 0x55cfb8250900 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.132+0000 7fceae0d7640 1 -- 172.21.15.28:0/2346552272 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8150e00 2024-09-19T11:25:08.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.132+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82511e0 con 0x55cfb8150e00 2024-09-19T11:25:08.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.132+0000 7fceae0d7640 1 -- 172.21.15.28:0/2346552272 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8150e00 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.132+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8252ca0 0x55cfb8255140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.132+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82558d0 con 0x55cfb8252ca0 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.132+0000 7fcead8d6640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8252ca0 0x55cfb8255140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.132+0000 7fcead8d6640 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8252ca0 0x55cfb8255140 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.133+0000 7fcead8d6640 1 -- 172.21.15.28:0/2346552272 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fcea001ab20 con 0x55cfb8252ca0 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.133+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8251a30 con 0x55cfb8150e00 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.133+0000 7fceae0d7640 1 -- 172.21.15.28:0/2346552272 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8150e00 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.133+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8251dd0 con 0x55cfb8252ca0 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.133+0000 7fcead8d6640 1 -- 172.21.15.28:0/2346552272 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fcea001ab20 con 0x55cfb8252ca0 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.133+0000 7fceb1b28800 1 Processor -- start 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceb1b28800 1 -- start start 2024-09-19T11:25:08.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceb1b28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8264260 0x55cfb8264680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceb1b28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfb80464a0 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceae0d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8264260 0x55cfb8264680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceae0d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8264260 0x55cfb8264680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44512/0 (socket says 172.21.15.28:44512) 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceae0d7640 1 -- 172.21.15.28:0/4195265515 learned_addr learned my addr 172.21.15.28:0/4195265515 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceae0d7640 1 -- 172.21.15.28:0/4195265515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb8264cc0 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.134+0000 7fceae0d7640 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8264260 0x55cfb8264680 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55cfb824a140 tx=0x7fcea80167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.135+0000 7fcd937fe640 1 -- 172.21.15.28:0/4195265515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea8013640 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.135+0000 7fcd937fe640 1 -- 172.21.15.28:0/4195265515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea80137e0 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.135+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfb82654f0 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.135+0000 7fcd937fe640 1 -- 172.21.15.28:0/4195265515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea8013b00 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.135+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb8265a90 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.135+0000 7fcd937fe640 1 -- 172.21.15.28:0/4195265515 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fcea8011040 con 0x55cfb8264260 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fcd937fe640 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd8803d070 0x7fcd8803f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fcd937fe640 1 -- 172.21.15.28:0/4195265515 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fcea800e040 con 0x55cfb8264260 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fcead8d6640 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd8803d070 0x7fcd8803f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fceb1b28800 1 Processor -- start 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fcead8d6640 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd8803d070 0x7fcd8803f530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55cfb824b3b0 tx=0x7fcea00529c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fceb1b28800 1 -- start start 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fceb1b28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb82710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fceb1b28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfb814e850 con 0x55cfb8270ce0 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.136+0000 7fceae0d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb82710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fceae0d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb82710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44528/0 (socket says 172.21.15.28:44528) 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fceae0d7640 1 -- 172.21.15.28:0/2336511894 learned_addr learned my addr 172.21.15.28:0/2336511894 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fceae0d7640 1 -- 172.21.15.28:0/2336511894 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb82716d0 con 0x55cfb8270ce0 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fceae0d7640 1 --2- 172.21.15.28:0/2336511894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb82710e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcea800afe0 tx=0x7fcea8020f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23b77d4d3851486 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fcd917fa640 1 -- 172.21.15.28:0/2336511894 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea8013ca0 con 0x55cfb8270ce0 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fcd917fa640 1 -- 172.21.15.28:0/2336511894 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea8052070 con 0x55cfb8270ce0 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fcd917fa640 1 -- 172.21.15.28:0/2336511894 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea802a3f0 con 0x55cfb8270ce0 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fceb1b28800 1 -- 172.21.15.28:0/2336511894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 msgr2=0x55cfb82710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.137+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2336511894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb82710e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcea800afe0 tx=0x7fcea8020f00 comp rx=0 tx=0).stop 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 -- 172.21.15.28:0/2336511894 shutdown_connections 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2336511894 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb82710e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 -- 172.21.15.28:0/2336511894 >> 172.21.15.28:0/2336511894 conn(0x55cfb826d0e0 msgr2=0x55cfb826b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 -- 172.21.15.28:0/2336511894 shutdown_connections 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 -- 172.21.15.28:0/2336511894 wait complete. 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 Processor -- start 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 -- start start 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb8278290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceb1b28800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cfb8271cf0 con 0x55cfb8270ce0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceae0d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb8278290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceae0d7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb8278290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44534/0 (socket says 172.21.15.28:44534) 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 learned_addr learned my addr 172.21.15.28:0/4224501230 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.138+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb82787f0 con 0x55cfb8270ce0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.139+0000 7fceae0d7640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb8278290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcea8020f40 tx=0x7fcea80547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.139+0000 7fcd7f7fe640 1 -- 172.21.15.28:0/4224501230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea8045c00 con 0x55cfb8270ce0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.139+0000 7fcd7f7fe640 1 -- 172.21.15.28:0/4224501230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea804de50 con 0x55cfb8270ce0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.139+0000 7fcd7f7fe640 1 -- 172.21.15.28:0/4224501230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea8054960 con 0x55cfb8270ce0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.139+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfb8278a80 con 0x55cfb8270ce0 2024-09-19T11:25:08.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.139+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb8279610 con 0x55cfb8270ce0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fcd7f7fe640 1 -- 172.21.15.28:0/4224501230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fcea8054b00 con 0x55cfb8270ce0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fcd7f7fe640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd7803d000 0x7fcd7803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fcd7f7fe640 1 -- 172.21.15.28:0/4224501230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fcea8004830 con 0x55cfb8270ce0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fcead8d6640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd7803d000 0x7fcd7803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8283b80 0x55cfb8286040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8286790 con 0x55cfb8283b80 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fcea50d5640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8283b80 0x55cfb8286040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.140+0000 7fcead8d6640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd7803d000 0x7fcd7803f4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcea0001250 tx=0x7fcea0045000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fcea50d5640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8283b80 0x55cfb8286040 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 20 realm 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8286bc0 con 0x55cfb8283b80 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8287490 con 0x55cfb8283b80 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.141+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8287830 con 0x55cfb8283b80 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8288590 0x55cfb828a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828aec0 con 0x55cfb8288590 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceae0d7640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8288590 0x55cfb828a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceae0d7640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8288590 0x55cfb828a980 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.142+0000 7fceb1b28800 20 searching for the correct realm 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fceae0d7640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcea803c3f0 0x7fcea80451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fcead8d6640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcea803c3f0 0x7fcea80451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fcead8d6640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcea803c3f0 0x7fcea80451d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fcead8d6640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcea0017050 0x7fcea004b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.143+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea0017050 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.144+0000 7fcea50d5640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcea0017050 0x7fcea004b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.144+0000 7fcea50d5640 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcea0017050 0x7fcea004b890 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.144+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.144+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.144+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.144+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea0017050 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea0017050 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.145+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea0017050 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.146+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.147+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea0017050 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.148+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea0017050 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.149+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea0017050 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8288590 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.150+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x55cfb8283b80 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c830 con 0x7fcea803c3f0 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828b900 con 0x55cfb8283b80 2024-09-19T11:25:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.151+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828c1a0 con 0x55cfb8283b80 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828b540 con 0x55cfb8288590 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 zone default found 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 4 Realm: () 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 zonegroup default 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828bdd0 con 0x7fcea0017050 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.152+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828ec20 con 0x55cfb8288590 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828f0b0 con 0x55cfb8288590 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceb1b28800 20 started sync module instance, tier type = 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceb1b28800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.153+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828e0c0 con 0x7fcea803c3f0 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.154+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.154+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94350636276752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828f510 con 0x7fcea803c3f0 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.154+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb828fd60 con 0x55cfb8283b80 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.155+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94350636276752] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.155+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.155+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94350636284080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8290180 con 0x55cfb8283b80 2024-09-19T11:25:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.155+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8290b40 con 0x55cfb8288590 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.156+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94350636284080] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.156+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.156+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94350636287936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8290180 con 0x55cfb8288590 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.156+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8291550 con 0x7fcea803c3f0 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.157+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94350636287936] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.158+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.158+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94350636285136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82900f0 con 0x7fcea803c3f0 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.158+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82923c0 con 0x55cfb8283b80 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.159+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94350636285136] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.159+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.159+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94350636292336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82923c0 con 0x55cfb8283b80 2024-09-19T11:25:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.159+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8292e10 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.160+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.160+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94350636292336] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.160+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94350636296480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8293270 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.160+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8293860 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.161+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94350636296480] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.161+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.161+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94350636293904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8293860 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.161+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8293860 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94350636293904] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94350636297536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8294220 con 0x7fcea0017050 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 20 add_watcher() i=0 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 20 add_watcher() i=1 2024-09-19T11:25:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 20 add_watcher() i=2 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 20 add_watcher() i=3 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 20 add_watcher() i=4 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 20 add_watcher() i=5 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.162+0000 7fceb1b28800 20 add_watcher() i=6 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.163+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94350636297536] v33'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.163+0000 7fceb1b28800 20 add_watcher() i=7 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.163+0000 7fceb1b28800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.163+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8297650 con 0x55cfb8283b80 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.163+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.163+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94350636302144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb8294b70 con 0x55cfb8283b80 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94350636302144] v33'237 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fcd5f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:08.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fceb1b28800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fceb1b28800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fceb1b28800 5 note: GC not initialized 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fcd5e7fc640 20 reqs_thread_entry: start 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fceb1b28800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fceb1b28800 20 Filter name: none 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fcccffff640 20 reqs_thread_entry: start 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fceb1b28800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.165+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82ca000 con 0x55cfb8283b80 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.166+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.166+0000 7fceb1b28800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:could not fetch user info: no user info saved 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.167+0000 7fcd5f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.167+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94350636302144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x55cfb8283b80 2024-09-19T11:25:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.168+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94350636302144] v33'238 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.168+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94350636276752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x7fcea803c3f0 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.169+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94350636276752] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.169+0000 7fceb1b28800 20 remove_watcher() i=0 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.169+0000 7fceb1b28800 2 removed watcher, disabling cache 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.169+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94350636284080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x55cfb8283b80 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.170+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94350636284080] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.170+0000 7fceb1b28800 20 remove_watcher() i=1 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.170+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94350636287936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x55cfb8288590 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.171+0000 7fceae0d7640 1 -- 172.21.15.28:0/4224501230 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94350636287936] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea800a4d0 con 0x55cfb8288590 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.171+0000 7fceb1b28800 20 remove_watcher() i=2 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.171+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94350636285136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x7fcea803c3f0 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.172+0000 7fcead8d6640 1 -- 172.21.15.28:0/4224501230 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94350636285136] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcea001ab20 con 0x7fcea803c3f0 2024-09-19T11:25:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.172+0000 7fceb1b28800 20 remove_watcher() i=3 2024-09-19T11:25:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.172+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94350636292336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x55cfb8283b80 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.173+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94350636292336] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x55cfb8283b80 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.173+0000 7fceb1b28800 20 remove_watcher() i=4 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.173+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94350636296480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x7fcea0017050 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.174+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94350636296480] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.174+0000 7fceb1b28800 20 remove_watcher() i=5 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.174+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94350636293904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x7fcea0017050 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.175+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94350636293904] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.175+0000 7fceb1b28800 20 remove_watcher() i=6 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.175+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94350636297536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55cfb82c9fc0 con 0x7fcea0017050 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94350636297536] v33'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fce98051070 con 0x7fcea0017050 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 20 remove_watcher() i=7 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcea0017050 msgr2=0x7fcea004b890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fcea0017050 0x7fcea004b890 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8283b80 msgr2=0x55cfb8286040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8283b80 0x55cfb8286040 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcea803c3f0 msgr2=0x7fcea80451d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fcea803c3f0 0x7fcea80451d0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8288590 msgr2=0x55cfb828a980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8288590 0x55cfb828a980 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd7803d000 msgr2=0x7fcd7803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd7803d000 0x7fcd7803f4c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcea0001250 tx=0x7fcea0045000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 msgr2=0x55cfb8278290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fcea50d5640 1 -- 172.21.15.28:0/4224501230 reap_dead start 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb8278290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcea8020f40 tx=0x7fcea80547b0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 shutdown_connections 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4224501230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8270ce0 0x55cfb8278290 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 >> 172.21.15.28:0/4224501230 conn(0x55cfb826d0e0 msgr2=0x55cfb826b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 shutdown_connections 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4224501230 wait complete. 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd8803d070 msgr2=0x7fcd8803f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd8803d070 0x7fcd8803f530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55cfb824b3b0 tx=0x7fcea00529c0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8264260 msgr2=0x55cfb8264680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.176+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8264260 0x55cfb8264680 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55cfb824a140 tx=0x7fcea80167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 shutdown_connections 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd8803d070 0x7fcd8803f530 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/4195265515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb8264260 0x55cfb8264680 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 >> 172.21.15.28:0/4195265515 conn(0x55cfb8258610 msgr2=0x55cfb82589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 shutdown_connections 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/4195265515 wait complete. 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8150e00 msgr2=0x55cfb8250900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8150e00 0x55cfb8250900 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8252ca0 msgr2=0x55cfb8255140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8252ca0 0x55cfb8255140 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd9403cfb0 msgr2=0x7fcd9403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd9403cfb0 0x7fcd9403f470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce98057000 tx=0x7fce98052000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 msgr2=0x55cfb82492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb82492b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcea000c010 tx=0x7fcea0000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 shutdown_connections 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55cfb8252ca0 0x55cfb8255140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fcd9403cfb0 0x7fcd9403f470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55cfb8150e00 0x55cfb8250900 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 --2- 172.21.15.28:0/2346552272 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cfb814dce0 0x55cfb82492b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 >> 172.21.15.28:0/2346552272 conn(0x55cfb81491c0 msgr2=0x55cfb823aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 shutdown_connections 2024-09-19T11:25:08.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.177+0000 7fceb1b28800 1 -- 172.21.15.28:0/2346552272 wait complete. 2024-09-19T11:25:08.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T11:25:08.188 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-09-19T11:25:08.188 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-09-19T11:25:08.188 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-09-19T11:25:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.231+0000 7ff1f8539800 1 Processor -- start 2024-09-19T11:25:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.231+0000 7ff1f8539800 1 -- start start 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f8539800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a4301aabd0 0x55a4301b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f8539800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a4301a9e70 con 0x55a4301a97c0 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f8539800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55a4300357d0 con 0x55a4301aabd0 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f39c6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a4301aabd0 0x55a4301b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f39c6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a4301aabd0 0x55a4301b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44544/0 (socket says 172.21.15.28:44544) 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f39c6640 1 -- 172.21.15.28:0/121790283 learned_addr learned my addr 172.21.15.28:0/121790283 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f39c6640 1 -- 172.21.15.28:0/121790283 >> v1:172.21.15.22:6789/0 conn(0x55a4301a97c0 legacy=0x55a4301aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.232+0000 7ff1f39c6640 1 -- 172.21.15.28:0/121790283 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4301ab080 con 0x55a4301aabd0 2024-09-19T11:25:08.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.234+0000 7ff1f39c6640 1 --2- 172.21.15.28:0/121790283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301aabd0 0x55a4301b0770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800a6b0 tx=0x7ff1e8047e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36c7fdec2cf8d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.234+0000 7ff1f31c5640 1 -- 172.21.15.28:0/121790283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e800a200 con 0x55a4301aabd0 2024-09-19T11:25:08.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.234+0000 7ff1f31c5640 1 -- 172.21.15.28:0/121790283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e800a3a0 con 0x55a4301aabd0 2024-09-19T11:25:08.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.234+0000 7ff1f31c5640 1 -- 172.21.15.28:0/121790283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e8008700 con 0x55a4301aabd0 2024-09-19T11:25:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.235+0000 7ff1f8539800 1 -- 172.21.15.28:0/121790283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301aabd0 msgr2=0x55a4301b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.235+0000 7ff1f8539800 1 --2- 172.21.15.28:0/121790283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301aabd0 0x55a4301b0770 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800a6b0 tx=0x7ff1e8047e30 comp rx=0 tx=0).stop 2024-09-19T11:25:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.235+0000 7ff1f8539800 1 -- 172.21.15.28:0/121790283 shutdown_connections 2024-09-19T11:25:08.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.235+0000 7ff1f8539800 1 --2- 172.21.15.28:0/121790283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301aabd0 0x55a4301b0770 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.235+0000 7ff1f8539800 1 -- 172.21.15.28:0/121790283 >> 172.21.15.28:0/121790283 conn(0x55a4301a4bb0 msgr2=0x55a4301a6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.235+0000 7ff1f8539800 1 -- 172.21.15.28:0/121790283 shutdown_connections 2024-09-19T11:25:08.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.235+0000 7ff1f8539800 1 -- 172.21.15.28:0/121790283 wait complete. 2024-09-19T11:25:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.250+0000 7ff1f8539800 1 Processor -- start 2024-09-19T11:25:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.250+0000 7ff1f8539800 1 -- start start 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.250+0000 7ff1f8539800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a43029c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.250+0000 7ff1f8539800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a430042dd0 con 0x55a4301a97c0 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff1f41c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a43029c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff1f41c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a43029c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44554/0 (socket says 172.21.15.28:44554) 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff1f41c7640 1 -- 172.21.15.28:0/485312803 learned_addr learned my addr 172.21.15.28:0/485312803 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff1f41c7640 1 -- 172.21.15.28:0/485312803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43029cb60 con 0x55a4301a97c0 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff1f41c7640 1 --2- 172.21.15.28:0/485312803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a43029c570 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4001da0 tx=0x7ff1e400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550b847afa209a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff0e260c640 1 -- 172.21.15.28:0/485312803 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e400f400 con 0x55a4301a97c0 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff0e260c640 1 -- 172.21.15.28:0/485312803 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e400f5a0 con 0x55a4301a97c0 2024-09-19T11:25:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.251+0000 7ff0e260c640 1 -- 172.21.15.28:0/485312803 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e400f8c0 con 0x55a4301a97c0 2024-09-19T11:25:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 -- 172.21.15.28:0/485312803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 msgr2=0x55a43029c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 --2- 172.21.15.28:0/485312803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a43029c570 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4001da0 tx=0x7ff1e400b5b0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 -- 172.21.15.28:0/485312803 shutdown_connections 2024-09-19T11:25:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 --2- 172.21.15.28:0/485312803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a43029c570 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 -- 172.21.15.28:0/485312803 >> 172.21.15.28:0/485312803 conn(0x55a4301a4bb0 msgr2=0x55a430293ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 -- 172.21.15.28:0/485312803 shutdown_connections 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 -- 172.21.15.28:0/485312803 wait complete. 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 Processor -- start 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 -- start start 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a4302a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.252+0000 7ff1f8539800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a43011d1e0 con 0x55a4301a97c0 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff1f41c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a4302a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff1f41c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a4302a40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44566/0 (socket says 172.21.15.28:44566) 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff1f41c7640 1 -- 172.21.15.28:0/4029377233 learned_addr learned my addr 172.21.15.28:0/4029377233 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff1f41c7640 1 -- 172.21.15.28:0/4029377233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4302a4620 con 0x55a4301a97c0 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff1f41c7640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a4302a40c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55a43029d500 tx=0x7ff1e4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff0e0e09640 1 -- 172.21.15.28:0/4029377233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e401a690 con 0x55a4301a97c0 2024-09-19T11:25:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff0e0e09640 1 -- 172.21.15.28:0/4029377233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e401a830 con 0x55a4301a97c0 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4302a48b0 con 0x55a4301a97c0 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff0e0e09640 1 -- 172.21.15.28:0/4029377233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e401ab50 con 0x55a4301a97c0 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.253+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4302a5230 con 0x55a4301a97c0 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff1f8539800 1 Processor -- start 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff1f8539800 1 -- start start 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff1f8539800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff1f8539800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4300a14a0 con 0x55a4302b1650 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff0e0e09640 1 -- 172.21.15.28:0/4029377233 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff1e4017070 con 0x55a4301a97c0 2024-09-19T11:25:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff1f49c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff1f49c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44568/0 (socket says 172.21.15.28:44568) 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.254+0000 7ff1f49c8640 1 -- 172.21.15.28:0/2355916741 learned_addr learned my addr 172.21.15.28:0/2355916741 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff0e0e09640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0dc03d000 0x7ff0dc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff0e0e09640 1 -- 172.21.15.28:0/4029377233 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff1e4015020 con 0x55a4301a97c0 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff1f49c8640 1 -- 172.21.15.28:0/2355916741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4302b4010 con 0x55a4302b1650 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff1f41c7640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0dc03d000 0x7ff0dc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff1f49c8640 1 --2- 172.21.15.28:0/2355916741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b3a40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00a3e0 tx=0x7ff1ec00a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23184d00946e2af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff0da7fc640 1 -- 172.21.15.28:0/2355916741 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1ec015070 con 0x55a4302b1650 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff1f41c7640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0dc03d000 0x7ff0dc03f4c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff1e401bf90 tx=0x7ff1e4035040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff0da7fc640 1 -- 172.21.15.28:0/2355916741 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec001670 con 0x55a4302b1650 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff0da7fc640 1 -- 172.21.15.28:0/2355916741 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1ec001990 con 0x55a4302b1650 2024-09-19T11:25:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff1f8539800 1 -- 172.21.15.28:0/2355916741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 msgr2=0x55a4302b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.255+0000 7ff1f8539800 1 --2- 172.21.15.28:0/2355916741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b3a40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00a3e0 tx=0x7ff1ec00a8a0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 -- 172.21.15.28:0/2355916741 shutdown_connections 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 --2- 172.21.15.28:0/2355916741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b3a40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 -- 172.21.15.28:0/2355916741 >> 172.21.15.28:0/2355916741 conn(0x55a4301aabd0 msgr2=0x55a4302abf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 -- 172.21.15.28:0/2355916741 shutdown_connections 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 -- 172.21.15.28:0/2355916741 wait complete. 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 Processor -- start 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 -- start start 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f8539800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a4302b4360 con 0x55a4302b1650 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f49c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f49c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44574/0 (socket says 172.21.15.28:44574) 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.256+0000 7ff1f49c8640 1 -- 172.21.15.28:0/1747832253 learned_addr learned my addr 172.21.15.28:0/1747832253 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.257+0000 7ff1f49c8640 1 -- 172.21.15.28:0/1747832253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4302b8b10 con 0x55a4302b1650 2024-09-19T11:25:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.257+0000 7ff1f49c8640 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b85b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec013040 tx=0x7ff1ec003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.257+0000 7ff0d8ff9640 1 -- 172.21.15.28:0/1747832253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1ec020020 con 0x55a4302b1650 2024-09-19T11:25:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.257+0000 7ff0d8ff9640 1 -- 172.21.15.28:0/1747832253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec010780 con 0x55a4302b1650 2024-09-19T11:25:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.257+0000 7ff0d8ff9640 1 -- 172.21.15.28:0/1747832253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1ec015040 con 0x55a4302b1650 2024-09-19T11:25:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.257+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4302b8da0 con 0x55a4302b1650 2024-09-19T11:25:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.257+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4302b9720 con 0x55a4302b1650 2024-09-19T11:25:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.258+0000 7ff0d8ff9640 1 -- 172.21.15.28:0/1747832253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff1ec023020 con 0x55a4302b1650 2024-09-19T11:25:08.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.258+0000 7ff0d8ff9640 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0cc03d000 0x7ff0cc03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.258+0000 7ff0d8ff9640 1 -- 172.21.15.28:0/1747832253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff1ec053e10 con 0x55a4302b1650 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.258+0000 7ff1f39c6640 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0cc03d000 0x7ff0cc03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.259+0000 7ff1f8539800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.259+0000 7ff1f8539800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.259+0000 7ff1f39c6640 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0cc03d000 0x7ff0cc03f4c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8058000 tx=0x7ff1e8053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.259+0000 7ff1f8539800 1 Processor -- start 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.259+0000 7ff1f8539800 1 -- start start 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.259+0000 7ff1f8539800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302d2440 0x55a4302d2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.259+0000 7ff1f8539800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a43029cf90 con 0x55a4302d2440 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff1f41c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302d2440 0x55a4302d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff1f41c7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302d2440 0x55a4302d2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44590/0 (socket says 172.21.15.28:44590) 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff1f41c7640 1 -- 172.21.15.28:0/819691462 learned_addr learned my addr 172.21.15.28:0/819691462 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff1f41c7640 1 -- 172.21.15.28:0/819691462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4302d2ea0 con 0x55a4302d2440 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff1f41c7640 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302d2440 0x55a4302d2860 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e750 tx=0x7ff1e400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff0b8fe1640 1 -- 172.21.15.28:0/819691462 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e401caa0 con 0x55a4302d2440 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff0b8fe1640 1 -- 172.21.15.28:0/819691462 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e40193e0 con 0x55a4302d2440 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4302d36d0 con 0x55a4302d2440 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff0b8fe1640 1 -- 172.21.15.28:0/819691462 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e400ecc0 con 0x55a4302d2440 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.260+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4302d36d0 con 0x55a4302d2440 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.261+0000 7ff0b8fe1640 1 -- 172.21.15.28:0/819691462 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff1e400e070 con 0x55a4302d2440 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.261+0000 7ff0b8fe1640 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff03403d050 0x7ff03403f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.261+0000 7ff0b8fe1640 1 -- 172.21.15.28:0/819691462 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff1e404ad40 con 0x55a4302d2440 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.261+0000 7ff1f39c6640 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff03403d050 0x7ff03403f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff0627fc640 20 reqs_thread_entry: start 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f39c6640 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff03403d050 0x7ff03403f510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8047e70 tx=0x7ff1e8005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4302dfba0 0x55a4302e2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a4302e2700 con 0x55a4302dfba0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f49c8640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4302dfba0 0x55a4302e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f49c8640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4302dfba0 0x55a4302e2030 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f49c8640 1 -- 172.21.15.28:0/4029377233 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff1ec00e070 con 0x55a4302dfba0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4302e44d0 0x55a4302e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.262+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:2 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a4302e70d0 con 0x55a4302e44d0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.263+0000 7ff1f41c7640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4302e44d0 0x55a4302e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.263+0000 7ff1f41c7640 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4302e44d0 0x55a4302e6940 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.263+0000 7ff1f41c7640 1 -- 172.21.15.28:0/4029377233 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(2 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff1e4050070 con 0x55a4302e44d0 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0cc03d000 msgr2=0x7ff0cc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0cc03d000 0x7ff0cc03f4c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8058000 tx=0x7ff1e8053000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 msgr2=0x55a4302b85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b85b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec013040 tx=0x7ff1ec003370 comp rx=0 tx=0).stop 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 shutdown_connections 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0cc03d000 0x7ff0cc03f4c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/1747832253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302b1650 0x55a4302b85b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 >> 172.21.15.28:0/1747832253 conn(0x55a4301aabd0 msgr2=0x55a4302abf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 shutdown_connections 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/1747832253 wait complete. 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff03403d050 msgr2=0x7ff03403f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff03403d050 0x7ff03403f510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8047e70 tx=0x7ff1e8005ac0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302d2440 msgr2=0x55a4302d2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302d2440 0x55a4302d2860 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400e750 tx=0x7ff1e400eb90 comp rx=0 tx=0).stop 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 shutdown_connections 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff03403d050 0x7ff03403f510 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 --2- 172.21.15.28:0/819691462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4302d2440 0x55a4302d2860 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.264+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 >> 172.21.15.28:0/819691462 conn(0x55a4302c79a0 msgr2=0x55a4302c7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 shutdown_connections 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/819691462 wait complete. 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4302dfba0 msgr2=0x55a4302e2030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4302dfba0 0x55a4302e2030 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4302e44d0 msgr2=0x55a4302e6940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4302e44d0 0x55a4302e6940 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0dc03d000 msgr2=0x7ff0dc03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0dc03d000 0x7ff0dc03f4c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff1e401bf90 tx=0x7ff1e4035040 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 msgr2=0x55a4302a40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a4302a40c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x55a43029d500 tx=0x7ff1e4000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 shutdown_connections 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a4302e44d0 0x55a4302e6940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff0dc03d000 0x7ff0dc03f4c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a4302dfba0 0x55a4302e2030 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 --2- 172.21.15.28:0/4029377233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a4301a97c0 0x55a4302a40c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 >> 172.21.15.28:0/4029377233 conn(0x55a4301a4bb0 msgr2=0x55a43029da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 shutdown_connections 2024-09-19T11:25:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.265+0000 7ff1f8539800 1 -- 172.21.15.28:0/4029377233 wait complete. 2024-09-19T11:25:08.276 INFO:teuthology.orchestra.run.smithi028.stdout:{"id":"293899f1-2738-4b64-a8e1-a18ff039d626","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-09-19T11:25:08.277 DEBUG:tasks.util.rgw: json result: {'id': '293899f1-2738-4b64-a8e1-a18ff039d626', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-09-19T11:25:08.277 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-09-19T11:25:08.277 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-09-19T11:25:08.277 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-09-19T11:25:08.321 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:08.321 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:08.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.320+0000 7fa373d25800 1 Processor -- start 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.320+0000 7fa373d25800 1 -- start start 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa373d25800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564e152faf10 0x564e152fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa373d25800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564e153015c0 con 0x564e152fd8a0 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa373d25800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x564e151857d0 con 0x564e152faf10 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa36f933640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x564e152faf10 0x564e152fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa36f132640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x564e152fd8a0 0x564e152ffd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48226/0 (socket says 172.21.15.28:48226) 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa36f132640 1 -- 172.21.15.28:0/3131567336 learned_addr learned my addr 172.21.15.28:0/3131567336 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:08.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa36f933640 1 -- 172.21.15.28:0/3131567336 >> v1:172.21.15.22:6789/0 conn(0x564e152fd8a0 legacy=0x564e152ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.321+0000 7fa36f933640 1 -- 172.21.15.28:0/3131567336 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e153027f0 con 0x564e152faf10 2024-09-19T11:25:08.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.323+0000 7fa36f933640 1 --2- 172.21.15.28:0/3131567336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fd300 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa36000a470 tx=0x7fa360045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5297a560f977cc5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.323+0000 7fa36e931640 1 -- 172.21.15.28:0/3131567336 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa360053570 con 0x564e152faf10 2024-09-19T11:25:08.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.323+0000 7fa36e931640 1 -- 172.21.15.28:0/3131567336 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa360058070 con 0x564e152faf10 2024-09-19T11:25:08.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.323+0000 7fa36e931640 1 -- 172.21.15.28:0/3131567336 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa360053880 con 0x564e152faf10 2024-09-19T11:25:08.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.324+0000 7fa373d25800 1 -- 172.21.15.28:0/3131567336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 msgr2=0x564e152fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.324+0000 7fa373d25800 1 --2- 172.21.15.28:0/3131567336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fd300 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa36000a470 tx=0x7fa360045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:08.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.324+0000 7fa373d25800 1 -- 172.21.15.28:0/3131567336 shutdown_connections 2024-09-19T11:25:08.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.324+0000 7fa373d25800 1 --2- 172.21.15.28:0/3131567336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fd300 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.324+0000 7fa373d25800 1 -- 172.21.15.28:0/3131567336 >> 172.21.15.28:0/3131567336 conn(0x564e152f4a50 msgr2=0x564e152f6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.324+0000 7fa373d25800 1 -- 172.21.15.28:0/3131567336 shutdown_connections 2024-09-19T11:25:08.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.324+0000 7fa373d25800 1 -- 172.21.15.28:0/3131567336 wait complete. 2024-09-19T11:25:08.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.339+0000 7fa373d25800 1 Processor -- start 2024-09-19T11:25:08.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.339+0000 7fa373d25800 1 -- start start 2024-09-19T11:25:08.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.339+0000 7fa373d25800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e153ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.339+0000 7fa373d25800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564e15192dd0 con 0x564e152faf10 2024-09-19T11:25:08.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.339+0000 7fa36f933640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e153ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.339+0000 7fa36f933640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e153ea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44622/0 (socket says 172.21.15.28:44622) 2024-09-19T11:25:08.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.339+0000 7fa36f933640 1 -- 172.21.15.28:0/3493753841 learned_addr learned my addr 172.21.15.28:0/3493753841 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.340+0000 7fa36f933640 1 -- 172.21.15.28:0/3493753841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e153ea910 con 0x564e152faf10 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.340+0000 7fa36f933640 1 --2- 172.21.15.28:0/3493753841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e153ea320 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa360025c20 tx=0x7fa360051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e740140ca5867b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.340+0000 7fa25de0b640 1 -- 172.21.15.28:0/3493753841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa36005c580 con 0x564e152faf10 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.340+0000 7fa25de0b640 1 -- 172.21.15.28:0/3493753841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa360063030 con 0x564e152faf10 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.340+0000 7fa25de0b640 1 -- 172.21.15.28:0/3493753841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa36005c850 con 0x564e152faf10 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 -- 172.21.15.28:0/3493753841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 msgr2=0x564e153ea320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 --2- 172.21.15.28:0/3493753841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e153ea320 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa360025c20 tx=0x7fa360051000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 -- 172.21.15.28:0/3493753841 shutdown_connections 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 --2- 172.21.15.28:0/3493753841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e153ea320 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 -- 172.21.15.28:0/3493753841 >> 172.21.15.28:0/3493753841 conn(0x564e152f4a50 msgr2=0x564e153e5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 -- 172.21.15.28:0/3493753841 shutdown_connections 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 -- 172.21.15.28:0/3493753841 wait complete. 2024-09-19T11:25:08.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 Processor -- start 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 -- start start 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa373d25800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564e15301a90 con 0x564e152faf10 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa36f933640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa36f933640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fa630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44630/0 (socket says 172.21.15.28:44630) 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.341+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 learned_addr learned my addr 172.21.15.28:0/354996136 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.342+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e152fab90 con 0x564e152faf10 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.342+0000 7fa36f933640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fa630 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa360045f50 tx=0x7fa360002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.342+0000 7fa257fff640 1 -- 172.21.15.28:0/354996136 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa360054ad0 con 0x564e152faf10 2024-09-19T11:25:08.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.342+0000 7fa257fff640 1 -- 172.21.15.28:0/354996136 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa360054c70 con 0x564e152faf10 2024-09-19T11:25:08.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.342+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e152f8f40 con 0x564e152faf10 2024-09-19T11:25:08.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.342+0000 7fa257fff640 1 -- 172.21.15.28:0/354996136 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa360068420 con 0x564e152faf10 2024-09-19T11:25:08.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.342+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e152f9500 con 0x564e152faf10 2024-09-19T11:25:08.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa373d25800 1 Processor -- start 2024-09-19T11:25:08.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa373d25800 1 -- start start 2024-09-19T11:25:08.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa373d25800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e154039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa373d25800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564e151f14a0 con 0x564e154015e0 2024-09-19T11:25:08.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa257fff640 1 -- 172.21.15.28:0/354996136 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa3600685c0 con 0x564e152faf10 2024-09-19T11:25:08.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa370134640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e154039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa370134640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e154039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44638/0 (socket says 172.21.15.28:44638) 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa370134640 1 -- 172.21.15.28:0/2023456219 learned_addr learned my addr 172.21.15.28:0/2023456219 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa257fff640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa25803d000 0x7fa25803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa257fff640 1 -- 172.21.15.28:0/354996136 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa360070030 con 0x564e152faf10 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa370134640 1 -- 172.21.15.28:0/2023456219 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e15403fa0 con 0x564e154015e0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.343+0000 7fa36f933640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa25803d000 0x7fa25803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa370134640 1 --2- 172.21.15.28:0/2023456219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e154039d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa36800a3e0 tx=0x7fa36800a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4faa5c47386b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa36f933640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa25803d000 0x7fa25803f4c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa360054fb0 tx=0x7fa360068910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa255ffb640 1 -- 172.21.15.28:0/2023456219 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa368015070 con 0x564e154015e0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa255ffb640 1 -- 172.21.15.28:0/2023456219 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368001670 con 0x564e154015e0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa255ffb640 1 -- 172.21.15.28:0/2023456219 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa368001990 con 0x564e154015e0 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 -- 172.21.15.28:0/2023456219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 msgr2=0x564e154039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 --2- 172.21.15.28:0/2023456219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e154039d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa36800a3e0 tx=0x7fa36800a8a0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 -- 172.21.15.28:0/2023456219 shutdown_connections 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 --2- 172.21.15.28:0/2023456219 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e154039d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 -- 172.21.15.28:0/2023456219 >> 172.21.15.28:0/2023456219 conn(0x564e152fd8a0 msgr2=0x564e153fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 -- 172.21.15.28:0/2023456219 shutdown_connections 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 -- 172.21.15.28:0/2023456219 wait complete. 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.344+0000 7fa373d25800 1 Processor -- start 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa373d25800 1 -- start start 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa373d25800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e15408510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa373d25800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564e15404290 con 0x564e154015e0 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa370134640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e15408510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa370134640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e15408510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44642/0 (socket says 172.21.15.28:44642) 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa370134640 1 -- 172.21.15.28:0/4280751333 learned_addr learned my addr 172.21.15.28:0/4280751333 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa370134640 1 -- 172.21.15.28:0/4280751333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e15408a70 con 0x564e154015e0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa370134640 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e15408510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa368013040 tx=0x7fa368003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa24ffff640 1 -- 172.21.15.28:0/4280751333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa368020020 con 0x564e154015e0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa24ffff640 1 -- 172.21.15.28:0/4280751333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368010780 con 0x564e154015e0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.345+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e15408d00 con 0x564e154015e0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.346+0000 7fa24ffff640 1 -- 172.21.15.28:0/4280751333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa368015040 con 0x564e154015e0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.346+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e15409680 con 0x564e154015e0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.346+0000 7fa24ffff640 1 -- 172.21.15.28:0/4280751333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa368023020 con 0x564e154015e0 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.347+0000 7fa24ffff640 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa24803d000 0x7fa24803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.347+0000 7fa24ffff640 1 -- 172.21.15.28:0/4280751333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa368053e90 con 0x564e154015e0 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.347+0000 7fa36f132640 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa24803d000 0x7fa24803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.347+0000 7fa373d25800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.347+0000 7fa373d25800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.347+0000 7fa36f132640 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa24803d000 0x7fa24803f4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa364001cf0 tx=0x7fa364001250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.347+0000 7fa373d25800 1 Processor -- start 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa373d25800 1 -- start start 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa373d25800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154223a0 0x564e154227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa373d25800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564e153eace0 con 0x564e154223a0 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa36f933640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154223a0 0x564e154227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa36f933640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154223a0 0x564e154227c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44648/0 (socket says 172.21.15.28:44648) 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa36f933640 1 -- 172.21.15.28:0/4024820610 learned_addr learned my addr 172.21.15.28:0/4024820610 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa36f933640 1 -- 172.21.15.28:0/4024820610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e15422e00 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa36f933640 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154223a0 0x564e154227c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa360068a20 tx=0x7fa36007df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.348+0000 7fa1e7fff640 1 -- 172.21.15.28:0/4024820610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa360074420 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.349+0000 7fa1e7fff640 1 -- 172.21.15.28:0/4024820610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa360096b90 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.349+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e15423630 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.349+0000 7fa1e7fff640 1 -- 172.21.15.28:0/4024820610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3600747a0 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.349+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e15423bd0 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.349+0000 7fa1e7fff640 1 -- 172.21.15.28:0/4024820610 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa36009f020 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa1e7fff640 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1b003d000 0x7fa1b003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa1e7fff640 1 -- 172.21.15.28:0/4024820610 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa360085040 con 0x564e154223a0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa36f132640 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1b003d000 0x7fa1b003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa1e5ffb640 20 reqs_thread_entry: start 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564e1542fcc0 0x564e154320d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564e15432750 con 0x564e1542fcc0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa36f132640 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1b003d000 0x7fa1b003f4c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa364000f60 tx=0x7fa36400f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa370134640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564e1542fcc0 0x564e154320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.350+0000 7fa370134640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564e1542fcc0 0x564e154320d0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.351+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.351+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564e15433030 con 0x564e1542fcc0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.351+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.351+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564e15434940 0x564e15436d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.351+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564e15437520 con 0x564e15434940 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.351+0000 7fa36f933640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564e15434940 0x564e15436d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.351+0000 7fa36f933640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564e15434940 0x564e15436d90 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.352+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.352+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564e15444660 con 0x564e1542fcc0 2024-09-19T11:25:08.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.352+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.369 INFO:teuthology.orchestra.run.smithi028.stderr:NOTICE: set zone's realm_id= 2024-09-19T11:25:08.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.352+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f933640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa360096020 0x7fa3600988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f132640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa360096020 0x7fa3600988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f132640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa360096020 0x7fa3600988a0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa364009050 con 0x7fa360096020 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f132640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa36401b460 0x7fa36401b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa36401b460 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.353+0000 7fa370134640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa36401b460 0x7fa36401b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.354+0000 7fa370134640 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa36401b460 0x7fa36401b880 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.354+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x7fa36401b460 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.354+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.354+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.354+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.354+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa36401b460 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x7fa36401b460 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa36401b460 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x7fa36401b460 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa364009050 con 0x7fa360096020 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.355+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa36401b460 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x7fa36401b460 2024-09-19T11:25:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(17 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa36402a040 con 0x7fa360096020 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.356+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(19 [pgnls start_epoch 33 out=170b] v31'3 uv3 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36402a040 con 0x7fa360096020 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.357+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=74b] v24'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa36401b460 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x7fa36401b460 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.358+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36402a040 con 0x7fa360096020 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa36401b460 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x7fa36401b460 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.359+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa36401b460 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36800e070 con 0x7fa36401b460 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36402a040 con 0x7fa360096020 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e15434940 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36402a040 con 0x7fa360096020 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.360+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x564e1542fcc0 2024-09-19T11:25:08.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.361+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(35 [pgnls start_epoch 33 out=115b] v24'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.361+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x564e1543ba20 con 0x7fa360096020 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.361+0000 7fa36f132640 1 -- 172.21.15.28:0/354996136 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(36 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa36402a040 con 0x7fa360096020 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.361+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564e1543e4d0 con 0x564e1542fcc0 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.361+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.361+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:38 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x564e15442530 con 0x564e15434940 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.361+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(38 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv3 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.362+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:39 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [create,call version.set in=58b,writefull 0~1155 in=1155b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564e15437950 con 0x564e15434940 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.363+0000 7fa36f933640 1 -- 172.21.15.28:0/354996136 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(39 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [create,call,writefull 0~1155] v33'4 uv4 ondisk = 0) ==== 274+0+0 (crc 0 0 0) 0x7fa360093070 con 0x564e15434940 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.363+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564e15440eb0 con 0x564e1542fcc0 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.364+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v33'3 uv3 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.364+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x564e1543e950 con 0x564e1542fcc0 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.365+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v33'2 uv1 ondisk = -17 ((17) File exists)) ==== 199+0+0 (crc 0 0 0) 0x7fa36800e070 con 0x564e1542fcc0 2024-09-19T11:25:08.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa24803d000 msgr2=0x7fa24803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa24803d000 0x7fa24803f4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa364001cf0 tx=0x7fa364001250 comp rx=0 tx=0).stop 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 msgr2=0x564e15408510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e15408510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa368013040 tx=0x7fa368003370 comp rx=0 tx=0).stop 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 shutdown_connections 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa24803d000 0x7fa24803f4c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4280751333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154015e0 0x564e15408510 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 >> 172.21.15.28:0/4280751333 conn(0x564e152fd8a0 msgr2=0x564e153fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 shutdown_connections 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4280751333 wait complete. 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1b003d000 msgr2=0x7fa1b003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1b003d000 0x7fa1b003f4c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa364000f60 tx=0x7fa36400f000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154223a0 msgr2=0x564e154227c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154223a0 0x564e154227c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa360068a20 tx=0x7fa36007df50 comp rx=0 tx=0).stop 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 shutdown_connections 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1b003d000 0x7fa1b003f4c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 --2- 172.21.15.28:0/4024820610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e154223a0 0x564e154227c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 >> 172.21.15.28:0/4024820610 conn(0x564e154178e0 msgr2=0x564e15417cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 shutdown_connections 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.366+0000 7fa373d25800 1 -- 172.21.15.28:0/4024820610 wait complete. 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa36401b460 msgr2=0x7fa36401b880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa36401b460 0x7fa36401b880 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564e1542fcc0 msgr2=0x564e154320d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x564e1542fcc0 0x564e154320d0 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa360096020 msgr2=0x7fa3600988a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa360096020 0x7fa3600988a0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564e15434940 msgr2=0x564e15436d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x564e15434940 0x564e15436d90 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa25803d000 msgr2=0x7fa25803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa25803d000 0x7fa25803f4c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa360054fb0 tx=0x7fa360068910 comp rx=0 tx=0).stop 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 msgr2=0x564e152fa630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 --2- 172.21.15.28:0/354996136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564e152faf10 0x564e152fa630 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa360045f50 tx=0x7fa360002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 reap_dead start 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa370134640 1 -- 172.21.15.28:0/354996136 reap_dead start 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 shutdown_connections 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 >> 172.21.15.28:0/354996136 conn(0x564e152f4a50 msgr2=0x564e153e4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 shutdown_connections 2024-09-19T11:25:08.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.367+0000 7fa373d25800 1 -- 172.21.15.28:0/354996136 wait complete. 2024-09-19T11:25:08.379 INFO:teuthology.orchestra.run.smithi028.stdout:{"id":"293899f1-2738-4b64-a8e1-a18ff039d626","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-09-19T11:25:08.379 DEBUG:tasks.util.rgw: json result: {'id': '293899f1-2738-4b64-a8e1-a18ff039d626', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-09-19T11:25:08.380 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-09-19T11:25:08.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-09-19T11:25:08.380 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-09-19T11:25:08.466 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:08.466 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:08.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.465+0000 7f3ff0216800 1 Processor -- start 2024-09-19T11:25:08.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.465+0000 7f3ff0216800 1 -- start start 2024-09-19T11:25:08.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.465+0000 7f3ff0216800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5607d83b5f10 0x5607d83b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.465+0000 7f3ff0216800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5607d83bc5c0 con 0x5607d83b88a0 2024-09-19T11:25:08.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.465+0000 7f3ff0216800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5607d82407d0 con 0x5607d83b5f10 2024-09-19T11:25:08.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.466+0000 7f3feb9e7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5607d83b5f10 0x5607d83b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.466+0000 7f3feb1e6640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5607d83b88a0 0x5607d83bad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48236/0 (socket says 172.21.15.28:48236) 2024-09-19T11:25:08.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.466+0000 7f3feb9e7640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5607d83b5f10 0x5607d83b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44654/0 (socket says 172.21.15.28:44654) 2024-09-19T11:25:08.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.466+0000 7f3feb1e6640 1 -- 172.21.15.28:0/3927744433 learned_addr learned my addr 172.21.15.28:0/3927744433 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:08.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.466+0000 7f3feb9e7640 1 -- 172.21.15.28:0/3927744433 >> v1:172.21.15.22:6789/0 conn(0x5607d83b88a0 legacy=0x5607d83bad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.466+0000 7f3fea9e5640 1 -- 172.21.15.28:0/3927744433 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2641149401 0 0) 0x5607d83bc5c0 con 0x5607d83b88a0 2024-09-19T11:25:08.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.466+0000 7f3feb9e7640 1 -- 172.21.15.28:0/3927744433 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d83bd7f0 con 0x5607d83b5f10 2024-09-19T11:25:08.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/3927744433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b8300 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00a470 tx=0x7f3fdc045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297f346816970bed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3fea9e5640 1 -- 172.21.15.28:0/3927744433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc053570 con 0x5607d83b5f10 2024-09-19T11:25:08.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3fea9e5640 1 -- 172.21.15.28:0/3927744433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc058070 con 0x5607d83b5f10 2024-09-19T11:25:08.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3fea9e5640 1 -- 172.21.15.28:0/3927744433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc053880 con 0x5607d83b5f10 2024-09-19T11:25:08.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3ff0216800 1 -- 172.21.15.28:0/3927744433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 msgr2=0x5607d83b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3ff0216800 1 --2- 172.21.15.28:0/3927744433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b8300 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00a470 tx=0x7f3fdc045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:08.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3ff0216800 1 -- 172.21.15.28:0/3927744433 shutdown_connections 2024-09-19T11:25:08.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3ff0216800 1 --2- 172.21.15.28:0/3927744433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b8300 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.468+0000 7f3ff0216800 1 -- 172.21.15.28:0/3927744433 >> 172.21.15.28:0/3927744433 conn(0x5607d83afa50 msgr2=0x5607d83b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.469+0000 7f3ff0216800 1 -- 172.21.15.28:0/3927744433 shutdown_connections 2024-09-19T11:25:08.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.469+0000 7f3ff0216800 1 -- 172.21.15.28:0/3927744433 wait complete. 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.483+0000 7f3ff0216800 1 Processor -- start 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.483+0000 7f3ff0216800 1 -- start start 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.484+0000 7f3ff0216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.484+0000 7f3ff0216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607d824ddd0 con 0x5607d83b5f10 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.484+0000 7f3feb9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.484+0000 7f3feb9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44668/0 (socket says 172.21.15.28:44668) 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.484+0000 7f3feb9e7640 1 -- 172.21.15.28:0/3237279736 learned_addr learned my addr 172.21.15.28:0/3237279736 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.484+0000 7f3feb9e7640 1 -- 172.21.15.28:0/3237279736 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d83b5c70 con 0x5607d83b5f10 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.484+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/3237279736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b5680 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc005870 tx=0x7f3fdc051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8e68612a94d320 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ed9e0b640 1 -- 172.21.15.28:0/3237279736 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc05c580 con 0x5607d83b5f10 2024-09-19T11:25:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ed9e0b640 1 -- 172.21.15.28:0/3237279736 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc063030 con 0x5607d83b5f10 2024-09-19T11:25:08.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ed9e0b640 1 -- 172.21.15.28:0/3237279736 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc05c850 con 0x5607d83b5f10 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ff0216800 1 -- 172.21.15.28:0/3237279736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 msgr2=0x5607d83b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ff0216800 1 --2- 172.21.15.28:0/3237279736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b5680 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc005870 tx=0x7f3fdc051000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ff0216800 1 -- 172.21.15.28:0/3237279736 shutdown_connections 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ff0216800 1 --2- 172.21.15.28:0/3237279736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d83b5680 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ff0216800 1 -- 172.21.15.28:0/3237279736 >> 172.21.15.28:0/3237279736 conn(0x5607d83afa50 msgr2=0x5607d84a0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ff0216800 1 -- 172.21.15.28:0/3237279736 shutdown_connections 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.485+0000 7f3ff0216800 1 -- 172.21.15.28:0/3237279736 wait complete. 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3ff0216800 1 Processor -- start 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3ff0216800 1 -- start start 2024-09-19T11:25:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3ff0216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d84aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3ff0216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607d83bca90 con 0x5607d83b5f10 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3feb9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d84aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3feb9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d84aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44680/0 (socket says 172.21.15.28:44680) 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3feb9e7640 1 -- 172.21.15.28:0/1334988098 learned_addr learned my addr 172.21.15.28:0/1334988098 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.486+0000 7f3feb9e7640 1 -- 172.21.15.28:0/1334988098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d84af300 con 0x5607d83b5f10 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d84aeda0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc045f50 tx=0x7f3fdc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3ed3fff640 1 -- 172.21.15.28:0/1334988098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc078070 con 0x5607d83b5f10 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3ed3fff640 1 -- 172.21.15.28:0/1334988098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc054a70 con 0x5607d83b5f10 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607d84af590 con 0x5607d83b5f10 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3ed3fff640 1 -- 172.21.15.28:0/1334988098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc054d50 con 0x5607d83b5f10 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d84b01a0 con 0x5607d83b5f10 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3ff0216800 1 Processor -- start 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.487+0000 7f3ff0216800 1 -- start start 2024-09-19T11:25:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ed3fff640 1 -- 172.21.15.28:0/1334988098 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3fdc07b070 con 0x5607d83b5f10 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ff0216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ff0216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607d82ac4a0 con 0x5607d84bc470 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ed3fff640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ed403cfb0 0x7f3ed403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3fec1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ed3fff640 1 -- 172.21.15.28:0/1334988098 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3fdc070070 con 0x5607d83b5f10 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3fec1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84be860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44686/0 (socket says 172.21.15.28:44686) 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3fec1e8640 1 -- 172.21.15.28:0/2620392993 learned_addr learned my addr 172.21.15.28:0/2620392993 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ed403cfb0 0x7f3ed403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3fec1e8640 1 -- 172.21.15.28:0/2620392993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d84bee30 con 0x5607d84bc470 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3fec1e8640 1 --2- 172.21.15.28:0/2620392993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84be860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400a3e0 tx=0x7f3fe400a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bac6690785cdb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ed403cfb0 0x7f3ed403f470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc07c940 tx=0x7f3fdc054f70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ed1ffb640 1 -- 172.21.15.28:0/2620392993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fe4015070 con 0x5607d84bc470 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ed1ffb640 1 -- 172.21.15.28:0/2620392993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe4001670 con 0x5607d84bc470 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.488+0000 7f3ed1ffb640 1 -- 172.21.15.28:0/2620392993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fe4001990 con 0x5607d84bc470 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 -- 172.21.15.28:0/2620392993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 msgr2=0x5607d84be860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 --2- 172.21.15.28:0/2620392993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84be860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400a3e0 tx=0x7f3fe400a8a0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 -- 172.21.15.28:0/2620392993 shutdown_connections 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 --2- 172.21.15.28:0/2620392993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84be860 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 -- 172.21.15.28:0/2620392993 >> 172.21.15.28:0/2620392993 conn(0x5607d83b88a0 msgr2=0x5607d84b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 -- 172.21.15.28:0/2620392993 shutdown_connections 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 -- 172.21.15.28:0/2620392993 wait complete. 2024-09-19T11:25:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 Processor -- start 2024-09-19T11:25:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 -- start start 2024-09-19T11:25:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.489+0000 7f3ff0216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607d84bf120 con 0x5607d84bc470 2024-09-19T11:25:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3fec1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3fec1e8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84c33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44696/0 (socket says 172.21.15.28:44696) 2024-09-19T11:25:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3fec1e8640 1 -- 172.21.15.28:0/900424036 learned_addr learned my addr 172.21.15.28:0/900424036 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3fec1e8640 1 -- 172.21.15.28:0/900424036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d84c3900 con 0x5607d84bc470 2024-09-19T11:25:08.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3fec1e8640 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84c33a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4013040 tx=0x7f3fe4003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3ecbfff640 1 -- 172.21.15.28:0/900424036 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fe4020020 con 0x5607d84bc470 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3ecbfff640 1 -- 172.21.15.28:0/900424036 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe4010780 con 0x5607d84bc470 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3ecbfff640 1 -- 172.21.15.28:0/900424036 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fe4015040 con 0x5607d84bc470 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607d84c3b90 con 0x5607d84bc470 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.490+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d84c4510 con 0x5607d84bc470 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.491+0000 7f3ecbfff640 1 -- 172.21.15.28:0/900424036 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3fe4023020 con 0x5607d84bc470 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.492+0000 7f3ecbfff640 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ec403d000 0x7f3ec403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.492+0000 7f3ecbfff640 1 -- 172.21.15.28:0/900424036 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3fe4053e10 con 0x5607d84bc470 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.492+0000 7f3feb1e6640 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ec403d000 0x7f3ec403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.492+0000 7f3ff0216800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:08.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.492+0000 7f3ff0216800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.492+0000 7f3feb1e6640 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ec403d000 0x7f3ec403f4c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe0001330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.492+0000 7f3ff0216800 1 Processor -- start 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3ff0216800 1 -- start start 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3ff0216800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84dd210 0x5607d84dd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3ff0216800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5607d83b40c0 con 0x5607d84dd210 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3feb9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84dd210 0x5607d84dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3feb9e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84dd210 0x5607d84dd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44700/0 (socket says 172.21.15.28:44700) 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3feb9e7640 1 -- 172.21.15.28:0/201221651 learned_addr learned my addr 172.21.15.28:0/201221651 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3feb9e7640 1 -- 172.21.15.28:0/201221651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d84ddc70 con 0x5607d84dd210 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84dd210 0x5607d84dd630 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc096700 tx=0x7f3fdc009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.493+0000 7f3e73fff640 1 -- 172.21.15.28:0/201221651 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc09d070 con 0x5607d84dd210 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.494+0000 7f3e73fff640 1 -- 172.21.15.28:0/201221651 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc05e420 con 0x5607d84dd210 2024-09-19T11:25:08.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.494+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5607d84de4a0 con 0x5607d84dd210 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.494+0000 7f3e73fff640 1 -- 172.21.15.28:0/201221651 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fdc068420 con 0x5607d84dd210 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.494+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d84dea40 con 0x5607d84dd210 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.494+0000 7f3e73fff640 1 -- 172.21.15.28:0/201221651 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f3fdc0a0070 con 0x5607d84dd210 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3e73fff640 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3e2c03d000 0x7f3e2c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3e73fff640 1 -- 172.21.15.28:0/201221651 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f3fdc092030 con 0x5607d84dd210 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3feb1e6640 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3e2c03d000 0x7f3e2c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3feb1e6640 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3e2c03d000 0x7f3e2c03f4c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000f60 tx=0x7f3fe0006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3e71ffb640 20 reqs_thread_entry: start 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607d84eab30 0x5607d84ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5607d84ed600 con 0x5607d84eab30 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.495+0000 7f3fec1e8640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607d84eab30 0x5607d84ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.496+0000 7f3fec1e8640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607d84eab30 0x5607d84ecf80 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.496+0000 7f3fec1e8640 1 -- 172.21.15.28:0/1334988098 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3fe400e070 con 0x5607d84eab30 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.496+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5607d84edee0 con 0x5607d84eab30 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.496+0000 7f3fec1e8640 1 -- 172.21.15.28:0/1334988098 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3fe400e070 con 0x5607d84eab30 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.496+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607d84ef7f0 0x5607d84f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.496+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5607d84f23d0 con 0x5607d84ef7f0 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.496+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607d84ef7f0 0x5607d84f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.497+0000 7f3feb9e7640 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607d84ef7f0 0x5607d84f1c40 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.497+0000 7f3feb9e7640 1 -- 172.21.15.28:0/1334988098 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~1155 out=1155b] v0'0 uv4 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7f3fdc09f030 con 0x5607d84ef7f0 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ec403d000 msgr2=0x7f3ec403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ec403d000 0x7f3ec403f4c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe0001330 comp rx=0 tx=0).stop 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 msgr2=0x5607d84c33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84c33a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4013040 tx=0x7f3fe4003370 comp rx=0 tx=0).stop 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 shutdown_connections 2024-09-19T11:25:08.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ec403d000 0x7f3ec403f4c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/900424036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84bc470 0x5607d84c33a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 >> 172.21.15.28:0/900424036 conn(0x5607d83b88a0 msgr2=0x5607d84b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 shutdown_connections 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/900424036 wait complete. 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3e2c03d000 msgr2=0x7f3e2c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3e2c03d000 0x7f3e2c03f4c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000f60 tx=0x7f3fe0006000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84dd210 msgr2=0x5607d84dd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84dd210 0x5607d84dd630 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc096700 tx=0x7f3fdc009d90 comp rx=0 tx=0).stop 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 shutdown_connections 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3e2c03d000 0x7f3e2c03f4c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 --2- 172.21.15.28:0/201221651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d84dd210 0x5607d84dd630 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 >> 172.21.15.28:0/201221651 conn(0x5607d84d2790 msgr2=0x5607d84d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 shutdown_connections 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.498+0000 7f3ff0216800 1 -- 172.21.15.28:0/201221651 wait complete. 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607d84eab30 msgr2=0x5607d84ecf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607d84eab30 0x5607d84ecf80 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607d84ef7f0 msgr2=0x5607d84f1c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607d84ef7f0 0x5607d84f1c40 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ed403cfb0 msgr2=0x7f3ed403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ed403cfb0 0x7f3ed403f470 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc07c940 tx=0x7f3fdc054f70 comp rx=0 tx=0).stop 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 msgr2=0x5607d84aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d84aeda0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc045f50 tx=0x7f3fdc002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 shutdown_connections 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5607d84ef7f0 0x5607d84f1c40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f3ed403cfb0 0x7f3ed403f470 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5607d84eab30 0x5607d84ecf80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 --2- 172.21.15.28:0/1334988098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5607d83b5f10 0x5607d84aeda0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 >> 172.21.15.28:0/1334988098 conn(0x5607d83afa50 msgr2=0x5607d84a0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 shutdown_connections 2024-09-19T11:25:08.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.499+0000 7f3ff0216800 1 -- 172.21.15.28:0/1334988098 wait complete. 2024-09-19T11:25:08.509 INFO:teuthology.orchestra.run.smithi028.stdout:{"id":"293899f1-2738-4b64-a8e1-a18ff039d626","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-09-19T11:25:08.509 DEBUG:tasks.util.rgw: json result: {'id': '293899f1-2738-4b64-a8e1-a18ff039d626', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-09-19T11:25:08.510 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-19T11:25:08.510 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-19T11:25:08.510 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-09-19T11:25:08.558 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:08.558 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:08.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.557+0000 7ff951122800 1 Processor -- start 2024-09-19T11:25:08.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.557+0000 7ff951122800 1 -- start start 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.557+0000 7ff951122800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5609247d5e30 0x5609247d6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.557+0000 7ff951122800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5609247d7330 con 0x5609247d6770 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.557+0000 7ff951122800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5609246627d0 con 0x5609247d5e30 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.558+0000 7ff94cd02640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5609247d5e30 0x5609247d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.558+0000 7ff94c501640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5609247d6770 0x5609247d6c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48248/0 (socket says 172.21.15.28:48248) 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.558+0000 7ff94c501640 1 -- 172.21.15.28:0/524184402 learned_addr learned my addr 172.21.15.28:0/524184402 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.558+0000 7ff94bd00640 1 -- 172.21.15.28:0/524184402 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590059495 0 0) 0x5609247d7330 con 0x5609247d6770 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.558+0000 7ff94bd00640 1 -- 172.21.15.28:0/524184402 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff934001e10 con 0x5609247d6770 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.558+0000 7ff94cd02640 1 -- 172.21.15.28:0/524184402 >> v1:172.21.15.22:6789/0 conn(0x5609247d6770 legacy=0x5609247d6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.558+0000 7ff94cd02640 1 -- 172.21.15.28:0/524184402 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609247d7500 con 0x5609247d5e30 2024-09-19T11:25:08.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.560+0000 7ff94cd02640 1 --2- 172.21.15.28:0/524184402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609247d6230 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00a470 tx=0x7ff93c045920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b88f216c9a3d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.560+0000 7ff94bd00640 1 -- 172.21.15.28:0/524184402 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c059070 con 0x5609247d5e30 2024-09-19T11:25:08.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.560+0000 7ff94bd00640 1 -- 172.21.15.28:0/524184402 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c054570 con 0x5609247d5e30 2024-09-19T11:25:08.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.560+0000 7ff94bd00640 1 -- 172.21.15.28:0/524184402 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c054850 con 0x5609247d5e30 2024-09-19T11:25:08.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.560+0000 7ff951122800 1 -- 172.21.15.28:0/524184402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 msgr2=0x5609247d6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.560+0000 7ff951122800 1 --2- 172.21.15.28:0/524184402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609247d6230 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00a470 tx=0x7ff93c045920 comp rx=0 tx=0).stop 2024-09-19T11:25:08.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.561+0000 7ff951122800 1 -- 172.21.15.28:0/524184402 shutdown_connections 2024-09-19T11:25:08.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.561+0000 7ff951122800 1 --2- 172.21.15.28:0/524184402 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609247d6230 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.561+0000 7ff951122800 1 -- 172.21.15.28:0/524184402 >> 172.21.15.28:0/524184402 conn(0x5609247d1b60 msgr2=0x5609247d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.561+0000 7ff951122800 1 -- 172.21.15.28:0/524184402 shutdown_connections 2024-09-19T11:25:08.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.561+0000 7ff951122800 1 -- 172.21.15.28:0/524184402 wait complete. 2024-09-19T11:25:08.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.576+0000 7ff951122800 1 Processor -- start 2024-09-19T11:25:08.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.576+0000 7ff951122800 1 -- start start 2024-09-19T11:25:08.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.576+0000 7ff951122800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248c9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.576+0000 7ff951122800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56092466fdd0 con 0x5609247d5e30 2024-09-19T11:25:08.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.576+0000 7ff94cd02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.576+0000 7ff94cd02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44706/0 (socket says 172.21.15.28:44706) 2024-09-19T11:25:08.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.576+0000 7ff94cd02640 1 -- 172.21.15.28:0/2972861070 learned_addr learned my addr 172.21.15.28:0/2972861070 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.577+0000 7ff94cd02640 1 -- 172.21.15.28:0/2972861070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609248c9a10 con 0x5609247d5e30 2024-09-19T11:25:08.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.577+0000 7ff94cd02640 1 --2- 172.21.15.28:0/2972861070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248c9420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff93c025b20 tx=0x7ff93c04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb01eed00bd8bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.577+0000 7ff83ae0d640 1 -- 172.21.15.28:0/2972861070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c059040 con 0x5609247d5e30 2024-09-19T11:25:08.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.577+0000 7ff83ae0d640 1 -- 172.21.15.28:0/2972861070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c00a2b0 con 0x5609247d5e30 2024-09-19T11:25:08.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.577+0000 7ff83ae0d640 1 -- 172.21.15.28:0/2972861070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c054850 con 0x5609247d5e30 2024-09-19T11:25:08.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 -- 172.21.15.28:0/2972861070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 msgr2=0x5609248c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 --2- 172.21.15.28:0/2972861070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248c9420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff93c025b20 tx=0x7ff93c04e000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 -- 172.21.15.28:0/2972861070 shutdown_connections 2024-09-19T11:25:08.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 --2- 172.21.15.28:0/2972861070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248c9420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 -- 172.21.15.28:0/2972861070 >> 172.21.15.28:0/2972861070 conn(0x5609247d1b60 msgr2=0x5609248c4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 -- 172.21.15.28:0/2972861070 shutdown_connections 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 -- 172.21.15.28:0/2972861070 wait complete. 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 Processor -- start 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 -- start start 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.578+0000 7ff951122800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5609247d7a30 con 0x5609247d5e30 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff94cd02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff94cd02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44710/0 (socket says 172.21.15.28:44710) 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff94cd02640 1 -- 172.21.15.28:0/314419632 learned_addr learned my addr 172.21.15.28:0/314419632 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff94cd02640 1 -- 172.21.15.28:0/314419632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609248d1340 con 0x5609247d5e30 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff94cd02640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248d0de0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff93c025b20 tx=0x7ff93c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff83960a640 1 -- 172.21.15.28:0/314419632 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c059030 con 0x5609247d5e30 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff83960a640 1 -- 172.21.15.28:0/314419632 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c054c20 con 0x5609247d5e30 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff83960a640 1 -- 172.21.15.28:0/314419632 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c05e410 con 0x5609247d5e30 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5609248d15d0 con 0x5609247d5e30 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.579+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609248d1fd0 con 0x5609247d5e30 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff951122800 1 Processor -- start 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff951122800 1 -- start start 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff951122800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff951122800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5609246ce4a0 con 0x5609248de3f0 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff83960a640 1 -- 172.21.15.28:0/314419632 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff93c05e5b0 con 0x5609247d5e30 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff94d503640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff94d503640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44724/0 (socket says 172.21.15.28:44724) 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff94d503640 1 -- 172.21.15.28:0/861395443 learned_addr learned my addr 172.21.15.28:0/861395443 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff83960a640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83403d000 0x7ff83403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff83960a640 1 -- 172.21.15.28:0/314419632 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff93c097840 con 0x5609247d5e30 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.580+0000 7ff94cd02640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83403d000 0x7ff83403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff94d503640 1 -- 172.21.15.28:0/861395443 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609248e0db0 con 0x5609248de3f0 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff94d503640 1 --2- 172.21.15.28:0/861395443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e07e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff94400a3e0 tx=0x7ff94400a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d760d84ed0b26140 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff94cd02640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83403d000 0x7ff83403f4c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff93c079a80 tx=0x7ff93c07d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff832ffd640 1 -- 172.21.15.28:0/861395443 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944015070 con 0x5609248de3f0 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff832ffd640 1 -- 172.21.15.28:0/861395443 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff944001670 con 0x5609248de3f0 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff832ffd640 1 -- 172.21.15.28:0/861395443 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944001990 con 0x5609248de3f0 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff951122800 1 -- 172.21.15.28:0/861395443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 msgr2=0x5609248e07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff951122800 1 --2- 172.21.15.28:0/861395443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e07e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff94400a3e0 tx=0x7ff94400a8a0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff951122800 1 -- 172.21.15.28:0/861395443 shutdown_connections 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff951122800 1 --2- 172.21.15.28:0/861395443 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e07e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff951122800 1 -- 172.21.15.28:0/861395443 >> 172.21.15.28:0/861395443 conn(0x5609247d6770 msgr2=0x5609248d8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff951122800 1 -- 172.21.15.28:0/861395443 shutdown_connections 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.581+0000 7ff951122800 1 -- 172.21.15.28:0/861395443 wait complete. 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.582+0000 7ff951122800 1 Processor -- start 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.582+0000 7ff951122800 1 -- start start 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.582+0000 7ff951122800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.582+0000 7ff951122800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5609248e10a0 con 0x5609248de3f0 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.582+0000 7ff94d503640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.582+0000 7ff94d503640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44732/0 (socket says 172.21.15.28:44732) 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.582+0000 7ff94d503640 1 -- 172.21.15.28:0/1432100729 learned_addr learned my addr 172.21.15.28:0/1432100729 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.583+0000 7ff94d503640 1 -- 172.21.15.28:0/1432100729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609248e58a0 con 0x5609248de3f0 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.583+0000 7ff94d503640 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e5340 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff944013040 tx=0x7ff944003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.583+0000 7ff8317fa640 1 -- 172.21.15.28:0/1432100729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944020020 con 0x5609248de3f0 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.583+0000 7ff8317fa640 1 -- 172.21.15.28:0/1432100729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff944010780 con 0x5609248de3f0 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.583+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5609248e5b30 con 0x5609248de3f0 2024-09-19T11:25:08.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.583+0000 7ff8317fa640 1 -- 172.21.15.28:0/1432100729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff944015040 con 0x5609248de3f0 2024-09-19T11:25:08.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.583+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609248e64b0 con 0x5609248de3f0 2024-09-19T11:25:08.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.584+0000 7ff8317fa640 1 -- 172.21.15.28:0/1432100729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff944023020 con 0x5609248de3f0 2024-09-19T11:25:08.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.584+0000 7ff8317fa640 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82403d000 0x7ff82403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.584+0000 7ff8317fa640 1 -- 172.21.15.28:0/1432100729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff944053e10 con 0x5609248de3f0 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.584+0000 7ff94c501640 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82403d000 0x7ff82403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff951122800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff951122800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff94c501640 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82403d000 0x7ff82403f4c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff940001d30 tx=0x7ff940001740 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff951122800 1 Processor -- start 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff951122800 1 -- start start 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff951122800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248ff1d0 0x5609248ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff951122800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5609248c9d80 con 0x5609248ff1d0 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff94cd02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248ff1d0 0x5609248ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff94cd02640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248ff1d0 0x5609248ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44746/0 (socket says 172.21.15.28:44746) 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.585+0000 7ff94cd02640 1 -- 172.21.15.28:0/2664117371 learned_addr learned my addr 172.21.15.28:0/2664117371 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.586+0000 7ff94cd02640 1 -- 172.21.15.28:0/2664117371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609248ffc30 con 0x5609248ff1d0 2024-09-19T11:25:08.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.586+0000 7ff94cd02640 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248ff1d0 0x5609248ff5f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005870 tx=0x7ff93c066780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.586+0000 7ff8117e2640 1 -- 172.21.15.28:0/2664117371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c066520 con 0x5609248ff1d0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.586+0000 7ff8117e2640 1 -- 172.21.15.28:0/2664117371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c079350 con 0x5609248ff1d0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.586+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560924900460 con 0x5609248ff1d0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.586+0000 7ff8117e2640 1 -- 172.21.15.28:0/2664117371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93c05d370 con 0x5609248ff1d0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.586+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560924900a00 con 0x5609248ff1d0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.587+0000 7ff8117e2640 1 -- 172.21.15.28:0/2664117371 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff93c097a90 con 0x5609248ff1d0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.587+0000 7ff8117e2640 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff78c03d050 0x7ff78c03f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.587+0000 7ff8117e2640 1 -- 172.21.15.28:0/2664117371 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff93c082070 con 0x5609248ff1d0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.587+0000 7ff94c501640 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff78c03d050 0x7ff78c03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.587+0000 7ff7a2ffd640 20 reqs_thread_entry: start 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff94c501640 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff78c03d050 0x7ff78c03f510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff940004170 tx=0x7ff94000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56092490cb40 0x56092490efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x56092490f680 con 0x56092490cb40 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff94d503640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56092490cb40 0x56092490efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff94d503640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56092490cb40 0x56092490efb0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff94d503640 1 -- 172.21.15.28:0/314419632 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff94400e070 con 0x56092490cb40 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560924911450 0x560924913840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:2 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x560924913f40 con 0x560924911450 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.588+0000 7ff94cd02640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560924911450 0x560924913840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.589+0000 7ff94cd02640 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560924911450 0x560924913840 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.589+0000 7ff94cd02640 1 -- 172.21.15.28:0/314419632 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(2 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~1155 out=1155b] v0'0 uv4 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7ff93c078070 con 0x560924911450 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.589+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x560924910af0 con 0x560924911450 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.590+0000 7ff94cd02640 1 -- 172.21.15.28:0/314419632 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [stat,call,call,writefull 0~993] v33'5 uv5 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7ff93c078070 con 0x560924911450 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82403d000 msgr2=0x7ff82403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82403d000 0x7ff82403f4c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff940001d30 tx=0x7ff940001740 comp rx=0 tx=0).stop 2024-09-19T11:25:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 msgr2=0x5609248e5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e5340 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff944013040 tx=0x7ff944003370 comp rx=0 tx=0).stop 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 shutdown_connections 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff82403d000 0x7ff82403f4c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 --2- 172.21.15.28:0/1432100729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248de3f0 0x5609248e5340 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 >> 172.21.15.28:0/1432100729 conn(0x5609247d6770 msgr2=0x5609248d8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 shutdown_connections 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.591+0000 7ff951122800 1 -- 172.21.15.28:0/1432100729 wait complete. 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff78c03d050 msgr2=0x7ff78c03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff78c03d050 0x7ff78c03f510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff940004170 tx=0x7ff94000f000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248ff1d0 msgr2=0x5609248ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248ff1d0 0x5609248ff5f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff93c005870 tx=0x7ff93c066780 comp rx=0 tx=0).stop 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 shutdown_connections 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff78c03d050 0x7ff78c03f510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/2664117371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609248ff1d0 0x5609248ff5f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 >> 172.21.15.28:0/2664117371 conn(0x5609248f4710 msgr2=0x5609248f4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 shutdown_connections 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/2664117371 wait complete. 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56092490cb40 msgr2=0x56092490efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56092490cb40 0x56092490efb0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560924911450 msgr2=0x560924913840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560924911450 0x560924913840 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83403d000 msgr2=0x7ff83403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83403d000 0x7ff83403f4c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff93c079a80 tx=0x7ff93c07d040 comp rx=0 tx=0).stop 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 msgr2=0x5609248d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248d0de0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff93c025b20 tx=0x7ff93c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 shutdown_connections 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x560924911450 0x560924913840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff83403d000 0x7ff83403f4c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x56092490cb40 0x56092490efb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 --2- 172.21.15.28:0/314419632 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5609247d5e30 0x5609248d0de0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 >> 172.21.15.28:0/314419632 conn(0x5609247d1b60 msgr2=0x5609248ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 shutdown_connections 2024-09-19T11:25:08.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.592+0000 7ff951122800 1 -- 172.21.15.28:0/314419632 wait complete. 2024-09-19T11:25:08.603 INFO:teuthology.orchestra.run.smithi028.stdout:{"id":"293899f1-2738-4b64-a8e1-a18ff039d626","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-09-19T11:25:08.603 DEBUG:tasks.util.rgw: json result: {'id': '293899f1-2738-4b64-a8e1-a18ff039d626', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-09-19T11:25:08.603 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-09-19T11:25:08.604 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-09-19T11:25:08.604 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-09-19T11:25:08.647 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:08.647 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f8094126800 1 Processor -- start 2024-09-19T11:25:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f8094126800 1 -- start start 2024-09-19T11:25:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f8094126800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e08f065230 0x55e08f0f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f8094126800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e08f0f9920 con 0x55e08f0665e0 2024-09-19T11:25:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f8094126800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55e08ef7b7d0 con 0x55e08f065230 2024-09-19T11:25:08.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f808f5d8640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55e08f065230 0x55e08f0f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f808fdd9640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55e08f0665e0 0x55e08f0649b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48256/0 (socket says 172.21.15.28:48256) 2024-09-19T11:25:08.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f808fdd9640 1 -- 172.21.15.28:0/3866793979 learned_addr learned my addr 172.21.15.28:0/3866793979 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:08.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f808f5d8640 1 -- 172.21.15.28:0/3866793979 >> v1:172.21.15.22:6789/0 conn(0x55e08f0665e0 legacy=0x55e08f0649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.647+0000 7f808f5d8640 1 -- 172.21.15.28:0/3866793979 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08f0fab00 con 0x55e08f065230 2024-09-19T11:25:08.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.649+0000 7f808f5d8640 1 --2- 172.21.15.28:0/3866793979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f0f93e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f808400a470 tx=0x7f8084045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4964dab8d314f4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.649+0000 7f808edd7640 1 -- 172.21.15.28:0/3866793979 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8084053570 con 0x55e08f065230 2024-09-19T11:25:08.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.649+0000 7f808edd7640 1 -- 172.21.15.28:0/3866793979 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8084058070 con 0x55e08f065230 2024-09-19T11:25:08.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.649+0000 7f808edd7640 1 -- 172.21.15.28:0/3866793979 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8084053880 con 0x55e08f065230 2024-09-19T11:25:08.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.650+0000 7f8094126800 1 -- 172.21.15.28:0/3866793979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 msgr2=0x55e08f0f93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.650+0000 7f8094126800 1 --2- 172.21.15.28:0/3866793979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f0f93e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f808400a470 tx=0x7f8084045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.650+0000 7f8094126800 1 -- 172.21.15.28:0/3866793979 shutdown_connections 2024-09-19T11:25:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.650+0000 7f8094126800 1 --2- 172.21.15.28:0/3866793979 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f0f93e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.650+0000 7f8094126800 1 -- 172.21.15.28:0/3866793979 >> 172.21.15.28:0/3866793979 conn(0x55e08f0ebc50 msgr2=0x55e08f0ee0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.650+0000 7f8094126800 1 -- 172.21.15.28:0/3866793979 shutdown_connections 2024-09-19T11:25:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.650+0000 7f8094126800 1 -- 172.21.15.28:0/3866793979 wait complete. 2024-09-19T11:25:08.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.665+0000 7f8094126800 1 Processor -- start 2024-09-19T11:25:08.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.665+0000 7f8094126800 1 -- start start 2024-09-19T11:25:08.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.665+0000 7f8094126800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.665+0000 7f8094126800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e08ef88dd0 con 0x55e08f065230 2024-09-19T11:25:08.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f808fdd9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f808fdd9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44760/0 (socket says 172.21.15.28:44760) 2024-09-19T11:25:08.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f808fdd9640 1 -- 172.21.15.28:0/224345096 learned_addr learned my addr 172.21.15.28:0/224345096 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f808fdd9640 1 -- 172.21.15.28:0/224345096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08f1e2ab0 con 0x55e08f065230 2024-09-19T11:25:08.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f808fdd9640 1 --2- 172.21.15.28:0/224345096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e24c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8080001e40 tx=0x7f8080001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a016f3c008ec33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f7f7de0b640 1 -- 172.21.15.28:0/224345096 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8080012020 con 0x55e08f065230 2024-09-19T11:25:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f7f7de0b640 1 -- 172.21.15.28:0/224345096 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808000e5a0 con 0x55e08f065230 2024-09-19T11:25:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.666+0000 7f7f7de0b640 1 -- 172.21.15.28:0/224345096 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f808000e8a0 con 0x55e08f065230 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 -- 172.21.15.28:0/224345096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 msgr2=0x55e08f1e24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 --2- 172.21.15.28:0/224345096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e24c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8080001e40 tx=0x7f8080001e70 comp rx=0 tx=0).stop 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 -- 172.21.15.28:0/224345096 shutdown_connections 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 --2- 172.21.15.28:0/224345096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e24c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 -- 172.21.15.28:0/224345096 >> 172.21.15.28:0/224345096 conn(0x55e08f0ebc50 msgr2=0x55e08f1dde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 -- 172.21.15.28:0/224345096 shutdown_connections 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 -- 172.21.15.28:0/224345096 wait complete. 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 Processor -- start 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.667+0000 7f8094126800 1 -- start start 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f8094126800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f8094126800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e08f0f9d60 con 0x55e08f065230 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f808fdd9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f808fdd9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44768/0 (socket says 172.21.15.28:44768) 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f808fdd9640 1 -- 172.21.15.28:0/4273282953 learned_addr learned my addr 172.21.15.28:0/4273282953 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f808fdd9640 1 -- 172.21.15.28:0/4273282953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08f1ea540 con 0x55e08f065230 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f808fdd9640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e9fe0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f80800012b0 tx=0x7f8080000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f7f77fff640 1 -- 172.21.15.28:0/4273282953 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8080012030 con 0x55e08f065230 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f7f77fff640 1 -- 172.21.15.28:0/4273282953 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808001a750 con 0x55e08f065230 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e08f1ea7d0 con 0x55e08f065230 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f7f77fff640 1 -- 172.21.15.28:0/4273282953 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f808001aa70 con 0x55e08f065230 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.668+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e08f1eb1d0 con 0x55e08f065230 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f8094126800 1 Processor -- start 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f8094126800 1 -- start start 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f8094126800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f7f77fff640 1 -- 172.21.15.28:0/4273282953 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8080015020 con 0x55e08f065230 2024-09-19T11:25:08.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f8094126800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e08efe74a0 con 0x55e08f1f75d0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f80905da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f80905da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44770/0 (socket says 172.21.15.28:44770) 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.669+0000 7f80905da640 1 -- 172.21.15.28:0/1046394606 learned_addr learned my addr 172.21.15.28:0/1046394606 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f7f77fff640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f7803d000 0x7f7f7803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f7f77fff640 1 -- 172.21.15.28:0/4273282953 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f8080020030 con 0x55e08f065230 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f808fdd9640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f7803d000 0x7f7f7803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f80905da640 1 -- 172.21.15.28:0/1046394606 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08f1f9f90 con 0x55e08f1f75d0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f80905da640 1 --2- 172.21.15.28:0/1046394606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1f99c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f808800a3e0 tx=0x7f808800a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa7cb072365f934 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f808fdd9640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f7803d000 0x7f7f7803f4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8080016b40 tx=0x7f808001ae70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f7f75ffb640 1 -- 172.21.15.28:0/1046394606 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088015070 con 0x55e08f1f75d0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f7f75ffb640 1 -- 172.21.15.28:0/1046394606 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8088001670 con 0x55e08f1f75d0 2024-09-19T11:25:08.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f7f75ffb640 1 -- 172.21.15.28:0/1046394606 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088001990 con 0x55e08f1f75d0 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.670+0000 7f8094126800 1 -- 172.21.15.28:0/1046394606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 msgr2=0x55e08f1f99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 --2- 172.21.15.28:0/1046394606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1f99c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f808800a3e0 tx=0x7f808800a8a0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 -- 172.21.15.28:0/1046394606 shutdown_connections 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 --2- 172.21.15.28:0/1046394606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1f99c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 -- 172.21.15.28:0/1046394606 >> 172.21.15.28:0/1046394606 conn(0x55e08f0665e0 msgr2=0x55e08f1f1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 -- 172.21.15.28:0/1046394606 shutdown_connections 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 -- 172.21.15.28:0/1046394606 wait complete. 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 Processor -- start 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 -- start start 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1fe530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f8094126800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e08f1fa280 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f80905da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f80905da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1fe530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44784/0 (socket says 172.21.15.28:44784) 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.671+0000 7f80905da640 1 -- 172.21.15.28:0/1260370315 learned_addr learned my addr 172.21.15.28:0/1260370315 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.672+0000 7f80905da640 1 -- 172.21.15.28:0/1260370315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08f1fea90 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.672+0000 7f80905da640 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1fe530 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8088013040 tx=0x7f8088003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.672+0000 7f7f6ffff640 1 -- 172.21.15.28:0/1260370315 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088020020 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.672+0000 7f7f6ffff640 1 -- 172.21.15.28:0/1260370315 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8088010780 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.672+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e08f1fed20 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.672+0000 7f7f6ffff640 1 -- 172.21.15.28:0/1260370315 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8088015040 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.672+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e08f1ff6a0 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.673+0000 7f7f6ffff640 1 -- 172.21.15.28:0/1260370315 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8088023020 con 0x55e08f1f75d0 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.673+0000 7f7f6ffff640 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f6803d000 0x7f7f6803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.673+0000 7f7f6ffff640 1 -- 172.21.15.28:0/1260370315 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f8088053e10 con 0x55e08f1f75d0 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.673+0000 7f808f5d8640 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f6803d000 0x7f7f6803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f8094126800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f8094126800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f808f5d8640 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f6803d000 0x7f7f6803f4c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8084057000 tx=0x7f8084052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f8094126800 1 Processor -- start 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f8094126800 1 -- start start 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f8094126800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f218340 0x55e08f218760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f8094126800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e08f1e2ee0 con 0x55e08f218340 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f808fdd9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f218340 0x55e08f218760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f808fdd9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f218340 0x55e08f218760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44800/0 (socket says 172.21.15.28:44800) 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.674+0000 7f808fdd9640 1 -- 172.21.15.28:0/3567814871 learned_addr learned my addr 172.21.15.28:0/3567814871 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.675+0000 7f808fdd9640 1 -- 172.21.15.28:0/3567814871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08f218da0 con 0x55e08f218340 2024-09-19T11:25:08.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.675+0000 7f808fdd9640 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f218340 0x55e08f218760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f808000efd0 tx=0x7f8080052a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.675+0000 7f7ef7fff640 1 -- 172.21.15.28:0/3567814871 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8080018e30 con 0x55e08f218340 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.675+0000 7f7ef7fff640 1 -- 172.21.15.28:0/3567814871 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080052890 con 0x55e08f218340 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.675+0000 7f7ef7fff640 1 -- 172.21.15.28:0/3567814871 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f808000c8f0 con 0x55e08f218340 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.675+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e08f2195d0 con 0x55e08f218340 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.675+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e08f219b70 con 0x55e08f218340 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.676+0000 7f7ef7fff640 1 -- 172.21.15.28:0/3567814871 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f808000ca90 con 0x55e08f218340 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.676+0000 7f7ef7fff640 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7ec803d000 0x7f7ec803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.676+0000 7f7ef7fff640 1 -- 172.21.15.28:0/3567814871 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f8080040020 con 0x55e08f218340 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.676+0000 7f808f5d8640 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7ec803d000 0x7f7ec803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.676+0000 7f7ef5ffb640 20 reqs_thread_entry: start 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.676+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e08f225cb0 0x55e08f228120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55e08f228760 con 0x55e08f225cb0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f808f5d8640 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7ec803d000 0x7f7ec803f4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8084045a60 tx=0x7f8084005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f80905da640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e08f225cb0 0x55e08f228120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f80905da640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e08f225cb0 0x55e08f228120 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f80905da640 1 -- 172.21.15.28:0/4273282953 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f808800e070 con 0x55e08f225cb0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55e08f228ce0 con 0x55e08f225cb0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f80905da640 1 -- 172.21.15.28:0/4273282953 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f808800e070 con 0x55e08f225cb0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e08f22a900 0x55e08f22cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.677+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55e08f22d520 con 0x55e08f22a900 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.678+0000 7f808fdd9640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e08f22a900 0x55e08f22cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.678+0000 7f808fdd9640 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e08f22a900 0x55e08f22cd90 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.678+0000 7f808fdd9640 1 -- 172.21.15.28:0/4273282953 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8080034070 con 0x55e08f22a900 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f6803d000 msgr2=0x7f7f6803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f6803d000 0x7f7f6803f4c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8084057000 tx=0x7f8084052000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 msgr2=0x55e08f1fe530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1fe530 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8088013040 tx=0x7f8088003370 comp rx=0 tx=0).stop 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 shutdown_connections 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f6803d000 0x7f7f6803f4c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/1260370315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f1f75d0 0x55e08f1fe530 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 >> 172.21.15.28:0/1260370315 conn(0x55e08f0665e0 msgr2=0x55e08f1f1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 shutdown_connections 2024-09-19T11:25:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/1260370315 wait complete. 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7ec803d000 msgr2=0x7f7ec803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7ec803d000 0x7f7ec803f4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8084045a60 tx=0x7f8084005a00 comp rx=0 tx=0).stop 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f218340 msgr2=0x55e08f218760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f218340 0x55e08f218760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f808000efd0 tx=0x7f8080052a70 comp rx=0 tx=0).stop 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 shutdown_connections 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7ec803d000 0x7f7ec803f4c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 --2- 172.21.15.28:0/3567814871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f218340 0x55e08f218760 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.679+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 >> 172.21.15.28:0/3567814871 conn(0x55e08f20d8a0 msgr2=0x55e08f20dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 shutdown_connections 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/3567814871 wait complete. 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e08f225cb0 msgr2=0x55e08f228120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e08f225cb0 0x55e08f228120 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e08f22a900 msgr2=0x55e08f22cd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e08f22a900 0x55e08f22cd90 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f7803d000 msgr2=0x7f7f7803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f7803d000 0x7f7f7803f4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8080016b40 tx=0x7f808001ae70 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 msgr2=0x55e08f1e9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e9fe0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f80800012b0 tx=0x7f8080000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 shutdown_connections 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55e08f22a900 0x55e08f22cd90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f7803d000 0x7f7f7803f4c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55e08f225cb0 0x55e08f228120 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 --2- 172.21.15.28:0/4273282953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e08f065230 0x55e08f1e9fe0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 >> 172.21.15.28:0/4273282953 conn(0x55e08f0ebc50 msgr2=0x55e08f1e39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 shutdown_connections 2024-09-19T11:25:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.680+0000 7f8094126800 1 -- 172.21.15.28:0/4273282953 wait complete. 2024-09-19T11:25:08.690 INFO:teuthology.orchestra.run.smithi028.stdout:{"id":"d3056e04-d444-48b9-8e61-96c58b445cdd","name":"default","api_name":"default","is_master":true,"endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"293899f1-2738-4b64-a8e1-a18ff039d626","zones":[{"id":"293899f1-2738-4b64-a8e1-a18ff039d626","name":"default","endpoints":[],"log_meta":false,"log_data":false,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]} 2024-09-19T11:25:08.691 DEBUG:tasks.util.rgw: json result: {'id': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': '293899f1-2738-4b64-a8e1-a18ff039d626', 'zones': [{'id': '293899f1-2738-4b64-a8e1-a18ff039d626', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['notification_v2', 'resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['notification_v2', 'resharding']} 2024-09-19T11:25:08.693 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2024-09-19T11:25:08.716 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-09-19T11:25:08.716 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-09-19T11:25:08.716 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2024-09-19T11:25:08.773 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:08.773 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:08.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.772+0000 7fd893d26800 1 Processor -- start 2024-09-19T11:25:08.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.772+0000 7fd893d26800 1 -- start start 2024-09-19T11:25:08.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.772+0000 7fd893d26800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a54dfde8e0 0x55a54dfdf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.772+0000 7fd893d26800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a54dfdef90 con 0x55a54dfdf9e0 2024-09-19T11:25:08.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.772+0000 7fd893d26800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55a54de697d0 con 0x55a54dfde8e0 2024-09-19T11:25:08.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.772+0000 7fd88f9d9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55a54dfde8e0 0x55a54dfdf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.772+0000 7fd88f1d8640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55a54dfdf9e0 0x55a54dfe4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48268/0 (socket says 172.21.15.28:48268) 2024-09-19T11:25:08.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.773+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1680644134 learned_addr learned my addr 172.21.15.28:0/1680644134 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:08.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.773+0000 7fd88e9d7640 1 -- 172.21.15.28:0/1680644134 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632338935 0 0) 0x55a54dfdef90 con 0x55a54dfdf9e0 2024-09-19T11:25:08.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.773+0000 7fd88e9d7640 1 -- 172.21.15.28:0/1680644134 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd87c001e10 con 0x55a54dfdf9e0 2024-09-19T11:25:08.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.773+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1680644134 >> v1:172.21.15.22:6789/0 conn(0x55a54dfdf9e0 legacy=0x55a54dfe4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.773+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1680644134 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54dfdfe90 con 0x55a54dfde8e0 2024-09-19T11:25:08.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/1680644134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54dfdf4a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd880045b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d891868511ea6014 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd88e9d7640 1 -- 172.21.15.28:0/1680644134 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880055070 con 0x55a54dfde8e0 2024-09-19T11:25:08.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd88e9d7640 1 -- 172.21.15.28:0/1680644134 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd880051540 con 0x55a54dfde8e0 2024-09-19T11:25:08.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd88e9d7640 1 -- 172.21.15.28:0/1680644134 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880051840 con 0x55a54dfde8e0 2024-09-19T11:25:08.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd893d26800 1 -- 172.21.15.28:0/1680644134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 msgr2=0x55a54dfdf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd893d26800 1 --2- 172.21.15.28:0/1680644134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54dfdf4a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd880045b30 comp rx=0 tx=0).stop 2024-09-19T11:25:08.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd893d26800 1 -- 172.21.15.28:0/1680644134 shutdown_connections 2024-09-19T11:25:08.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd893d26800 1 --2- 172.21.15.28:0/1680644134 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54dfdf4a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd893d26800 1 -- 172.21.15.28:0/1680644134 >> 172.21.15.28:0/1680644134 conn(0x55a54dfd9cb0 msgr2=0x55a54dfdc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd893d26800 1 -- 172.21.15.28:0/1680644134 shutdown_connections 2024-09-19T11:25:08.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.775+0000 7fd893d26800 1 -- 172.21.15.28:0/1680644134 wait complete. 2024-09-19T11:25:08.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.790+0000 7fd893d26800 1 Processor -- start 2024-09-19T11:25:08.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.790+0000 7fd893d26800 1 -- start start 2024-09-19T11:25:08.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.791+0000 7fd893d26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54e0ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.791+0000 7fd893d26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a54de76dd0 con 0x55a54dfde8e0 2024-09-19T11:25:08.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.791+0000 7fd88f9d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54e0ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.791+0000 7fd88f9d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54e0ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44818/0 (socket says 172.21.15.28:44818) 2024-09-19T11:25:08.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.791+0000 7fd88f9d9640 1 -- 172.21.15.28:0/3887395042 learned_addr learned my addr 172.21.15.28:0/3887395042 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.791+0000 7fd88f9d9640 1 -- 172.21.15.28:0/3887395042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54e0ce8d0 con 0x55a54dfde8e0 2024-09-19T11:25:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.791+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/3887395042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54e0ce2e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd880051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cea0e3680f05b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd77de0b640 1 -- 172.21.15.28:0/3887395042 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880069070 con 0x55a54dfde8e0 2024-09-19T11:25:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd77de0b640 1 -- 172.21.15.28:0/3887395042 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8800518e0 con 0x55a54dfde8e0 2024-09-19T11:25:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd77de0b640 1 -- 172.21.15.28:0/3887395042 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880055040 con 0x55a54dfde8e0 2024-09-19T11:25:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd893d26800 1 -- 172.21.15.28:0/3887395042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 msgr2=0x55a54e0ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd893d26800 1 --2- 172.21.15.28:0/3887395042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54e0ce2e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd880051000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd893d26800 1 -- 172.21.15.28:0/3887395042 shutdown_connections 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd893d26800 1 --2- 172.21.15.28:0/3887395042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54e0ce2e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd893d26800 1 -- 172.21.15.28:0/3887395042 >> 172.21.15.28:0/3887395042 conn(0x55a54dfd9cb0 msgr2=0x55a54e0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd893d26800 1 -- 172.21.15.28:0/3887395042 shutdown_connections 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.792+0000 7fd893d26800 1 -- 172.21.15.28:0/3887395042 wait complete. 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd893d26800 1 Processor -- start 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd893d26800 1 -- start start 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd893d26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54df54140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd893d26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a54dfe5e60 con 0x55a54dfde8e0 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd88f9d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54df54140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd88f9d9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54df54140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44828/0 (socket says 172.21.15.28:44828) 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd88f9d9640 1 -- 172.21.15.28:0/660341346 learned_addr learned my addr 172.21.15.28:0/660341346 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.793+0000 7fd88f9d9640 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54df546a0 con 0x55a54dfde8e0 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.794+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54df54140 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd880003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.794+0000 7fd777fff640 1 -- 172.21.15.28:0/660341346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880069040 con 0x55a54dfde8e0 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.794+0000 7fd777fff640 1 -- 172.21.15.28:0/660341346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd880072070 con 0x55a54dfde8e0 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.794+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a54df52a50 con 0x55a54dfde8e0 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.794+0000 7fd777fff640 1 -- 172.21.15.28:0/660341346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd880055040 con 0x55a54dfde8e0 2024-09-19T11:25:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.794+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54df52fb0 con 0x55a54dfde8e0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd777fff640 1 -- 172.21.15.28:0/660341346 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd880058070 con 0x55a54dfde8e0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd777fff640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd77803cfb0 0x7fd77803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd777fff640 1 -- 172.21.15.28:0/660341346 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fd880061030 con 0x55a54dfde8e0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd88f1d8640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd77803cfb0 0x7fd77803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54dfdf9e0 0x55a54e0df130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e0df780 con 0x55a54dfdf9e0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd8901da640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54dfdf9e0 0x55a54e0df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd88f1d8640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd77803cfb0 0x7fd77803f470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd878001db0 tx=0x7fd878001310 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.795+0000 7fd8901da640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54dfdf9e0 0x55a54e0df130 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.796+0000 7fd8901da640 1 -- 172.21.15.28:0/660341346 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54dfdf9e0 2024-09-19T11:25:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.796+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e0e0040 con 0x55a54dfdf9e0 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.796+0000 7fd8901da640 1 -- 172.21.15.28:0/660341346 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54dfdf9e0 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.796+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e0e1a70 0x55a54e0e3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.796+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e0e4690 con 0x55a54e0e1a70 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.796+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e0e1a70 0x55a54e0e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.796+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e0e1a70 0x55a54e0e3f00 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.797+0000 7fd88f9d9640 1 -- 172.21.15.28:0/660341346 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd880057030 con 0x55a54e0e1a70 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.797+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e0e0890 con 0x55a54dfdf9e0 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.797+0000 7fd8901da640 1 -- 172.21.15.28:0/660341346 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54dfdf9e0 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.797+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e0e0c30 con 0x55a54e0e1a70 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.797+0000 7fd88f9d9640 1 -- 172.21.15.28:0/660341346 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd880057030 con 0x55a54e0e1a70 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.798+0000 7fd893d26800 1 Processor -- start 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.798+0000 7fd893d26800 1 -- start start 2024-09-19T11:25:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.798+0000 7fd893d26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0f2ee0 0x55a54e0f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.798+0000 7fd893d26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a54ded54a0 con 0x55a54e0f2ee0 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.798+0000 7fd8901da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0f2ee0 0x55a54e0f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.798+0000 7fd8901da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0f2ee0 0x55a54e0f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44842/0 (socket says 172.21.15.28:44842) 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.798+0000 7fd8901da640 1 -- 172.21.15.28:0/2558454836 learned_addr learned my addr 172.21.15.28:0/2558454836 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.799+0000 7fd8901da640 1 -- 172.21.15.28:0/2558454836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54e0f3940 con 0x55a54e0f2ee0 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.799+0000 7fd8901da640 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0f2ee0 0x55a54e0f3300 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55a54df53b70 tx=0x7fd88800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.799+0000 7fd7757fa640 1 -- 172.21.15.28:0/2558454836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd88801c020 con 0x55a54e0f2ee0 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.799+0000 7fd7757fa640 1 -- 172.21.15.28:0/2558454836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd888007c90 con 0x55a54e0f2ee0 2024-09-19T11:25:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.799+0000 7fd7757fa640 1 -- 172.21.15.28:0/2558454836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd888018440 con 0x55a54e0f2ee0 2024-09-19T11:25:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.799+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a54e0f4170 con 0x55a54e0f2ee0 2024-09-19T11:25:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.799+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54e0f4710 con 0x55a54e0f2ee0 2024-09-19T11:25:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.800+0000 7fd7757fa640 1 -- 172.21.15.28:0/2558454836 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd88801f020 con 0x55a54e0f2ee0 2024-09-19T11:25:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.800+0000 7fd7757fa640 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd76803d000 0x7fd76803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.800+0000 7fd7757fa640 1 -- 172.21.15.28:0/2558454836 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fd888055ad0 con 0x55a54e0f2ee0 2024-09-19T11:25:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.800+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd76803d000 0x7fd76803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.800+0000 7fd893d26800 1 Processor -- start 2024-09-19T11:25:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd893d26800 1 -- start start 2024-09-19T11:25:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd76803d000 0x7fd76803f4c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55a54df536f0 tx=0x7fd880096130 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd893d26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e0ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd893d26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a54e0cec40 con 0x55a54e0ff9a0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd8901da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e0ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd8901da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e0ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44856/0 (socket says 172.21.15.28:44856) 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd8901da640 1 -- 172.21.15.28:0/3028821527 learned_addr learned my addr 172.21.15.28:0/3028821527 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd8901da640 1 -- 172.21.15.28:0/3028821527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54e100390 con 0x55a54e0ff9a0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd8901da640 1 --2- 172.21.15.28:0/3028821527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e0ffda0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd888053d80 tx=0x7fd888015630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3ba23a2b445a4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd76effd640 1 -- 172.21.15.28:0/3028821527 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd88804a070 con 0x55a54e0ff9a0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd76effd640 1 -- 172.21.15.28:0/3028821527 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8880174d0 con 0x55a54e0ff9a0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.801+0000 7fd76effd640 1 -- 172.21.15.28:0/3028821527 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd888052070 con 0x55a54e0ff9a0 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 -- 172.21.15.28:0/3028821527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 msgr2=0x55a54e0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 --2- 172.21.15.28:0/3028821527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e0ffda0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd888053d80 tx=0x7fd888015630 comp rx=0 tx=0).stop 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 -- 172.21.15.28:0/3028821527 shutdown_connections 2024-09-19T11:25:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 --2- 172.21.15.28:0/3028821527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e0ffda0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 -- 172.21.15.28:0/3028821527 >> 172.21.15.28:0/3028821527 conn(0x55a54e0fbda0 msgr2=0x55a54e0f9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 -- 172.21.15.28:0/3028821527 shutdown_connections 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 -- 172.21.15.28:0/3028821527 wait complete. 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 Processor -- start 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 -- start start 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.802+0000 7fd893d26800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a54e1009b0 con 0x55a54e0ff9a0 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd8901da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd8901da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44860/0 (socket says 172.21.15.28:44860) 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 learned_addr learned my addr 172.21.15.28:0/1054092954 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54e107560 con 0x55a54e0ff9a0 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd8901da640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e107000 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd888016010 tx=0x7fd888053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd76d7fa640 1 -- 172.21.15.28:0/1054092954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd88804a040 con 0x55a54e0ff9a0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd76d7fa640 1 -- 172.21.15.28:0/1054092954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd888013320 con 0x55a54e0ff9a0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a54e1077f0 con 0x55a54e0ff9a0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd76d7fa640 1 -- 172.21.15.28:0/1054092954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd888013640 con 0x55a54e0ff9a0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.803+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54e108380 con 0x55a54e0ff9a0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.804+0000 7fd76d7fa640 1 -- 172.21.15.28:0/1054092954 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fd88804d070 con 0x55a54e0ff9a0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.804+0000 7fd76d7fa640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd75803d000 0x7fd75803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.804+0000 7fd76d7fa640 1 -- 172.21.15.28:0/1054092954 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fd88800aec0 con 0x55a54e0ff9a0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.804+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd75803d000 0x7fd75803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.805+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd75803d000 0x7fd75803f4c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd880046030 tx=0x7fd880091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.805+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.805+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54e1128c0 0x55a54e114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.805+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e1154d0 con 0x55a54e1128c0 2024-09-19T11:25:08.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.805+0000 7fd88f1d8640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54e1128c0 0x55a54e114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.805+0000 7fd88f1d8640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54e1128c0 0x55a54e114d80 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd878009050 con 0x55a54e1128c0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 realm 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e111c10 con 0x55a54e1128c0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd878009050 con 0x55a54e1128c0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e115fc0 con 0x55a54e1128c0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd878009050 con 0x55a54e1128c0 2024-09-19T11:25:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e116890 con 0x55a54e1128c0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd878009050 con 0x55a54e1128c0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e1172b0 0x55a54e1196a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e119de0 con 0x55a54e1172b0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.806+0000 7fd8901da640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e1172b0 0x55a54e1196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd8901da640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e1172b0 0x55a54e1196a0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd893d26800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd893d26800 20 searching for the correct realm 2024-09-19T11:25:08.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd8901da640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd888021020 0x7fd88800a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.807+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd888021020 0x7fd88800a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.808+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd888021020 0x7fd88800a500 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.808+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.808+0000 7fd88f9d9640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd88005a070 0x7fd880097860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.808+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd88005a070 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.808+0000 7fd88f1d8640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd88005a070 0x7fd880097860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.809+0000 7fd88f1d8640 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd88005a070 0x7fd880097860 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.809+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878009050 con 0x7fd88005a070 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.809+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.809+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878009050 con 0x55a54e1128c0 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.809+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.809+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.809+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd88005a070 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.810+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.810+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd88005a070 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.810+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.810+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.810+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.810+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.810+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.811+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.811+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.811+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd88005a070 2024-09-19T11:25:08.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.811+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.811+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.811+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.811+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.812+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.813+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd88005a070 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.814+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd88005a070 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd88005a070 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.815+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1172b0 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x55a54e1128c0 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b730 con 0x7fd888021020 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:08.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.816+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11a7a0 con 0x55a54e1128c0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11b040 con 0x55a54e1128c0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11bbb0 con 0x55a54e1172b0 2024-09-19T11:25:08.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 zone default found 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 4 Realm: () 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 zonegroup default 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.817+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11bf50 con 0x7fd88005a070 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11e210 con 0x55a54e1172b0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11e5b0 con 0x55a54e1172b0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 20 started sync module instance, tier type = 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.818+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11d650 con 0x7fd888021020 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.819+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.819+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94168467755424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11e9c0 con 0x7fd888021020 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.819+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11f210 con 0x55a54e1128c0 2024-09-19T11:25:08.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.820+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94168467755424] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.821+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.821+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94168467761600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11f290 con 0x55a54e1128c0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.821+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11fae0 con 0x55a54e1172b0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.822+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94168467761600] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.822+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.822+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94168467763136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e11f9b0 con 0x55a54e1172b0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.822+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e1204a0 con 0x7fd888021020 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.823+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94168467763136] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.823+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.823+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94168467767152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e120460 con 0x7fd888021020 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.823+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e120f50 con 0x55a54e1128c0 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.824+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94168467767152] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.825+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.825+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94168467764256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e120f50 con 0x55a54e1128c0 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.825+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e1219a0 con 0x7fd88005a070 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.826+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.826+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94168467772592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e121910 con 0x7fd88005a070 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.826+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e122920 con 0x7fd88005a070 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.826+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94168467764256] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.827+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.827+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94168467769904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e122920 con 0x7fd88005a070 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.827+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e123330 con 0x7fd88005a070 2024-09-19T11:25:08.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.827+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94168467772592] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94168467769904] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94168467776480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e122e80 con 0x7fd88005a070 2024-09-19T11:25:08.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 20 add_watcher() i=0 2024-09-19T11:25:08.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 20 add_watcher() i=1 2024-09-19T11:25:08.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 20 add_watcher() i=2 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 20 add_watcher() i=3 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 20 add_watcher() i=4 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 20 add_watcher() i=5 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.828+0000 7fd893d26800 20 add_watcher() i=6 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.829+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94168467776480] v33'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.829+0000 7fd893d26800 20 add_watcher() i=7 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.829+0000 7fd893d26800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.829+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e122e80 con 0x55a54e1128c0 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.829+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.829+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94168467774816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e123980 con 0x55a54e1128c0 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.830+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94168467774816] v33'239 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.830+0000 7fd745fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:08.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 5 note: GC not initialized 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd744fe1640 20 reqs_thread_entry: start 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 20 Filter name: none 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd6be7fc640 20 reqs_thread_entry: start 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e159180 con 0x7fd88005a070 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 20 rados->read ofs=0 len=0 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.831+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:63 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e157820 con 0x55a54e1128c0 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.832+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(63 MH1WC2XQ1S8UISFDZC8W [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.832+0000 7fd893d26800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.832+0000 7fd893d26800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.832+0000 7fd893d26800 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.832+0000 7fd893d26800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.832+0000 7fd893d26800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-09-19T11:25:08.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.832+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e1580f0 con 0x7fd88005a070 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(64 ada [delete,create,call,writefull 0~473] v33'3 uv3 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd893d26800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd893d26800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd893d26800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94168467995712 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e15cac0 con 0x55a54e1128c0 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== watch-notify(notify (1) cookie 94168467764256 notify 141733920791 ret 0) ==== 691+0+0 (crc 0 0 0) 0x55a54e158490 con 0x55a54e1128c0 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd76effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920791 cookie 94168467764256 notifier 4831 bl.length()=649 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd76effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:08.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd76effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd76effd640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fd754003ea0 con 0x55a54e1128c0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify cookie 94168467995712 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.833+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.834+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== watch-notify(notify_complete (2) cookie 94168467995712 notify 141733920791 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a54e158c30 con 0x55a54e1128c0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.834+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:67 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e15a5f0 con 0x55a54e1128c0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(67 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v33'22 uv22 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd893d26800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd893d26800 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd893d26800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94168467995600 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a54e15a5f0 con 0x55a54e1172b0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== watch-notify(notify (1) cookie 94168467763136 notify 141733920776 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55a54e1077f0 con 0x55a54e1172b0 2024-09-19T11:25:08.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd76e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920776 cookie 94168467763136 notifier 4831 bl.length()=177 2024-09-19T11:25:08.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd76e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-09-19T11:25:08.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd76e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-09-19T11:25:08.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(68 notify.2 [notify cookie 94168467995600 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd76e7fc640 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fd6c4003400 con 0x55a54e1172b0 2024-09-19T11:25:08.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.835+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== watch-notify(notify_complete (2) cookie 94168467995600 notify 141733920776 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a54e107560 con 0x55a54e1172b0 2024-09-19T11:25:08.848 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:08.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.836+0000 7fd745fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:08.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.836+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94168467774816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x55a54e1128c0 2024-09-19T11:25:08.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.837+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94168467774816] v33'240 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.837+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94168467755424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x7fd888021020 2024-09-19T11:25:08.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.838+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94168467755424] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.839+0000 7fd893d26800 20 remove_watcher() i=0 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.839+0000 7fd893d26800 2 removed watcher, disabling cache 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.839+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94168467761600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x55a54e1128c0 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.840+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94168467761600] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.840+0000 7fd893d26800 20 remove_watcher() i=1 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.840+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94168467763136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x55a54e1172b0 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.841+0000 7fd8901da640 1 -- 172.21.15.28:0/1054092954 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94168467763136] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd8880083e0 con 0x55a54e1172b0 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.841+0000 7fd893d26800 20 remove_watcher() i=2 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.841+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94168467767152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x7fd888021020 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.842+0000 7fd88f9d9640 1 -- 172.21.15.28:0/1054092954 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94168467767152] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd880057030 con 0x7fd888021020 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.842+0000 7fd893d26800 20 remove_watcher() i=3 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.842+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94168467764256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x55a54e1128c0 2024-09-19T11:25:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.843+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94168467764256] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x55a54e1128c0 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.843+0000 7fd893d26800 20 remove_watcher() i=4 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.843+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94168467772592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x7fd88005a070 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.844+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94168467772592] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.844+0000 7fd893d26800 20 remove_watcher() i=5 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.844+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94168467769904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x7fd88005a070 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.845+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94168467769904] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.845+0000 7fd893d26800 20 remove_watcher() i=6 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.845+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94168467776480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55a54e156f50 con 0x7fd88005a070 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94168467776480] v33'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd878029040 con 0x7fd88005a070 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 20 remove_watcher() i=7 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd88005a070 msgr2=0x7fd880097860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fd88005a070 0x7fd880097860 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54e1128c0 msgr2=0x55a54e114d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54e1128c0 0x55a54e114d80 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd888021020 msgr2=0x7fd88800a500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fd888021020 0x7fd88800a500 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e1172b0 msgr2=0x55a54e1196a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e1172b0 0x55a54e1196a0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd75803d000 msgr2=0x7fd75803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd75803d000 0x7fd75803f4c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd880046030 tx=0x7fd880091000 comp rx=0 tx=0).stop 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 msgr2=0x55a54e107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd88f1d8640 1 -- 172.21.15.28:0/1054092954 reap_dead start 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.846+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e107000 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd888016010 tx=0x7fd888053ce0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 shutdown_connections 2024-09-19T11:25:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/1054092954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0ff9a0 0x55a54e107000 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 >> 172.21.15.28:0/1054092954 conn(0x55a54e0fbda0 msgr2=0x55a54e0f9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 shutdown_connections 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/1054092954 wait complete. 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd76803d000 msgr2=0x7fd76803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd76803d000 0x7fd76803f4c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55a54df536f0 tx=0x7fd880096130 comp rx=0 tx=0).stop 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0f2ee0 msgr2=0x55a54e0f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0f2ee0 0x55a54e0f3300 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55a54df53b70 tx=0x7fd88800bf30 comp rx=0 tx=0).stop 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 shutdown_connections 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd76803d000 0x7fd76803f4c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/2558454836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54e0f2ee0 0x55a54e0f3300 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 >> 172.21.15.28:0/2558454836 conn(0x55a54e0e7290 msgr2=0x55a54e0e7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 shutdown_connections 2024-09-19T11:25:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/2558454836 wait complete. 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54dfdf9e0 msgr2=0x55a54e0df130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54dfdf9e0 0x55a54e0df130 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e0e1a70 msgr2=0x55a54e0e3f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e0e1a70 0x55a54e0e3f00 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd77803cfb0 msgr2=0x7fd77803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd77803cfb0 0x7fd77803f470 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd878001db0 tx=0x7fd878001310 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 msgr2=0x55a54df54140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.847+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54df54140 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd880005fd0 tx=0x7fd880003330 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 shutdown_connections 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55a54e0e1a70 0x55a54e0e3f00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fd77803cfb0 0x7fd77803f470 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55a54dfdf9e0 0x55a54e0df130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 --2- 172.21.15.28:0/660341346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a54dfde8e0 0x55a54df54140 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 >> 172.21.15.28:0/660341346 conn(0x55a54dfd9cb0 msgr2=0x55a54e0cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:08.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 shutdown_connections 2024-09-19T11:25:08.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:08.848+0000 7fd893d26800 1 -- 172.21.15.28:0/660341346 wait complete. 2024-09-19T11:25:08.859 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:08.860 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:08.860 ERROR:root:0 2024-09-19T11:25:08.861 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:08.874 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx000008aab84d2da8eb7c6-0066ec0a14-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:25:08.875 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2024-09-19T11:25:08.894 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:08.894 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2024-09-19T11:25:08.915 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2024-09-19T11:25:08.915 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2024-09-19T11:25:08.939 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-09-19T11:25:08.939 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-09-19T11:25:08.957 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2024-09-19T11:25:08.957 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:08.969 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:08.970 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2024-09-19T11:25:08.982 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:08.982 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:08.982 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:08.983 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write' 2024-09-19T11:25:09.033 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:09.033 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:09.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.032+0000 7f7663414800 1 Processor -- start 2024-09-19T11:25:09.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.032+0000 7f7663414800 1 -- start start 2024-09-19T11:25:09.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.032+0000 7f7663414800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d9d42a5b70 0x55d9d42ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.032+0000 7f7663414800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d9d42a4f50 con 0x55d9d42a48a0 2024-09-19T11:25:09.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.032+0000 7f7663414800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55d9d412f7d0 con 0x55d9d42a5b70 2024-09-19T11:25:09.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.033+0000 7f765e2ca640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55d9d42a5b70 0x55d9d42ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.033+0000 7f765eacb640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55d9d42a48a0 0x55d9d42a5460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48284/0 (socket says 172.21.15.28:48284) 2024-09-19T11:25:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.033+0000 7f765eacb640 1 -- 172.21.15.28:0/2079022330 learned_addr learned my addr 172.21.15.28:0/2079022330 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.033+0000 7f765e2ca640 1 -- 172.21.15.28:0/2079022330 >> v1:172.21.15.22:6789/0 conn(0x55d9d42a48a0 legacy=0x55d9d42a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.033+0000 7f765e2ca640 1 -- 172.21.15.28:0/2079022330 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d42a6020 con 0x55d9d42a5b70 2024-09-19T11:25:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f765e2ca640 1 --2- 172.21.15.28:0/2079022330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a5b70 0x55d9d42ab710 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f765400a470 tx=0x7f7654045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6427bc754950c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f765dac9640 1 -- 172.21.15.28:0/2079022330 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7654053570 con 0x55d9d42a5b70 2024-09-19T11:25:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f765dac9640 1 -- 172.21.15.28:0/2079022330 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7654058070 con 0x55d9d42a5b70 2024-09-19T11:25:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f765dac9640 1 -- 172.21.15.28:0/2079022330 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7654053880 con 0x55d9d42a5b70 2024-09-19T11:25:09.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f7663414800 1 -- 172.21.15.28:0/2079022330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a5b70 msgr2=0x55d9d42ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f7663414800 1 --2- 172.21.15.28:0/2079022330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a5b70 0x55d9d42ab710 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f765400a470 tx=0x7f7654045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:09.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f7663414800 1 -- 172.21.15.28:0/2079022330 shutdown_connections 2024-09-19T11:25:09.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f7663414800 1 --2- 172.21.15.28:0/2079022330 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a5b70 0x55d9d42ab710 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.035+0000 7f7663414800 1 -- 172.21.15.28:0/2079022330 >> 172.21.15.28:0/2079022330 conn(0x55d9d429fc70 msgr2=0x55d9d42a20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.036+0000 7f7663414800 1 -- 172.21.15.28:0/2079022330 shutdown_connections 2024-09-19T11:25:09.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.036+0000 7f7663414800 1 -- 172.21.15.28:0/2079022330 wait complete. 2024-09-19T11:25:09.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.050+0000 7f7663414800 1 Processor -- start 2024-09-19T11:25:09.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.050+0000 7f7663414800 1 -- start start 2024-09-19T11:25:09.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f7663414800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d421a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f7663414800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9d413cdd0 con 0x55d9d42a48a0 2024-09-19T11:25:09.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f765eacb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d421a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f765eacb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d421a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44886/0 (socket says 172.21.15.28:44886) 2024-09-19T11:25:09.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f765eacb640 1 -- 172.21.15.28:0/3031971205 learned_addr learned my addr 172.21.15.28:0/3031971205 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f765eacb640 1 -- 172.21.15.28:0/3031971205 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d421a780 con 0x55d9d42a48a0 2024-09-19T11:25:09.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f765eacb640 1 --2- 172.21.15.28:0/3031971205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d421a190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7650001e40 tx=0x7f7650001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db95ac8f10948010 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f754ce09640 1 -- 172.21.15.28:0/3031971205 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7650012020 con 0x55d9d42a48a0 2024-09-19T11:25:09.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f754ce09640 1 -- 172.21.15.28:0/3031971205 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765000e5a0 con 0x55d9d42a48a0 2024-09-19T11:25:09.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.051+0000 7f754ce09640 1 -- 172.21.15.28:0/3031971205 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f765000e8a0 con 0x55d9d42a48a0 2024-09-19T11:25:09.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.052+0000 7f7663414800 1 -- 172.21.15.28:0/3031971205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 msgr2=0x55d9d421a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.052+0000 7f7663414800 1 --2- 172.21.15.28:0/3031971205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d421a190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7650001e40 tx=0x7f7650001e70 comp rx=0 tx=0).stop 2024-09-19T11:25:09.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.052+0000 7f7663414800 1 -- 172.21.15.28:0/3031971205 shutdown_connections 2024-09-19T11:25:09.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.052+0000 7f7663414800 1 --2- 172.21.15.28:0/3031971205 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d421a190 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.052+0000 7f7663414800 1 -- 172.21.15.28:0/3031971205 >> 172.21.15.28:0/3031971205 conn(0x55d9d429fc70 msgr2=0x55d9d438fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.052+0000 7f7663414800 1 -- 172.21.15.28:0/3031971205 shutdown_connections 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.052+0000 7f7663414800 1 -- 172.21.15.28:0/3031971205 wait complete. 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f7663414800 1 Processor -- start 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f7663414800 1 -- start start 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f7663414800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d439e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f7663414800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9d42abe60 con 0x55d9d42a48a0 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f765eacb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d439e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f765eacb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d439e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44888/0 (socket says 172.21.15.28:44888) 2024-09-19T11:25:09.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f765eacb640 1 -- 172.21.15.28:0/4189420373 learned_addr learned my addr 172.21.15.28:0/4189420373 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f765eacb640 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d439e5c0 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.053+0000 7f765eacb640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d439e060 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f76500012b0 tx=0x7f7650000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.054+0000 7f754b606640 1 -- 172.21.15.28:0/4189420373 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7650012030 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.054+0000 7f754b606640 1 -- 172.21.15.28:0/4189420373 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765001a750 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.054+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d9d439e850 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.054+0000 7f754b606640 1 -- 172.21.15.28:0/4189420373 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f765001aa70 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.054+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d439f250 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f754b606640 1 -- 172.21.15.28:0/4189420373 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7650015020 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f754b606640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f754403cfb0 0x7f754403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f754b606640 1 -- 172.21.15.28:0/4189420373 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f7650020030 con 0x55d9d42a48a0 2024-09-19T11:25:09.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f765e2ca640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f754403cfb0 0x7f754403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d42a5b70 0x55d9d43a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43a5bf0 con 0x55d9d42a5b70 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f765f2cc640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d42a5b70 0x55d9d43a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f765e2ca640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f754403cfb0 0x7f754403f470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7654057000 tx=0x7f7654052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.055+0000 7f765f2cc640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d42a5b70 0x55d9d43a56b0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.056+0000 7f765f2cc640 1 -- 172.21.15.28:0/4189420373 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f765800a4d0 con 0x55d9d42a5b70 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.056+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43a5f90 con 0x55d9d42a5b70 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.056+0000 7f765f2cc640 1 -- 172.21.15.28:0/4189420373 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f765800a4d0 con 0x55d9d42a5b70 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.056+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43a7a50 0x55d9d43a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.056+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43aa680 con 0x55d9d43a7a50 2024-09-19T11:25:09.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.056+0000 7f765eacb640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43a7a50 0x55d9d43a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.056+0000 7f765eacb640 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43a7a50 0x55d9d43a9ef0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.057+0000 7f765eacb640 1 -- 172.21.15.28:0/4189420373 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7650018ce0 con 0x55d9d43a7a50 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.057+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43a67e0 con 0x55d9d42a5b70 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.057+0000 7f765f2cc640 1 -- 172.21.15.28:0/4189420373 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f765800a4d0 con 0x55d9d42a5b70 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.057+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43a6b80 con 0x55d9d43a7a50 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.057+0000 7f765eacb640 1 -- 172.21.15.28:0/4189420373 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7650018ce0 con 0x55d9d43a7a50 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f7663414800 1 Processor -- start 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f7663414800 1 -- start start 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f7663414800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43b8fe0 0x55d9d43b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f7663414800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9d419b4a0 con 0x55d9d43b8fe0 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f765f2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43b8fe0 0x55d9d43b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f765f2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43b8fe0 0x55d9d43b9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44898/0 (socket says 172.21.15.28:44898) 2024-09-19T11:25:09.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f765f2cc640 1 -- 172.21.15.28:0/1039994808 learned_addr learned my addr 172.21.15.28:0/1039994808 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.058+0000 7f765f2cc640 1 -- 172.21.15.28:0/1039994808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d43b9a40 con 0x55d9d43b8fe0 2024-09-19T11:25:09.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.059+0000 7f765f2cc640 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43b8fe0 0x55d9d43b9400 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x55d9d439eef0 tx=0x7f76580167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.059+0000 7f7548e01640 1 -- 172.21.15.28:0/1039994808 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7658013640 con 0x55d9d43b8fe0 2024-09-19T11:25:09.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.059+0000 7f7548e01640 1 -- 172.21.15.28:0/1039994808 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76580137e0 con 0x55d9d43b8fe0 2024-09-19T11:25:09.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.059+0000 7f7548e01640 1 -- 172.21.15.28:0/1039994808 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7658013b00 con 0x55d9d43b8fe0 2024-09-19T11:25:09.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.059+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d9d43ba270 con 0x55d9d43b8fe0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.059+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d43ba810 con 0x55d9d43b8fe0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.060+0000 7f7548e01640 1 -- 172.21.15.28:0/1039994808 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7658011040 con 0x55d9d43b8fe0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.060+0000 7f7548e01640 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f753803d000 0x7f753803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.060+0000 7f7548e01640 1 -- 172.21.15.28:0/1039994808 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f765800e040 con 0x55d9d43b8fe0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.060+0000 7f765eacb640 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f753803d000 0x7f753803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.060+0000 7f7663414800 1 Processor -- start 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.060+0000 7f765eacb640 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f753803d000 0x7f753803f4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55d9d43a0160 tx=0x7f7650052a40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.060+0000 7f7663414800 1 -- start start 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f7663414800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f7663414800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9d4218be0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f765f2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f765f2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44914/0 (socket says 172.21.15.28:44914) 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f765f2cc640 1 -- 172.21.15.28:0/2839871092 learned_addr learned my addr 172.21.15.28:0/2839871092 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f765f2cc640 1 -- 172.21.15.28:0/2839871092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d43c64b0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f765f2cc640 1 --2- 172.21.15.28:0/2839871092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43c5ec0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7658054950 tx=0x7f7658020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecc174a76eb94ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f75427fc640 1 -- 172.21.15.28:0/2839871092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7658012d50 con 0x55d9d43c5ac0 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f75427fc640 1 -- 172.21.15.28:0/2839871092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7658028350 con 0x55d9d43c5ac0 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.061+0000 7f75427fc640 1 -- 172.21.15.28:0/2839871092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7658020bd0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 -- 172.21.15.28:0/2839871092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 msgr2=0x55d9d43c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 --2- 172.21.15.28:0/2839871092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43c5ec0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7658054950 tx=0x7f7658020460 comp rx=0 tx=0).stop 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 -- 172.21.15.28:0/2839871092 shutdown_connections 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 --2- 172.21.15.28:0/2839871092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43c5ec0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 -- 172.21.15.28:0/2839871092 >> 172.21.15.28:0/2839871092 conn(0x55d9d43c1ec0 msgr2=0x55d9d43c0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 -- 172.21.15.28:0/2839871092 shutdown_connections 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 -- 172.21.15.28:0/2839871092 wait complete. 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 Processor -- start 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 -- start start 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.062+0000 7f7663414800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d9d43c6ad0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f765f2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f765f2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44922/0 (socket says 172.21.15.28:44922) 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 learned_addr learned my addr 172.21.15.28:0/2949215944 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d43cd5d0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f765f2cc640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43cd070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7658014f20 tx=0x7f76580548e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f7540ff9640 1 -- 172.21.15.28:0/2949215944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7658020ed0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f7540ff9640 1 -- 172.21.15.28:0/2949215944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765801fca0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f7540ff9640 1 -- 172.21.15.28:0/2949215944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f765801f650 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d9d43cd860 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.063+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d43ce1e0 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.064+0000 7f7540ff9640 1 -- 172.21.15.28:0/2949215944 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7658056070 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.064+0000 7f7540ff9640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f752803d000 0x7f752803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.064+0000 7f7540ff9640 1 -- 172.21.15.28:0/2949215944 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f765803d070 con 0x55d9d43c5ac0 2024-09-19T11:25:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.064+0000 7f765eacb640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f752803d000 0x7f752803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.065+0000 7f765eacb640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f752803d000 0x7f752803f4c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f765003f010 tx=0x7f7650046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.065+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.065+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d43d8a40 0x55d9d43daf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.065+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43db650 con 0x55d9d43d8a40 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.065+0000 7f765e2ca640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d43d8a40 0x55d9d43daf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.065+0000 7f765e2ca640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d43d8a40 0x55d9d43daf00 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 realm 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43dba80 con 0x55d9d43d8a40 2024-09-19T11:25:09.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43dc350 con 0x55d9d43d8a40 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43dc6f0 con 0x55d9d43d8a40 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43dd450 0x55d9d43df840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.066+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43dfe50 con 0x55d9d43dd450 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f765f2cc640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43dd450 0x55d9d43df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f765f2cc640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43dd450 0x55d9d43df840 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f7663414800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f7663414800 20 searching for the correct realm 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f765f2cc640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f76580508f0 0x7f7658004600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.067+0000 7f765eacb640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f76580508f0 0x7f7658004600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.068+0000 7f765eacb640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f76580508f0 0x7f7658004600 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.068+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.068+0000 7f765eacb640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7650017050 0x7f765000d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.068+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f7650017050 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.068+0000 7f765e2ca640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7650017050 0x7f765000d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7650017050 0x7f765000d3d0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f7650017050 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.069+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f7650017050 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.070+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.070+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.070+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.070+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.070+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.070+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.070+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.071+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f7650017050 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.071+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.071+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.071+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.071+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.071+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.071+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.072+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.073+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.073+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.073+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.073+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.073+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.073+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f7650017050 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.074+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f7650017050 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f7650017050 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.075+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43dd450 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x55d9d43d8a40 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1780 con 0x7f76580508f0 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.076+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e0840 con 0x55d9d43d8a40 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1090 con 0x55d9d43d8a40 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e1c00 con 0x55d9d43dd450 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 zone default found 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 4 Realm: () 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 zonegroup default 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.077+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e2010 con 0x7f7650017050 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e3e70 con 0x55d9d43dd450 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e4300 con 0x55d9d43dd450 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 20 started sync module instance, tier type = 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.078+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e32e0 con 0x7f76580508f0 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.079+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.079+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94394057109040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e4760 con 0x7f76580508f0 2024-09-19T11:25:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.079+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e4fb0 con 0x55d9d43d8a40 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.080+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94394057109040] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.081+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.081+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94394057116416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e53d0 con 0x55d9d43d8a40 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.081+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e5c20 con 0x55d9d43dd450 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.082+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94394057116416] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.082+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.082+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94394057117952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e5b50 con 0x55d9d43dd450 2024-09-19T11:25:09.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.082+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e66c0 con 0x7f76580508f0 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.083+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.083+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94394057117952] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.083+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94394057122224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e6f60 con 0x7f76580508f0 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.083+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e5620 con 0x55d9d43d8a40 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.084+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94394057122224] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.084+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.084+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94394057124528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e6ed0 con 0x55d9d43d8a40 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.084+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e7ff0 con 0x7f7650017050 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.085+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.085+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94394057128768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e8410 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.085+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e8ed0 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.086+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94394057124528] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.087+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94394057128768] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.087+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.087+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94394057132544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e7f60 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.087+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e9380 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94394057132544] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94394057127200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e9380 con 0x7f7650017050 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 20 add_watcher() i=0 2024-09-19T11:25:09.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 20 add_watcher() i=1 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 20 add_watcher() i=2 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 20 add_watcher() i=3 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 20 add_watcher() i=4 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 20 add_watcher() i=5 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.088+0000 7f7663414800 20 add_watcher() i=6 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.089+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94394057127200] v33'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.089+0000 7f7663414800 20 add_watcher() i=7 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.089+0000 7f7663414800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.089+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43ec780 con 0x55d9d43d8a40 2024-09-19T11:25:09.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.089+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.089+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94394057134240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d43e9cd0 con 0x55d9d43d8a40 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.090+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94394057134240] v33'241 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.090+0000 7f75157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.090+0000 7f7663414800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.090+0000 7f7663414800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.090+0000 7f7663414800 5 note: GC not initialized 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.091+0000 7f748ffff640 20 reqs_thread_entry: start 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.091+0000 7f7663414800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.091+0000 7f7663414800 20 Filter name: none 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.091+0000 7f748dffb640 20 reqs_thread_entry: start 2024-09-19T11:25:09.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.091+0000 7f7663414800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.091+0000 7f7663414800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.091+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441f5c0 con 0x7f7650017050 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.092+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.092+0000 7f7663414800 20 rados_obj.operate() r=0 bl.length=473 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.092+0000 7f7663414800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.092+0000 7f7663414800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.092+0000 7f7663414800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.092+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d4425d60 con 0x7f7650017050 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.093+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v33'5 uv5 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.093+0000 7f7663414800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.093+0000 7f7663414800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:09.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.093+0000 7f7663414800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.093+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94394057381728 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d9d44213e0 con 0x55d9d43d8a40 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.093+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 94394057124528 notify 141733920793 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7f7654082040 con 0x55d9d43d8a40 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.094+0000 7f75427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920793 cookie 94394057124528 notifier 4838 bl.length()=636 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.094+0000 7f75427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.094+0000 7f75427fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.094+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 94394057381728 out=8b] v0'0 uv243 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.094+0000 7f75427fc640 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f7524003ea0 con 0x55d9d43d8a40 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.094+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.094+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 94394057381728 notify 141733920793 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d9d42a6020 con 0x55d9d43d8a40 2024-09-19T11:25:09.112 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:09.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.095+0000 7f75157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.095+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94394057134240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x55d9d43d8a40 2024-09-19T11:25:09.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.096+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94394057134240] v33'242 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.096+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94394057109040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x7f76580508f0 2024-09-19T11:25:09.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.097+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94394057109040] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.097+0000 7f7663414800 20 remove_watcher() i=0 2024-09-19T11:25:09.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.097+0000 7f7663414800 2 removed watcher, disabling cache 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.097+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94394057116416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x55d9d43d8a40 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.098+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94394057116416] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.098+0000 7f7663414800 20 remove_watcher() i=1 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.098+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94394057117952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x55d9d43dd450 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.100+0000 7f765f2cc640 1 -- 172.21.15.28:0/2949215944 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94394057117952] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7658047020 con 0x55d9d43dd450 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.100+0000 7f7663414800 20 remove_watcher() i=2 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.100+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94394057122224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x7f76580508f0 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.101+0000 7f765eacb640 1 -- 172.21.15.28:0/2949215944 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94394057122224] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7650018ce0 con 0x7f76580508f0 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.101+0000 7f7663414800 20 remove_watcher() i=3 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.101+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94394057124528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x55d9d43d8a40 2024-09-19T11:25:09.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.102+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94394057124528] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x55d9d43d8a40 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.102+0000 7f7663414800 20 remove_watcher() i=4 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.102+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94394057128768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x7f7650017050 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.103+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94394057128768] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.103+0000 7f7663414800 20 remove_watcher() i=5 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.103+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94394057132544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x7f7650017050 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.104+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94394057132544] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.104+0000 7f7663414800 20 remove_watcher() i=6 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.104+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94394057127200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55d9d441d430 con 0x7f7650017050 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94394057127200] v33'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7654051070 con 0x7f7650017050 2024-09-19T11:25:09.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 20 remove_watcher() i=7 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7650017050 msgr2=0x7f765000d3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f7650017050 0x7f765000d3d0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d43d8a40 msgr2=0x55d9d43daf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d43d8a40 0x55d9d43daf00 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f76580508f0 msgr2=0x7f7658004600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f76580508f0 0x7f7658004600 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43dd450 msgr2=0x55d9d43df840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43dd450 0x55d9d43df840 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f752803d000 msgr2=0x7f752803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f752803d000 0x7f752803f4c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f765003f010 tx=0x7f7650046000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f765e2ca640 1 -- 172.21.15.28:0/2949215944 reap_dead start 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 msgr2=0x55d9d43cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43cd070 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7658014f20 tx=0x7f76580548e0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 shutdown_connections 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/2949215944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43c5ac0 0x55d9d43cd070 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 >> 172.21.15.28:0/2949215944 conn(0x55d9d43c1ec0 msgr2=0x55d9d43c00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 shutdown_connections 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/2949215944 wait complete. 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f753803d000 msgr2=0x7f753803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f753803d000 0x7f753803f4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55d9d43a0160 tx=0x7f7650052a40 comp rx=0 tx=0).stop 2024-09-19T11:25:09.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43b8fe0 msgr2=0x55d9d43b9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.105+0000 7f7663414800 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43b8fe0 0x55d9d43b9400 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x55d9d439eef0 tx=0x7f76580167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 shutdown_connections 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f753803d000 0x7f753803f4c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/1039994808 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d43b8fe0 0x55d9d43b9400 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 >> 172.21.15.28:0/1039994808 conn(0x55d9d43ad390 msgr2=0x55d9d43ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 shutdown_connections 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/1039994808 wait complete. 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d42a5b70 msgr2=0x55d9d43a56b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d42a5b70 0x55d9d43a56b0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43a7a50 msgr2=0x55d9d43a9ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43a7a50 0x55d9d43a9ef0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f754403cfb0 msgr2=0x7f754403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f754403cfb0 0x7f754403f470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7654057000 tx=0x7f7654052000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 msgr2=0x55d9d439e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d439e060 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f76500012b0 tx=0x7f7650000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 shutdown_connections 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55d9d43a7a50 0x55d9d43a9ef0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f754403cfb0 0x7f754403f470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55d9d42a5b70 0x55d9d43a56b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 --2- 172.21.15.28:0/4189420373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d9d42a48a0 0x55d9d439e060 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 >> 172.21.15.28:0/4189420373 conn(0x55d9d429fc70 msgr2=0x55d9d438f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 shutdown_connections 2024-09-19T11:25:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.106+0000 7f7663414800 1 -- 172.21.15.28:0/4189420373 wait complete. 2024-09-19T11:25:09.125 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.125 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.125 ERROR:root:0 2024-09-19T11:25:09.125 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-09-19T11:25:09.125 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-09-19T11:25:09.125 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid system_user --display-name 'System User' --access-key system_akey --secret system_skey --caps user-info-without-keys=read --system 2024-09-19T11:25:09.181 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:09.181 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4353d19800 1 Processor -- start 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4353d19800 1 -- start start 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4353d19800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x557cc339cbf0 0x557cc339cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4353d19800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557cc339e300 con 0x557cc339d620 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4353d19800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x557cc32287d0 con 0x557cc339cbf0 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4351803640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x557cc339cbf0 0x557cc339cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4351002640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x557cc339d620 0x557cc339dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48296/0 (socket says 172.21.15.28:48296) 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4351803640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x557cc339cbf0 0x557cc339cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44924/0 (socket says 172.21.15.28:44924) 2024-09-19T11:25:09.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4351002640 1 -- 172.21.15.28:0/787970338 learned_addr learned my addr 172.21.15.28:0/787970338 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:09.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4351803640 1 -- 172.21.15.28:0/787970338 >> v1:172.21.15.22:6789/0 conn(0x557cc339d620 legacy=0x557cc339dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.181+0000 7f4351803640 1 -- 172.21.15.28:0/787970338 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc33a47a0 con 0x557cc339cbf0 2024-09-19T11:25:09.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.183+0000 7f4351803640 1 --2- 172.21.15.28:0/787970338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc339cff0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f434400a600 tx=0x7f4344047cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccdec765387af03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.183+0000 7f4350801640 1 -- 172.21.15.28:0/787970338 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4344056070 con 0x557cc339cbf0 2024-09-19T11:25:09.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.183+0000 7f4350801640 1 -- 172.21.15.28:0/787970338 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434400a200 con 0x557cc339cbf0 2024-09-19T11:25:09.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.183+0000 7f4350801640 1 -- 172.21.15.28:0/787970338 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4344008590 con 0x557cc339cbf0 2024-09-19T11:25:09.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.184+0000 7f4353d19800 1 -- 172.21.15.28:0/787970338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 msgr2=0x557cc339cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.184+0000 7f4353d19800 1 --2- 172.21.15.28:0/787970338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc339cff0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f434400a600 tx=0x7f4344047cb0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.184+0000 7f4353d19800 1 -- 172.21.15.28:0/787970338 shutdown_connections 2024-09-19T11:25:09.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.184+0000 7f4353d19800 1 --2- 172.21.15.28:0/787970338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc339cff0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.184+0000 7f4353d19800 1 -- 172.21.15.28:0/787970338 >> 172.21.15.28:0/787970338 conn(0x557cc3397210 msgr2=0x557cc3399690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.184+0000 7f4353d19800 1 -- 172.21.15.28:0/787970338 shutdown_connections 2024-09-19T11:25:09.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.184+0000 7f4353d19800 1 -- 172.21.15.28:0/787970338 wait complete. 2024-09-19T11:25:09.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.199+0000 7f4353d19800 1 Processor -- start 2024-09-19T11:25:09.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.199+0000 7f4353d19800 1 -- start start 2024-09-19T11:25:09.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.200+0000 7f4353d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc348f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.200+0000 7f4353d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557cc3235dd0 con 0x557cc339cbf0 2024-09-19T11:25:09.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.200+0000 7f4351803640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc348f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.200+0000 7f4351803640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc348f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44932/0 (socket says 172.21.15.28:44932) 2024-09-19T11:25:09.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.200+0000 7f4351803640 1 -- 172.21.15.28:0/1269135569 learned_addr learned my addr 172.21.15.28:0/1269135569 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.200+0000 7f4351803640 1 -- 172.21.15.28:0/1269135569 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc348fc00 con 0x557cc339cbf0 2024-09-19T11:25:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.200+0000 7f4351803640 1 --2- 172.21.15.28:0/1269135569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc348f610 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4344027ae0 tx=0x7f4344053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2490d92030facc0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f423fe07640 1 -- 172.21.15.28:0/1269135569 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4344051660 con 0x557cc339cbf0 2024-09-19T11:25:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f423fe07640 1 -- 172.21.15.28:0/1269135569 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4344051800 con 0x557cc339cbf0 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f423fe07640 1 -- 172.21.15.28:0/1269135569 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4344051ae0 con 0x557cc339cbf0 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f4353d19800 1 -- 172.21.15.28:0/1269135569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 msgr2=0x557cc348f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f4353d19800 1 --2- 172.21.15.28:0/1269135569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc348f610 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4344027ae0 tx=0x7f4344053000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f4353d19800 1 -- 172.21.15.28:0/1269135569 shutdown_connections 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f4353d19800 1 --2- 172.21.15.28:0/1269135569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc348f610 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f4353d19800 1 -- 172.21.15.28:0/1269135569 >> 172.21.15.28:0/1269135569 conn(0x557cc3397210 msgr2=0x557cc3486b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f4353d19800 1 -- 172.21.15.28:0/1269135569 shutdown_connections 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.201+0000 7f4353d19800 1 -- 172.21.15.28:0/1269135569 wait complete. 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4353d19800 1 Processor -- start 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4353d19800 1 -- start start 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4353d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc3497020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4353d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557cc33a3a90 con 0x557cc339cbf0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4351803640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc3497020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4351803640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc3497020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44936/0 (socket says 172.21.15.28:44936) 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4351803640 1 -- 172.21.15.28:0/716862874 learned_addr learned my addr 172.21.15.28:0/716862874 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.202+0000 7f4351803640 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc3497580 con 0x557cc339cbf0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.203+0000 7f4351803640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc3497020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4344027ae0 tx=0x7f4344003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.203+0000 7f423e604640 1 -- 172.21.15.28:0/716862874 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4344073070 con 0x557cc339cbf0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.203+0000 7f423e604640 1 -- 172.21.15.28:0/716862874 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434405b450 con 0x557cc339cbf0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.203+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557cc3497810 con 0x557cc339cbf0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.203+0000 7f423e604640 1 -- 172.21.15.28:0/716862874 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434405b730 con 0x557cc339cbf0 2024-09-19T11:25:09.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.203+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc3498210 con 0x557cc339cbf0 2024-09-19T11:25:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f423e604640 1 -- 172.21.15.28:0/716862874 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f4344076070 con 0x557cc339cbf0 2024-09-19T11:25:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f423e604640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f423803cfb0 0x7f423803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f423e604640 1 -- 172.21.15.28:0/716862874 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f434406b070 con 0x557cc339cbf0 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f4351002640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f423803cfb0 0x7f423803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc339d620 0x557cc349e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc349ebb0 con 0x557cc339d620 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f4352004640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc339d620 0x557cc349e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f4351002640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f423803cfb0 0x7f423803f470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4348001d70 tx=0x7f43480012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.204+0000 7f4352004640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc339d620 0x557cc349e670 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.205+0000 7f4352004640 1 -- 172.21.15.28:0/716862874 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f434c00a4d0 con 0x557cc339d620 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.205+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc349ef50 con 0x557cc339d620 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.205+0000 7f4352004640 1 -- 172.21.15.28:0/716862874 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f434c00a4d0 con 0x557cc339d620 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.205+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34a0a50 0x557cc34a2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.205+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34a3660 con 0x557cc34a0a50 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.205+0000 7f4351803640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34a0a50 0x557cc34a2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.205+0000 7f4351803640 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34a0a50 0x557cc34a2ed0 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.206+0000 7f4351803640 1 -- 172.21.15.28:0/716862874 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4344075030 con 0x557cc34a0a50 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.206+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc349f7a0 con 0x557cc339d620 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.206+0000 7f4352004640 1 -- 172.21.15.28:0/716862874 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f434c00a4d0 con 0x557cc339d620 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.206+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc349fbf0 con 0x557cc34a0a50 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.206+0000 7f4351803640 1 -- 172.21.15.28:0/716862874 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4344075030 con 0x557cc34a0a50 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.207+0000 7f4353d19800 1 Processor -- start 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.207+0000 7f4353d19800 1 -- start start 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.207+0000 7f4353d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34b1f10 0x557cc34b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.207+0000 7f4353d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557cc32944a0 con 0x557cc34b1f10 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.207+0000 7f4352004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34b1f10 0x557cc34b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.207+0000 7f4352004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34b1f10 0x557cc34b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44952/0 (socket says 172.21.15.28:44952) 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.207+0000 7f4352004640 1 -- 172.21.15.28:0/2839777160 learned_addr learned my addr 172.21.15.28:0/2839777160 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.208+0000 7f4352004640 1 -- 172.21.15.28:0/2839777160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc34b2970 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.208+0000 7f4352004640 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34b1f10 0x557cc34b2330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x557cc3498020 tx=0x7f434c0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.208+0000 7f42377fe640 1 -- 172.21.15.28:0/2839777160 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c013640 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.208+0000 7f42377fe640 1 -- 172.21.15.28:0/2839777160 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434c0137e0 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.208+0000 7f42377fe640 1 -- 172.21.15.28:0/2839777160 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c013ac0 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.208+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557cc34b31a0 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.208+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc34b3740 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.209+0000 7f42377fe640 1 -- 172.21.15.28:0/2839777160 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f434c011040 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.209+0000 7f42377fe640 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f422c03d050 0x7f422c03f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.209+0000 7f42377fe640 1 -- 172.21.15.28:0/2839777160 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f434c00e040 con 0x557cc34b1f10 2024-09-19T11:25:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.209+0000 7f4351803640 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f422c03d050 0x7f422c03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.209+0000 7f4353d19800 1 Processor -- start 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4351803640 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f422c03d050 0x7f422c03f510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4344006770 tx=0x7f4344091040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4353d19800 1 -- start start 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4353d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34bed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4353d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557cc348ff70 con 0x557cc34be970 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4352004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4352004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34bed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44960/0 (socket says 172.21.15.28:44960) 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4352004640 1 -- 172.21.15.28:0/435450066 learned_addr learned my addr 172.21.15.28:0/435450066 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4352004640 1 -- 172.21.15.28:0/435450066 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc34bf360 con 0x557cc34be970 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f4352004640 1 --2- 172.21.15.28:0/435450066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34bed70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f434c054890 tx=0x7f434c013e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2670799aaafbaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f42357fa640 1 -- 172.21.15.28:0/435450066 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c051020 con 0x557cc34be970 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f42357fa640 1 -- 172.21.15.28:0/435450066 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434c021a90 con 0x557cc34be970 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.210+0000 7f42357fa640 1 -- 172.21.15.28:0/435450066 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c021570 con 0x557cc34be970 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 -- 172.21.15.28:0/435450066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 msgr2=0x557cc34bed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 --2- 172.21.15.28:0/435450066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34bed70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f434c054890 tx=0x7f434c013e40 comp rx=0 tx=0).stop 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 -- 172.21.15.28:0/435450066 shutdown_connections 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 --2- 172.21.15.28:0/435450066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34bed70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 -- 172.21.15.28:0/435450066 >> 172.21.15.28:0/435450066 conn(0x557cc34bad70 msgr2=0x557cc34b8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 -- 172.21.15.28:0/435450066 shutdown_connections 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 -- 172.21.15.28:0/435450066 wait complete. 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 Processor -- start 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 -- start start 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.211+0000 7f4353d19800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557cc34bf980 con 0x557cc34be970 2024-09-19T11:25:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f4352004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f4352004640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44968/0 (socket says 172.21.15.28:44968) 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 learned_addr learned my addr 172.21.15.28:0/3477485959 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc34c64f0 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f4352004640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34c5f90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f434c0547c0 tx=0x7f434c0547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f42237fe640 1 -- 172.21.15.28:0/3477485959 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c01fa90 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f42237fe640 1 -- 172.21.15.28:0/3477485959 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434c04c070 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557cc34c6780 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f42237fe640 1 -- 172.21.15.28:0/3477485959 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f434c01fd60 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.212+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc34c7100 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.213+0000 7f42237fe640 1 -- 172.21.15.28:0/3477485959 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f434c01f500 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.213+0000 7f42237fe640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f421c03cfb0 0x7f421c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.213+0000 7f42237fe640 1 -- 172.21.15.28:0/3477485959 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f434c03c3e0 con 0x557cc34be970 2024-09-19T11:25:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.213+0000 7f4351803640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f421c03cfb0 0x7f421c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.214+0000 7f4351803640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f421c03cfb0 0x7f421c03f470 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4344096230 tx=0x7f4344083000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.214+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.214+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc34d17c0 0x557cc34d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.214+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34d43d0 con 0x557cc34d17c0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.214+0000 7f4351002640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc34d17c0 0x557cc34d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.214+0000 7f4351002640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc34d17c0 0x557cc34d3c80 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4348009050 con 0x557cc34d17c0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 realm 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34d0b10 con 0x557cc34d17c0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4348009050 con 0x557cc34d17c0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34d4ec0 con 0x557cc34d17c0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4348009050 con 0x557cc34d17c0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34d5790 con 0x557cc34d17c0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4348009050 con 0x557cc34d17c0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34d61b0 0x557cc34d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.215+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34d8ce0 con 0x557cc34d61b0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4352004640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34d61b0 0x557cc34d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4352004640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34d61b0 0x557cc34d85a0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4353d19800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4353d19800 20 searching for the correct realm 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4352004640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f434c0197d0 0x7f434c019000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.216+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.217+0000 7f4351803640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f434c0197d0 0x7f434c019000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.217+0000 7f4351803640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f434c0197d0 0x7f434c019000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.217+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434408ed70 con 0x7f434c0197d0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.217+0000 7f4351803640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4344078060 0x7f4344007400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.217+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f4344078060 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.217+0000 7f4351002640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4344078060 0x7f4344007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.218+0000 7f4351002640 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4344078060 0x7f4344007400 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.218+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348009050 con 0x7f4344078060 2024-09-19T11:25:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.218+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.218+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348009050 con 0x557cc34d17c0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.218+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.218+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.218+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f4344078060 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.219+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.219+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f4344078060 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.219+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.219+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.219+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.219+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.219+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f4344078060 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.220+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.221+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.221+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.221+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.221+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.221+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.221+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.221+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f4344078060 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.222+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.223+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.223+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.223+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.223+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.223+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.223+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.223+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f4344078060 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f4344078060 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.224+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d61b0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x557cc34d17c0 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557cc34da610 con 0x7f434c0197d0 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.225+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34d96a0 con 0x557cc34d17c0 2024-09-19T11:25:09.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34d9ef0 con 0x557cc34d17c0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34daa90 con 0x557cc34d61b0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 zone default found 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 4 Realm: () 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:09.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 zonegroup default 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.226+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34dae50 con 0x7f4344078060 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34dd130 con 0x557cc34d61b0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34dd4d0 con 0x557cc34d61b0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 20 started sync module instance, tier type = 2024-09-19T11:25:09.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:09.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.227+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34dc5c0 con 0x7f434c0197d0 2024-09-19T11:25:09.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.228+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.228+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93994340960736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34dddc0 con 0x7f434c0197d0 2024-09-19T11:25:09.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.228+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34de610 con 0x557cc34d17c0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.229+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93994340960736] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.230+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.230+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93994340964624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34de650 con 0x557cc34d17c0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.230+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34df140 con 0x557cc34d61b0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.231+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93994340964624] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.231+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.231+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93994340969488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34df100 con 0x557cc34d61b0 2024-09-19T11:25:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.231+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34dfbb0 con 0x7f434c0197d0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.232+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.232+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93994340972192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e0480 con 0x7f434c0197d0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.232+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e0f60 con 0x557cc34d17c0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.232+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93994340969488] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.233+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.233+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93994340977264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e0f60 con 0x557cc34d17c0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.233+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e1a10 con 0x7f4344078060 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.233+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93994340972192] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.234+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.234+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93994340974544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e1a10 con 0x7f4344078060 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.234+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93994340977264] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.234+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e2460 con 0x7f4344078060 2024-09-19T11:25:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.235+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 93994340974544] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.235+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.235+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93994340966752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e1f70 con 0x7f4344078060 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.235+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e2d30 con 0x7f4344078060 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 93994340966752] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93994340979952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e1f70 con 0x7f4344078060 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 20 add_watcher() i=0 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 20 add_watcher() i=1 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 20 add_watcher() i=2 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 20 add_watcher() i=3 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 20 add_watcher() i=4 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 20 add_watcher() i=5 2024-09-19T11:25:09.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.236+0000 7f4353d19800 20 add_watcher() i=6 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.237+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93994340979952] v33'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.237+0000 7f4353d19800 20 add_watcher() i=7 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.237+0000 7f4353d19800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.237+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e5d60 con 0x557cc34d17c0 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.238+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.238+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93994340984960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc34e31f0 con 0x557cc34d17c0 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93994340984960] v33'243 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f42037fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4353d19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4353d19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4353d19800 5 note: GC not initialized 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f42027fc640 20 reqs_thread_entry: start 2024-09-19T11:25:09.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4353d19800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4353d19800 20 Filter name: none 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4173fff640 20 reqs_thread_entry: start 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4353d19800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.239+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc3518ce0 con 0x7f4344078060 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 system_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 239+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x0 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 10 cache get: name=default.rgw.meta+users.keys+system_akey : miss 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:63 8.0 8:01223cbf:users.keys::system_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc35173a0 con 0x557cc34d61b0 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(63 system_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x0 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 10 adding default.rgw.meta+users.keys+system_akey to cache LRU end 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.240+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:64 8.1e 8:78ce4db0:users.uid::system_user:head [delete,create,call version.set in=58b,writefull 0~349 in=349b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3517c70 con 0x7f4344078060 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(64 system_user [delete,create,call,writefull 0~349] v33'1 uv1 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4353d19800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4353d19800 10 moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4353d19800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [notify cookie 93994341201856 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc351c620 con 0x7f434c0197d0 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== watch-notify(notify (1) cookie 93994340960736 notify 141733920801 ret 0) ==== 575+0+0 (crc 0 0 0) 0x557cc3498210 con 0x7f434c0197d0 2024-09-19T11:25:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f42357fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920801 cookie 93994340960736 notifier 4845 bl.length()=533 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f42357fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f42357fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(65 notify.0 [notify cookie 93994341201856 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f42357fa640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f4214003ea0 con 0x7f434c0197d0 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(66 notify.0 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== watch-notify(notify_complete (2) cookie 93994341201856 notify 141733920801 ret 0) ==== 42+0+48 (crc 0 0 0) 0x557cc3497810 con 0x7f434c0197d0 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.242+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 8.0 8:01223cbf:users.keys::system_akey:head [delete,create,writefull 0~15 in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc351a150 con 0x557cc34d61b0 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.243+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 system_akey [delete,create,writefull 0~15] v33'1 uv1 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.243+0000 7f4353d19800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.243+0000 7f4353d19800 10 moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-09-19T11:25:09.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.243+0000 7f4353d19800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.243+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify cookie 93994341201856 in=188b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557cc351c620 con 0x557cc34d17c0 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 93994340964624 notify 141733920794 ret 0) ==== 218+0+0 (crc 0 0 0) 0x557cc3518010 con 0x557cc34d17c0 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4234ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920794 cookie 93994340964624 notifier 4845 bl.length()=176 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4234ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4234ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4234ff9640 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f417c003400 con 0x557cc34d17c0 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(68 notify.1 [notify cookie 93994341201856 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== watch-notify(notify_complete (2) cookie 93994341201856 notify 141733920794 ret 0) ==== 42+0+48 (crc 0 0 0) 0x557cc3517500 con 0x557cc34d17c0 2024-09-19T11:25:09.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.244+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(69 notify.1 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.258 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"system_user","display_name":"System User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"system_user","access_key":"system_akey","secret_key":"system_skey","active":true,"create_date":"2024-09-19T11:25:09.241404Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","system":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:09.241393Z","tags":[],"group_ids":[]} 2024-09-19T11:25:09.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.245+0000 7f42037fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.245+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93994340984960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x557cc34d17c0 2024-09-19T11:25:09.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.246+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93994340984960] v33'244 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.246+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93994340960736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x7f434c0197d0 2024-09-19T11:25:09.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.247+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93994340960736] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.247+0000 7f4353d19800 20 remove_watcher() i=0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.247+0000 7f4353d19800 2 removed watcher, disabling cache 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.247+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93994340964624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x557cc34d17c0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.248+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93994340964624] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.248+0000 7f4353d19800 20 remove_watcher() i=1 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.248+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93994340969488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x557cc34d61b0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.249+0000 7f4352004640 1 -- 172.21.15.28:0/3477485959 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93994340969488] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f434c03a070 con 0x557cc34d61b0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.249+0000 7f4353d19800 20 remove_watcher() i=2 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.249+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93994340972192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x7f434c0197d0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.250+0000 7f4351803640 1 -- 172.21.15.28:0/3477485959 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93994340972192] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43440a2040 con 0x7f434c0197d0 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.250+0000 7f4353d19800 20 remove_watcher() i=3 2024-09-19T11:25:09.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.250+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93994340977264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x557cc34d17c0 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.251+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93994340977264] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x557cc34d17c0 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.251+0000 7f4353d19800 20 remove_watcher() i=4 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.251+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93994340974544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x7f4344078060 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.252+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93994340974544] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.252+0000 7f4353d19800 20 remove_watcher() i=5 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.252+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93994340966752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x7f4344078060 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.253+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93994340966752] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.253+0000 7f4353d19800 20 remove_watcher() i=6 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.253+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93994340979952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557cc3516b50 con 0x7f4344078060 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93994340979952] v33'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4348029040 con 0x7f4344078060 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 20 remove_watcher() i=7 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4344078060 msgr2=0x7f4344007400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f4344078060 0x7f4344007400 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc34d17c0 msgr2=0x557cc34d3c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc34d17c0 0x557cc34d3c80 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f434c0197d0 msgr2=0x7f434c019000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f434c0197d0 0x7f434c019000 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34d61b0 msgr2=0x557cc34d85a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34d61b0 0x557cc34d85a0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f421c03cfb0 msgr2=0x7f421c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f421c03cfb0 0x7f421c03f470 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4344096230 tx=0x7f4344083000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 msgr2=0x557cc34c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/3477485959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34be970 0x557cc34c5f90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f434c0547c0 tx=0x7f434c0547f0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4351002640 1 -- 172.21.15.28:0/3477485959 reap_dead start 2024-09-19T11:25:09.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 shutdown_connections 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 >> 172.21.15.28:0/3477485959 conn(0x557cc34bad70 msgr2=0x557cc34b8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 shutdown_connections 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/3477485959 wait complete. 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f422c03d050 msgr2=0x7f422c03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f422c03d050 0x7f422c03f510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4344006770 tx=0x7f4344091040 comp rx=0 tx=0).stop 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34b1f10 msgr2=0x557cc34b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34b1f10 0x557cc34b2330 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x557cc3498020 tx=0x7f434c0167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 shutdown_connections 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f422c03d050 0x7f422c03f510 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 --2- 172.21.15.28:0/2839777160 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc34b1f10 0x557cc34b2330 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 >> 172.21.15.28:0/2839777160 conn(0x557cc34a62c0 msgr2=0x557cc34a66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.255+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 shutdown_connections 2024-09-19T11:25:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/2839777160 wait complete. 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc339d620 msgr2=0x557cc349e670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc339d620 0x557cc349e670 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34a0a50 msgr2=0x557cc34a2ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34a0a50 0x557cc34a2ed0 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f423803cfb0 msgr2=0x7f423803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f423803cfb0 0x7f423803f470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4348001d70 tx=0x7f43480012d0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 msgr2=0x557cc3497020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc3497020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4344027ae0 tx=0x7f4344003020 comp rx=0 tx=0).stop 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 shutdown_connections 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557cc34a0a50 0x557cc34a2ed0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f423803cfb0 0x7f423803f470 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557cc339d620 0x557cc349e670 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 --2- 172.21.15.28:0/716862874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557cc339cbf0 0x557cc3497020 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 >> 172.21.15.28:0/716862874 conn(0x557cc3397210 msgr2=0x557cc3490940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 shutdown_connections 2024-09-19T11:25:09.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.256+0000 7f4353d19800 1 -- 172.21.15.28:0/716862874 wait complete. 2024-09-19T11:25:09.269 DEBUG:tasks.util.rgw: json result: {'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-09-19T11:25:09.241404Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:09.241393Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.269 ERROR:root:{'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-09-19T11:25:09.241404Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:09.241393Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.269 ERROR:root:0 2024-09-19T11:25:09.269 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:09.282 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:09.282 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'system_user'] 2024-09-19T11:25:09.282 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'system_user'] 2024-09-19T11:25:09.282 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid system_user 2024-09-19T11:25:09.328 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:09.328 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:09.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.327+0000 7f5afc620800 1 Processor -- start 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5afc620800 1 -- start start 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5afc620800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5610b5da0f40 0x5610b5da3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5afc620800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5610b5da75d0 con 0x5610b5da38d0 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5afc620800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5610b5c2b7d0 con 0x5610b5da0f40 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5afa10a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5610b5da0f40 0x5610b5da3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5af9909640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5610b5da38d0 0x5610b5da5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48310/0 (socket says 172.21.15.28:48310) 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5af9909640 1 -- 172.21.15.28:0/159093942 learned_addr learned my addr 172.21.15.28:0/159093942 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:09.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5afa10a640 1 -- 172.21.15.28:0/159093942 >> v1:172.21.15.22:6789/0 conn(0x5610b5da38d0 legacy=0x5610b5da5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.328+0000 7f5afa10a640 1 -- 172.21.15.28:0/159093942 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5da8800 con 0x5610b5da0f40 2024-09-19T11:25:09.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.330+0000 7f5afa10a640 1 --2- 172.21.15.28:0/159093942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da3330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00a470 tx=0x7f5aec045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35b1f09f93ce2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.330+0000 7f5af9108640 1 -- 172.21.15.28:0/159093942 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec053570 con 0x5610b5da0f40 2024-09-19T11:25:09.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.330+0000 7f5af9108640 1 -- 172.21.15.28:0/159093942 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec058070 con 0x5610b5da0f40 2024-09-19T11:25:09.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.330+0000 7f5af9108640 1 -- 172.21.15.28:0/159093942 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec053880 con 0x5610b5da0f40 2024-09-19T11:25:09.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.331+0000 7f5afc620800 1 -- 172.21.15.28:0/159093942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 msgr2=0x5610b5da3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.331+0000 7f5afc620800 1 --2- 172.21.15.28:0/159093942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da3330 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00a470 tx=0x7f5aec045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:09.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.331+0000 7f5afc620800 1 -- 172.21.15.28:0/159093942 shutdown_connections 2024-09-19T11:25:09.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.331+0000 7f5afc620800 1 --2- 172.21.15.28:0/159093942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da3330 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.331+0000 7f5afc620800 1 -- 172.21.15.28:0/159093942 >> 172.21.15.28:0/159093942 conn(0x5610b5d9aa60 msgr2=0x5610b5d9cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.331+0000 7f5afc620800 1 -- 172.21.15.28:0/159093942 shutdown_connections 2024-09-19T11:25:09.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.331+0000 7f5afc620800 1 -- 172.21.15.28:0/159093942 wait complete. 2024-09-19T11:25:09.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.346+0000 7f5afc620800 1 Processor -- start 2024-09-19T11:25:09.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.346+0000 7f5afc620800 1 -- start start 2024-09-19T11:25:09.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.346+0000 7f5afc620800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5e90330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.346+0000 7f5afc620800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610b5c38dd0 con 0x5610b5da0f40 2024-09-19T11:25:09.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.346+0000 7f5afa10a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5e90330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.347+0000 7f5afa10a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5e90330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:44994/0 (socket says 172.21.15.28:44994) 2024-09-19T11:25:09.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.347+0000 7f5afa10a640 1 -- 172.21.15.28:0/2179589977 learned_addr learned my addr 172.21.15.28:0/2179589977 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.347+0000 7f5afa10a640 1 -- 172.21.15.28:0/2179589977 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5e90920 con 0x5610b5da0f40 2024-09-19T11:25:09.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.347+0000 7f5afa10a640 1 --2- 172.21.15.28:0/2179589977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5e90330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5aec025c20 tx=0x7f5aec051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19496e1fe29490d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.347+0000 7f59e8608640 1 -- 172.21.15.28:0/2179589977 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec05c580 con 0x5610b5da0f40 2024-09-19T11:25:09.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.347+0000 7f59e8608640 1 -- 172.21.15.28:0/2179589977 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec063030 con 0x5610b5da0f40 2024-09-19T11:25:09.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.347+0000 7f59e8608640 1 -- 172.21.15.28:0/2179589977 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec05c850 con 0x5610b5da0f40 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 -- 172.21.15.28:0/2179589977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 msgr2=0x5610b5e90330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 --2- 172.21.15.28:0/2179589977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5e90330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5aec025c20 tx=0x7f5aec051000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 -- 172.21.15.28:0/2179589977 shutdown_connections 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 --2- 172.21.15.28:0/2179589977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5e90330 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 -- 172.21.15.28:0/2179589977 >> 172.21.15.28:0/2179589977 conn(0x5610b5d9aa60 msgr2=0x5610b5e8bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 -- 172.21.15.28:0/2179589977 shutdown_connections 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 -- 172.21.15.28:0/2179589977 wait complete. 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 Processor -- start 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 -- start start 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.348+0000 7f5afc620800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f5afc620800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610b5da7aa0 con 0x5610b5da0f40 2024-09-19T11:25:09.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f5afa10a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f5afa10a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45010/0 (socket says 172.21.15.28:45010) 2024-09-19T11:25:09.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f5afa10a640 1 -- 172.21.15.28:0/3344349516 learned_addr learned my addr 172.21.15.28:0/3344349516 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f5afa10a640 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5da0bc0 con 0x5610b5da0f40 2024-09-19T11:25:09.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f5afa10a640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da0660 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5aec045f50 tx=0x7f5aec002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f59e6e05640 1 -- 172.21.15.28:0/3344349516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec054ad0 con 0x5610b5da0f40 2024-09-19T11:25:09.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f59e6e05640 1 -- 172.21.15.28:0/3344349516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec054c70 con 0x5610b5da0f40 2024-09-19T11:25:09.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f59e6e05640 1 -- 172.21.15.28:0/3344349516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec068420 con 0x5610b5da0f40 2024-09-19T11:25:09.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.349+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610b5d9ef70 con 0x5610b5da0f40 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.350+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b5d9f530 con 0x5610b5da0f40 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.350+0000 7f59e6e05640 1 -- 172.21.15.28:0/3344349516 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5aec0685c0 con 0x5610b5da0f40 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f59e6e05640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003cfb0 0x7f59e003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f59e6e05640 1 -- 172.21.15.28:0/3344349516 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f5aec070030 con 0x5610b5da0f40 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f5af9909640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003cfb0 0x7f59e003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5da38d0 0x5610b5ea1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ea17d0 con 0x5610b5da38d0 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f5afa90b640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5da38d0 0x5610b5ea1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f5af9909640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003cfb0 0x7f59e003f470 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5af0001d70 tx=0x7f5af00012d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f5afa90b640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5da38d0 0x5610b5ea1290 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.351+0000 7f5afa90b640 1 -- 172.21.15.28:0/3344349516 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5af400a4d0 con 0x5610b5da38d0 2024-09-19T11:25:09.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.352+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ea20b0 con 0x5610b5da38d0 2024-09-19T11:25:09.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.352+0000 7f5afa90b640 1 -- 172.21.15.28:0/3344349516 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5af400a4d0 con 0x5610b5da38d0 2024-09-19T11:25:09.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.352+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ea39c0 0x5610b5ea5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.352+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ea65f0 con 0x5610b5ea39c0 2024-09-19T11:25:09.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.352+0000 7f5afa10a640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ea39c0 0x5610b5ea5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.352+0000 7f5afa10a640 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ea39c0 0x5610b5ea5e60 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.353+0000 7f5afa10a640 1 -- 172.21.15.28:0/3344349516 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5aec051070 con 0x5610b5ea39c0 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.353+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ea2900 con 0x5610b5da38d0 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.353+0000 7f5afa90b640 1 -- 172.21.15.28:0/3344349516 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5af400a4d0 con 0x5610b5da38d0 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.353+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ea2ca0 con 0x5610b5ea39c0 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.353+0000 7f5afa10a640 1 -- 172.21.15.28:0/3344349516 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5aec051070 con 0x5610b5ea39c0 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.353+0000 7f5afc620800 1 Processor -- start 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afc620800 1 -- start start 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afc620800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5eb4f50 0x5610b5eb5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afc620800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610b5c974a0 con 0x5610b5eb4f50 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afa90b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5eb4f50 0x5610b5eb5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afa90b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5eb4f50 0x5610b5eb5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45024/0 (socket says 172.21.15.28:45024) 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afa90b640 1 -- 172.21.15.28:0/2654082773 learned_addr learned my addr 172.21.15.28:0/2654082773 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afa90b640 1 -- 172.21.15.28:0/2654082773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5eb59b0 con 0x5610b5eb4f50 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.354+0000 7f5afa90b640 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5eb4f50 0x5610b5eb5370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5610b5d9f750 tx=0x7f5af40167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.355+0000 7f59dffff640 1 -- 172.21.15.28:0/2654082773 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af4013640 con 0x5610b5eb4f50 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.355+0000 7f59dffff640 1 -- 172.21.15.28:0/2654082773 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af40137e0 con 0x5610b5eb4f50 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.355+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610b5eb61e0 con 0x5610b5eb4f50 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.355+0000 7f59dffff640 1 -- 172.21.15.28:0/2654082773 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af4013b00 con 0x5610b5eb4f50 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.355+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b5eb6780 con 0x5610b5eb4f50 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.355+0000 7f59dffff640 1 -- 172.21.15.28:0/2654082773 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5af4011040 con 0x5610b5eb4f50 2024-09-19T11:25:09.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f59dffff640 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d403d050 0x7f59d403f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f59dffff640 1 -- 172.21.15.28:0/2654082773 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f5af400e040 con 0x5610b5eb4f50 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f5afa10a640 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d403d050 0x7f59d403f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f5afc620800 1 Processor -- start 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f5afa10a640 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d403d050 0x7f59d403f510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5610b5d9fbc0 tx=0x7f5aec074f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f5afc620800 1 -- start start 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f5afc620800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.356+0000 7f5afc620800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610b5e90cf0 con 0x5610b5ec19d0 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f5afa90b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f5afa90b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45034/0 (socket says 172.21.15.28:45034) 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f5afa90b640 1 -- 172.21.15.28:0/1150439172 learned_addr learned my addr 172.21.15.28:0/1150439172 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f5afa90b640 1 -- 172.21.15.28:0/1150439172 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5ec23c0 con 0x5610b5ec19d0 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f5afa90b640 1 --2- 172.21.15.28:0/1150439172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec1dd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5af400afe0 tx=0x7f5af401ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33971a1f3c0c63d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f59ddffb640 1 -- 172.21.15.28:0/1150439172 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af402a910 con 0x5610b5ec19d0 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f59ddffb640 1 -- 172.21.15.28:0/1150439172 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af404de50 con 0x5610b5ec19d0 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.357+0000 7f59ddffb640 1 -- 172.21.15.28:0/1150439172 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af4013ca0 con 0x5610b5ec19d0 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 -- 172.21.15.28:0/1150439172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 msgr2=0x5610b5ec1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 --2- 172.21.15.28:0/1150439172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec1dd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5af400afe0 tx=0x7f5af401ff20 comp rx=0 tx=0).stop 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 -- 172.21.15.28:0/1150439172 shutdown_connections 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 --2- 172.21.15.28:0/1150439172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec1dd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 -- 172.21.15.28:0/1150439172 >> 172.21.15.28:0/1150439172 conn(0x5610b5ebddd0 msgr2=0x5610b5ebc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 -- 172.21.15.28:0/1150439172 shutdown_connections 2024-09-19T11:25:09.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 -- 172.21.15.28:0/1150439172 wait complete. 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 Processor -- start 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 -- start start 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afc620800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5610b5ec29e0 con 0x5610b5ec19d0 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afa90b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afa90b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45040/0 (socket says 172.21.15.28:45040) 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.358+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 learned_addr learned my addr 172.21.15.28:0/1912908469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.359+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b5ec94e0 con 0x5610b5ec19d0 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.359+0000 7f5afa90b640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec8f80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5af4054780 tx=0x7f5af40547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.359+0000 7f59cbfff640 1 -- 172.21.15.28:0/1912908469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af40459f0 con 0x5610b5ec19d0 2024-09-19T11:25:09.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.359+0000 7f59cbfff640 1 -- 172.21.15.28:0/1912908469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af4045b90 con 0x5610b5ec19d0 2024-09-19T11:25:09.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.359+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610b5ec9770 con 0x5610b5ec19d0 2024-09-19T11:25:09.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.359+0000 7f59cbfff640 1 -- 172.21.15.28:0/1912908469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af4054960 con 0x5610b5ec19d0 2024-09-19T11:25:09.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.359+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b5eca300 con 0x5610b5ec19d0 2024-09-19T11:25:09.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.360+0000 7f59cbfff640 1 -- 172.21.15.28:0/1912908469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5af4054b00 con 0x5610b5ec19d0 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.360+0000 7f59cbfff640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59c403d000 0x7f59c403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.360+0000 7f59cbfff640 1 -- 172.21.15.28:0/1912908469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f5af4017450 con 0x5610b5ec19d0 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.360+0000 7f5afa10a640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59c403d000 0x7f59c403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afa10a640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59c403d000 0x7f59c403f4c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009800 tx=0x7f5aec092000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5ed48a0 0x5610b5ed6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ed74b0 con 0x5610b5ed48a0 2024-09-19T11:25:09.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5af9909640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5ed48a0 0x5610b5ed6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5af9909640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5ed48a0 0x5610b5ed6d60 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5af0009050 con 0x5610b5ed48a0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afc620800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afc620800 20 realm 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.361+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ed3bf0 con 0x5610b5ed48a0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5af0009050 con 0x5610b5ed48a0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ed7fa0 con 0x5610b5ed48a0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5af0009050 con 0x5610b5ed48a0 2024-09-19T11:25:09.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ed8870 con 0x5610b5ed48a0 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5af0009050 con 0x5610b5ed48a0 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ed9290 0x5610b5edb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edbdc0 con 0x5610b5ed9290 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afa90b640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ed9290 0x5610b5edb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.362+0000 7f5afa90b640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ed9290 0x5610b5edb680 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afc620800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afc620800 20 searching for the correct realm 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afa90b640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5af404d990 0x7f5af40203a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afa10a640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5af404d990 0x7f5af40203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.363+0000 7f5afa10a640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5af404d990 0x7f5af40203a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.364+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.364+0000 7f5afa10a640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5aec0a6040 0x7f5aec095190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.364+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.364+0000 7f5af9909640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5aec0a6040 0x7f5aec095190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.364+0000 7f5af9909640 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5aec0a6040 0x7f5aec095190 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0009050 con 0x7f5aec0a6040 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0009050 con 0x5610b5ed48a0 2024-09-19T11:25:09.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.365+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.366+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.367+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.368+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.369+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.369+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.369+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.369+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.370+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.371+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed9290 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x5610b5ed48a0 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edd6f0 con 0x7f5af404d990 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afc620800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afc620800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afc620800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afc620800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afc620800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.372+0000 7f5afc620800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edc780 con 0x5610b5ed48a0 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edcfd0 con 0x5610b5ed48a0 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5eddb70 con 0x5610b5ed9290 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 zone default found 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 4 Realm: () 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 zonegroup default 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.373+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5eddf10 con 0x7f5aec0a6040 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee01f0 con 0x5610b5ed9290 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee05c0 con 0x5610b5ed9290 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 20 started sync module instance, tier type = 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.374+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edf630 con 0x7f5af404d990 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.375+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.375+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94629771735424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee0a20 con 0x7f5af404d990 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.375+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee1270 con 0x5610b5ed48a0 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.377+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.377+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94629771742928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee18e0 con 0x5610b5ed48a0 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.377+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee2400 con 0x5610b5ed9290 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.377+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94629771735424] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.378+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94629771742928] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.378+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.378+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94629771747024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee23c0 con 0x5610b5ed9290 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.378+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee2ca0 con 0x7f5af404d990 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.379+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.379+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94629771749360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee34f0 con 0x7f5af404d990 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.379+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94629771747024] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.379+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee3d40 con 0x5610b5ed48a0 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.380+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94629771749360] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.380+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.380+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94629771737632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee3d40 con 0x5610b5ed48a0 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.380+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee45d0 con 0x7f5aec0a6040 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.381+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.381+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94629771754816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee41a0 con 0x7f5aec0a6040 2024-09-19T11:25:09.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.381+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee5580 con 0x7f5aec0a6040 2024-09-19T11:25:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.382+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94629771737632] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.382+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.382+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94629771753616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee5540 con 0x7f5aec0a6040 2024-09-19T11:25:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.383+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee5f80 con 0x7f5aec0a6040 2024-09-19T11:25:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.383+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94629771754816] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.383+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94629771753616] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94629771762256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee5090 con 0x7f5aec0a6040 2024-09-19T11:25:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=0 2024-09-19T11:25:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=1 2024-09-19T11:25:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=2 2024-09-19T11:25:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=3 2024-09-19T11:25:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=4 2024-09-19T11:25:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=5 2024-09-19T11:25:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=6 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94629771762256] v33'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 20 add_watcher() i=7 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.384+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee9080 con 0x5610b5ed48a0 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.385+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.385+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94629771757936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee6500 con 0x5610b5ed48a0 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.386+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94629771757936] v33'245 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.386+0000 7f59b0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.386+0000 7f5afc620800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.386+0000 7f5afc620800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.386+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef17b0 con 0x5610b5ed48a0 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.387+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.387+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef17b0 con 0x7f5aec0a6040 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.388+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.388+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2220 con 0x5610b5ed9290 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.389+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.389+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef25e0 con 0x5610b5ed48a0 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.390+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'246 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.390+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef27c0 con 0x7f5af404d990 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.391+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'77 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.391+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2ba0 con 0x5610b5ed9290 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.392+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'51 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.392+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2e90 con 0x5610b5ed48a0 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.393+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.393+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2e70 con 0x7f5aec0a6040 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.394+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'51 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.394+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2e70 con 0x7f5af404d990 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.396+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'30 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.396+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2e70 con 0x5610b5ed9290 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.397+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.397+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2f00 con 0x7f5af404d990 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.397+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'78 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.398+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2f00 con 0x7f5aec0a6040 2024-09-19T11:25:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.399+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'35 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.399+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2f00 con 0x7f5af404d990 2024-09-19T11:25:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.400+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.400+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2fd0 con 0x7f5aec0a6040 2024-09-19T11:25:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.401+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'58 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.401+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2fd0 con 0x5610b5ed48a0 2024-09-19T11:25:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.402+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'247 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.402+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef2ff0 con 0x5610b5ed9290 2024-09-19T11:25:09.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.403+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'71 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.403+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef30e0 con 0x7f5af404d990 2024-09-19T11:25:09.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.404+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'79 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.404+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef30e0 con 0x5610b5ed9290 2024-09-19T11:25:09.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.405+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.405+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef3120 con 0x5610b5ed48a0 2024-09-19T11:25:09.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.407+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'37 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.407+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef3120 con 0x5610b5ed48a0 2024-09-19T11:25:09.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.408+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'248 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.408+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef3120 con 0x5610b5ed9290 2024-09-19T11:25:09.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.409+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'72 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.409+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef3160 con 0x7f5af404d990 2024-09-19T11:25:09.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.410+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'39 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.410+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef31a0 con 0x5610b5ed9290 2024-09-19T11:25:09.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.410+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.411+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef31e0 con 0x5610b5ed9290 2024-09-19T11:25:09.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.412+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.412+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef3220 con 0x5610b5ed9290 2024-09-19T11:25:09.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.413+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'25 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.413+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef3220 con 0x5610b5ed9290 2024-09-19T11:25:09.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.414+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'52 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.414+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef3260 con 0x7f5aec0a6040 2024-09-19T11:25:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.415+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'33 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.415+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef32a0 con 0x5610b5ed9290 2024-09-19T11:25:09.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.416+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.416+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef32a0 con 0x7f5aec0a6040 2024-09-19T11:25:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.417+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'52 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.417+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef32a0 con 0x5610b5ed9290 2024-09-19T11:25:09.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.418+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'73 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.418+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef32a0 con 0x7f5aec0a6040 2024-09-19T11:25:09.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.419+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'59 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.419+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef32e0 con 0x7f5af404d990 2024-09-19T11:25:09.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f592b7fe640 20 reqs_thread_entry: start 2024-09-19T11:25:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f5afc620800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f5afc620800 20 Filter name: none 2024-09-19T11:25:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f59297fa640 20 reqs_thread_entry: start 2024-09-19T11:25:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f5afc620800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-09-19T11:25:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f5afc620800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.420+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:94 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5f1dd50 con 0x7f5aec0a6040 2024-09-19T11:25:09.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.421+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(94 system_user [call out=48b,read 0~349 out=349b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 239+0+472 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.421+0000 7f5afc620800 20 rados_obj.operate() r=0 bl.length=349 2024-09-19T11:25:09.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.421+0000 7f5afc620800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x13 2024-09-19T11:25:09.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.421+0000 7f5afc620800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-09-19T11:25:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.421+0000 7f5afc620800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.421+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:95 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5f20660 con 0x7f5aec0a6040 2024-09-19T11:25:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.422+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(95 system_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.422+0000 7f5afc620800 10 removing key index: system_akey 2024-09-19T11:25:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.422+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:96 8.0 8:01223cbf:users.keys::system_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5f1e640 con 0x5610b5ed9290 2024-09-19T11:25:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(96 system_akey [delete] v33'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f5afc620800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-09-19T11:25:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [notify cookie 94629771997600 in=173b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5f20a80 con 0x5610b5ed48a0 2024-09-19T11:25:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== watch-notify(notify (1) cookie 94629771742928 notify 141733920795 ret 0) ==== 203+0+0 (crc 0 0 0) 0x5610b5ef2380 con 0x5610b5ed48a0 2024-09-19T11:25:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f59ddffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920795 cookie 94629771742928 notifier 4851 bl.length()=161 2024-09-19T11:25:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(97 notify.1 [notify cookie 94629771997600 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f59ddffb640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f59c0003d60 con 0x5610b5ed48a0 2024-09-19T11:25:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.423+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(98 notify.1 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.424+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== watch-notify(notify_complete (2) cookie 94629771997600 notify 141733920795 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5610b5ec9770 con 0x5610b5ed48a0 2024-09-19T11:25:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.424+0000 7f5afc620800 10 removing email index: 2024-09-19T11:25:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.424+0000 7f5afc620800 10 removing user buckets index 2024-09-19T11:25:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.424+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:99 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5f1e5f0 con 0x7f5aec0a6040 2024-09-19T11:25:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.425+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(99 system_user.buckets [delete] v33'2 uv1 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.425+0000 7f5afc620800 10 removing user index: system_user 2024-09-19T11:25:09.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.425+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:100 8.1e 8:78ce4db0:users.uid::system_user:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5f22790 con 0x7f5aec0a6040 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(100 system_user [call,call,delete] v33'3 uv3 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f5afc620800 10 removing default.rgw.meta+users.uid+system_user from cache 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f5afc620800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94629772010208 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5610b5f22700 con 0x7f5af404d990 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== watch-notify(notify (1) cookie 94629771735424 notify 141733920802 ret 0) ==== 202+0+0 (crc 0 0 0) 0x5610b5ef2020 con 0x7f5af404d990 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f59dd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920802 cookie 94629771735424 notifier 4851 bl.length()=160 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f59dd7fa640 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f59300032c0 con 0x7f5af404d990 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.426+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(101 notify.0 [notify cookie 94629772010208 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.427+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== watch-notify(notify_complete (2) cookie 94629772010208 notify 141733920802 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f5aec0b0060 con 0x7f5af404d990 2024-09-19T11:25:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.427+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(102 notify.0 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.427+0000 7f59b0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.427+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94629771757936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ef0790 con 0x5610b5ed48a0 2024-09-19T11:25:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.428+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94629771757936] v33'249 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.428+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94629771735424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee4970 con 0x7f5af404d990 2024-09-19T11:25:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.429+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94629771735424] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.429+0000 7f5afc620800 20 remove_watcher() i=0 2024-09-19T11:25:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.429+0000 7f5afc620800 2 removed watcher, disabling cache 2024-09-19T11:25:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.429+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94629771742928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5edf180 con 0x5610b5ed48a0 2024-09-19T11:25:09.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.430+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94629771742928] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.430+0000 7f5afc620800 20 remove_watcher() i=1 2024-09-19T11:25:09.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.430+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94629771747024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee0ed0 con 0x5610b5ed9290 2024-09-19T11:25:09.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.431+0000 7f5afa90b640 1 -- 172.21.15.28:0/1912908469 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94629771747024] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af4053020 con 0x5610b5ed9290 2024-09-19T11:25:09.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.431+0000 7f5afc620800 20 remove_watcher() i=2 2024-09-19T11:25:09.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.431+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94629771749360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee1ed0 con 0x7f5af404d990 2024-09-19T11:25:09.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.432+0000 7f5afa10a640 1 -- 172.21.15.28:0/1912908469 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94629771749360] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5aec08a050 con 0x7f5af404d990 2024-09-19T11:25:09.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.432+0000 7f5afc620800 20 remove_watcher() i=3 2024-09-19T11:25:09.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.432+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94629771737632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee1ed0 con 0x5610b5ed48a0 2024-09-19T11:25:09.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.433+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94629771737632] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x5610b5ed48a0 2024-09-19T11:25:09.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.434+0000 7f5afc620800 20 remove_watcher() i=4 2024-09-19T11:25:09.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.434+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94629771754816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee1ed0 con 0x7f5aec0a6040 2024-09-19T11:25:09.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.435+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94629771754816] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.435+0000 7f5afc620800 20 remove_watcher() i=5 2024-09-19T11:25:09.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.435+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94629771753616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee1ed0 con 0x7f5aec0a6040 2024-09-19T11:25:09.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.436+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94629771753616] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.436+0000 7f5afc620800 20 remove_watcher() i=6 2024-09-19T11:25:09.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.436+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94629771762256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5610b5ee1ed0 con 0x7f5aec0a6040 2024-09-19T11:25:09.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.436+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94629771762256] v33'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5af0029040 con 0x7f5aec0a6040 2024-09-19T11:25:09.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 20 remove_watcher() i=7 2024-09-19T11:25:09.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5aec0a6040 msgr2=0x7f5aec095190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5aec0a6040 0x7f5aec095190 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5ed48a0 msgr2=0x5610b5ed6d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5ed48a0 0x5610b5ed6d60 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5af404d990 msgr2=0x7f5af40203a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5af404d990 0x7f5af40203a0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ed9290 msgr2=0x5610b5edb680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ed9290 0x5610b5edb680 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59c403d000 msgr2=0x7f59c403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59c403d000 0x7f59c403f4c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009800 tx=0x7f5aec092000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 msgr2=0x5610b5ec8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/1912908469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5ec19d0 0x5610b5ec8f80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5af4054780 tx=0x7f5af40547b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5af9909640 1 -- 172.21.15.28:0/1912908469 reap_dead start 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 shutdown_connections 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 >> 172.21.15.28:0/1912908469 conn(0x5610b5ebddd0 msgr2=0x5610b5ebbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 shutdown_connections 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/1912908469 wait complete. 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d403d050 msgr2=0x7f59d403f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d403d050 0x7f59d403f510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5610b5d9fbc0 tx=0x7f5aec074f50 comp rx=0 tx=0).stop 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5eb4f50 msgr2=0x5610b5eb5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5eb4f50 0x5610b5eb5370 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x5610b5d9f750 tx=0x7f5af40167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 shutdown_connections 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59d403d050 0x7f59d403f510 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 --2- 172.21.15.28:0/2654082773 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5eb4f50 0x5610b5eb5370 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 >> 172.21.15.28:0/2654082773 conn(0x5610b5ea9300 msgr2=0x5610b5ea96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 shutdown_connections 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.437+0000 7f5afc620800 1 -- 172.21.15.28:0/2654082773 wait complete. 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5da38d0 msgr2=0x5610b5ea1290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5da38d0 0x5610b5ea1290 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ea39c0 msgr2=0x5610b5ea5e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ea39c0 0x5610b5ea5e60 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003cfb0 msgr2=0x7f59e003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003cfb0 0x7f59e003f470 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5af0001d70 tx=0x7f5af00012d0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 msgr2=0x5610b5da0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da0660 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5aec045f50 tx=0x7f5aec002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 shutdown_connections 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5610b5ea39c0 0x5610b5ea5e60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f59e003cfb0 0x7f59e003f470 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5610b5da38d0 0x5610b5ea1290 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 --2- 172.21.15.28:0/3344349516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5610b5da0f40 0x5610b5da0660 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 >> 172.21.15.28:0/3344349516 conn(0x5610b5d9aa60 msgr2=0x5610b5e8a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 shutdown_connections 2024-09-19T11:25:09.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.438+0000 7f5afc620800 1 -- 172.21.15.28:0/3344349516 wait complete. 2024-09-19T11:25:09.449 ERROR:root:None 2024-09-19T11:25:09.449 ERROR:root:0 2024-09-19T11:25:09.449 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-09-19T11:25:09.449 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-09-19T11:25:09.449 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid a_user --display-name 'True Admin User' --access-key true_admin_akey --secret true_admin_skey --caps user-info-without-keys=read --admin 2024-09-19T11:25:09.504 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:09.504 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:09.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.503+0000 7f887ec0d800 1 Processor -- start 2024-09-19T11:25:09.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.503+0000 7f887ec0d800 1 -- start start 2024-09-19T11:25:09.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887ec0d800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5560c7ecebb0 0x5560c7ed4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887ec0d800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5560c7ecdf90 con 0x5560c7ecd8e0 2024-09-19T11:25:09.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887ec0d800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5560c7d587d0 con 0x5560c7ecebb0 2024-09-19T11:25:09.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887bef6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5560c7ecebb0 0x5560c7ed4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887c6f7640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5560c7ecd8e0 0x5560c7ece4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48314/0 (socket says 172.21.15.28:48314) 2024-09-19T11:25:09.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887bef6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5560c7ecebb0 0x5560c7ed4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45052/0 (socket says 172.21.15.28:45052) 2024-09-19T11:25:09.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887c6f7640 1 -- 172.21.15.28:0/2611991705 learned_addr learned my addr 172.21.15.28:0/2611991705 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:09.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887bef6640 1 -- 172.21.15.28:0/2611991705 >> v1:172.21.15.22:6789/0 conn(0x5560c7ecd8e0 legacy=0x5560c7ece4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.504+0000 7f887bef6640 1 -- 172.21.15.28:0/2611991705 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7ecf060 con 0x5560c7ecebb0 2024-09-19T11:25:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.506+0000 7f887bef6640 1 --2- 172.21.15.28:0/2611991705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecebb0 0x5560c7ed4750 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f887000a470 tx=0x7f8870045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc4d9b52c514bba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.506+0000 7f887b6f5640 1 -- 172.21.15.28:0/2611991705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8870053570 con 0x5560c7ecebb0 2024-09-19T11:25:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.506+0000 7f887b6f5640 1 -- 172.21.15.28:0/2611991705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8870058070 con 0x5560c7ecebb0 2024-09-19T11:25:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.506+0000 7f887b6f5640 1 -- 172.21.15.28:0/2611991705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8870053880 con 0x5560c7ecebb0 2024-09-19T11:25:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.507+0000 7f887ec0d800 1 -- 172.21.15.28:0/2611991705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecebb0 msgr2=0x5560c7ed4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.507+0000 7f887ec0d800 1 --2- 172.21.15.28:0/2611991705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecebb0 0x5560c7ed4750 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f887000a470 tx=0x7f8870045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.507+0000 7f887ec0d800 1 -- 172.21.15.28:0/2611991705 shutdown_connections 2024-09-19T11:25:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.507+0000 7f887ec0d800 1 --2- 172.21.15.28:0/2611991705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecebb0 0x5560c7ed4750 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.507+0000 7f887ec0d800 1 -- 172.21.15.28:0/2611991705 >> 172.21.15.28:0/2611991705 conn(0x5560c7ec8cb0 msgr2=0x5560c7ecb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.507+0000 7f887ec0d800 1 -- 172.21.15.28:0/2611991705 shutdown_connections 2024-09-19T11:25:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.507+0000 7f887ec0d800 1 -- 172.21.15.28:0/2611991705 wait complete. 2024-09-19T11:25:09.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.522+0000 7f887ec0d800 1 Processor -- start 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.522+0000 7f887ec0d800 1 -- start start 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.523+0000 7f887ec0d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7e431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.523+0000 7f887ec0d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560c7d65dd0 con 0x5560c7ecd8e0 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.523+0000 7f887c6f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7e431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.523+0000 7f887c6f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7e431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45054/0 (socket says 172.21.15.28:45054) 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.523+0000 7f887c6f7640 1 -- 172.21.15.28:0/1632120475 learned_addr learned my addr 172.21.15.28:0/1632120475 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.523+0000 7f887c6f7640 1 -- 172.21.15.28:0/1632120475 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7e437c0 con 0x5560c7ecd8e0 2024-09-19T11:25:09.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.523+0000 7f887c6f7640 1 --2- 172.21.15.28:0/1632120475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7e431d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f886c001e40 tx=0x7f886c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4269bec750c3db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f876a60c640 1 -- 172.21.15.28:0/1632120475 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c012020 con 0x5560c7ecd8e0 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f876a60c640 1 -- 172.21.15.28:0/1632120475 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886c00e5a0 con 0x5560c7ecd8e0 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f876a60c640 1 -- 172.21.15.28:0/1632120475 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c00e8a0 con 0x5560c7ecd8e0 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f887ec0d800 1 -- 172.21.15.28:0/1632120475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 msgr2=0x5560c7e431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1632120475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7e431d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f886c001e40 tx=0x7f886c001e70 comp rx=0 tx=0).stop 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f887ec0d800 1 -- 172.21.15.28:0/1632120475 shutdown_connections 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1632120475 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7e431d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f887ec0d800 1 -- 172.21.15.28:0/1632120475 >> 172.21.15.28:0/1632120475 conn(0x5560c7ec8cb0 msgr2=0x5560c7fb8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f887ec0d800 1 -- 172.21.15.28:0/1632120475 shutdown_connections 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.524+0000 7f887ec0d800 1 -- 172.21.15.28:0/1632120475 wait complete. 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887ec0d800 1 Processor -- start 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887ec0d800 1 -- start start 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887ec0d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7fc7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887ec0d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560c7ed4ea0 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887c6f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7fc7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887c6f7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7fc7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45058/0 (socket says 172.21.15.28:45058) 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887c6f7640 1 -- 172.21.15.28:0/3854757129 learned_addr learned my addr 172.21.15.28:0/3854757129 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.525+0000 7f887c6f7640 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7fc7670 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.526+0000 7f887c6f7640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7fc7110 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c010 tx=0x7f886c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.526+0000 7f8768e09640 1 -- 172.21.15.28:0/3854757129 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c012030 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.526+0000 7f8768e09640 1 -- 172.21.15.28:0/3854757129 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886c0014a0 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.526+0000 7f8768e09640 1 -- 172.21.15.28:0/3854757129 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f886c01a7a0 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.526+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5560c7fc7900 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.526+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c7fc8300 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f8768e09640 1 -- 172.21.15.28:0/3854757129 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f886c015020 con 0x5560c7ecd8e0 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f8768e09640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f876403d000 0x7f876403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f8768e09640 1 -- 172.21.15.28:0/3854757129 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f886c020030 con 0x5560c7ecd8e0 2024-09-19T11:25:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f887bef6640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f876403d000 0x7f876403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c7ecebb0 0x5560c7fce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c7fc8eb0 con 0x5560c7ecebb0 2024-09-19T11:25:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f887cef8640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c7ecebb0 0x5560c7fce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.527+0000 7f887bef6640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f876403d000 0x7f876403f4c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8870057000 tx=0x7f8870052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.528+0000 7f887cef8640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c7ecebb0 0x5560c7fce760 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.528+0000 7f887cef8640 1 -- 172.21.15.28:0/3854757129 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c7ecebb0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.528+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c7fc8790 con 0x5560c7ecebb0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.528+0000 7f887cef8640 1 -- 172.21.15.28:0/3854757129 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c7ecebb0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.528+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c7fd0bd0 0x5560c7fd2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.528+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c7fd3690 con 0x5560c7fd0bd0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.528+0000 7f887c6f7640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c7fd0bd0 0x5560c7fd2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.529+0000 7f887c6f7640 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c7fd0bd0 0x5560c7fd2fc0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.529+0000 7f887c6f7640 1 -- 172.21.15.28:0/3854757129 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f886c0187b0 con 0x5560c7fd0bd0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.529+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c7fcfa70 con 0x5560c7ecebb0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.529+0000 7f887cef8640 1 -- 172.21.15.28:0/3854757129 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c7ecebb0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.529+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c7fcfed0 con 0x5560c7fd0bd0 2024-09-19T11:25:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887c6f7640 1 -- 172.21.15.28:0/3854757129 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f886c0187b0 con 0x5560c7fd0bd0 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887ec0d800 1 Processor -- start 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887ec0d800 1 -- start start 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887ec0d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7fe2000 0x5560c7fe2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887ec0d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560c7dc44a0 con 0x5560c7fe2000 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887cef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7fe2000 0x5560c7fe2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887cef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7fe2000 0x5560c7fe2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45068/0 (socket says 172.21.15.28:45068) 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.530+0000 7f887cef8640 1 -- 172.21.15.28:0/279146510 learned_addr learned my addr 172.21.15.28:0/279146510 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.531+0000 7f887cef8640 1 -- 172.21.15.28:0/279146510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7fe2a60 con 0x5560c7fe2000 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.531+0000 7f887cef8640 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7fe2000 0x5560c7fe2420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x5560c7fc7fa0 tx=0x7f88740167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.531+0000 7f8761ffb640 1 -- 172.21.15.28:0/279146510 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8874013640 con 0x5560c7fe2000 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.531+0000 7f8761ffb640 1 -- 172.21.15.28:0/279146510 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88740137e0 con 0x5560c7fe2000 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.531+0000 7f8761ffb640 1 -- 172.21.15.28:0/279146510 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8874013ae0 con 0x5560c7fe2000 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.531+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5560c7fe3290 con 0x5560c7fe2000 2024-09-19T11:25:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.531+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c7fe3830 con 0x5560c7fe2000 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.532+0000 7f8761ffb640 1 -- 172.21.15.28:0/279146510 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8874011040 con 0x5560c7fe2000 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.532+0000 7f8761ffb640 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875403d000 0x7f875403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.532+0000 7f8761ffb640 1 -- 172.21.15.28:0/279146510 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f887400e040 con 0x5560c7fe2000 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.532+0000 7f887c6f7640 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875403d000 0x7f875403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.532+0000 7f887ec0d800 1 Processor -- start 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.532+0000 7f887ec0d800 1 -- start start 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.532+0000 7f887c6f7640 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875403d000 0x7f875403f4c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c720 tx=0x7f886c052990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f887ec0d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7feee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f887ec0d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560c7e41c20 con 0x5560c7feea80 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f887cef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7feee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f887cef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7feee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45076/0 (socket says 172.21.15.28:45076) 2024-09-19T11:25:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f887cef8640 1 -- 172.21.15.28:0/2254201900 learned_addr learned my addr 172.21.15.28:0/2254201900 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f887cef8640 1 -- 172.21.15.28:0/2254201900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7fef470 con 0x5560c7feea80 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f887cef8640 1 --2- 172.21.15.28:0/2254201900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7feee80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8874054810 tx=0x7f88740143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e342cc52e05f2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f87537fe640 1 -- 172.21.15.28:0/2254201900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8874051070 con 0x5560c7feea80 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f87537fe640 1 -- 172.21.15.28:0/2254201900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8874053310 con 0x5560c7feea80 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.533+0000 7f87537fe640 1 -- 172.21.15.28:0/2254201900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8874021910 con 0x5560c7feea80 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 -- 172.21.15.28:0/2254201900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 msgr2=0x5560c7feee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 --2- 172.21.15.28:0/2254201900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7feee80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8874054810 tx=0x7f88740143c0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 -- 172.21.15.28:0/2254201900 shutdown_connections 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 --2- 172.21.15.28:0/2254201900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7feee80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 -- 172.21.15.28:0/2254201900 >> 172.21.15.28:0/2254201900 conn(0x5560c7feae80 msgr2=0x5560c7fe90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 -- 172.21.15.28:0/2254201900 shutdown_connections 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 -- 172.21.15.28:0/2254201900 wait complete. 2024-09-19T11:25:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 Processor -- start 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 -- start start 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7ff5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.534+0000 7f887ec0d800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5560c7fefa90 con 0x5560c7feea80 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f887cef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7ff5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f887cef8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7ff5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45084/0 (socket says 172.21.15.28:45084) 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 learned_addr learned my addr 172.21.15.28:0/1353983139 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c7ff6550 con 0x5560c7feea80 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f887cef8640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7ff5ff0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8874022ff0 tx=0x7f8874054770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f8751ffb640 1 -- 172.21.15.28:0/1353983139 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8874013c80 con 0x5560c7feea80 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f8751ffb640 1 -- 172.21.15.28:0/1353983139 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88740459b0 con 0x5560c7feea80 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f8751ffb640 1 -- 172.21.15.28:0/1353983139 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8874045ba0 con 0x5560c7feea80 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5560c7ff67e0 con 0x5560c7feea80 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.535+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c7ff7160 con 0x5560c7feea80 2024-09-19T11:25:09.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.536+0000 7f8751ffb640 1 -- 172.21.15.28:0/1353983139 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8874045130 con 0x5560c7feea80 2024-09-19T11:25:09.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.536+0000 7f8751ffb640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.536+0000 7f8751ffb640 1 -- 172.21.15.28:0/1353983139 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f8874017450 con 0x5560c7feea80 2024-09-19T11:25:09.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.536+0000 7f887c6f7640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.537+0000 7f887c6f7640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f886c018ac0 tx=0x7f886c047000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.537+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.537+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c80018e0 0x5560c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.537+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c80044f0 con 0x5560c80018e0 2024-09-19T11:25:09.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.537+0000 7f887bef6640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c80018e0 0x5560c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887bef6640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c80018e0 0x5560c8003da0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 realm 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c8004920 con 0x5560c80018e0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c80051f0 con 0x5560c80018e0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c8005590 con 0x5560c80018e0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c80062f0 0x5560c80086e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.538+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c8008cf0 con 0x5560c80062f0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887cef8640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c80062f0 0x5560c80086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887cef8640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c80062f0 0x5560c80086e0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887ec0d800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887ec0d800 20 searching for the correct realm 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887cef8640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8874020800 0x7f8874022840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.539+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.540+0000 7f887c6f7640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8874020800 0x7f8874022840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.540+0000 7f887c6f7640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8874020800 0x7f8874022840 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.540+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.540+0000 7f887c6f7640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f886c044c20 0x7f886c04b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.540+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f886c044c20 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.540+0000 7f887bef6640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f886c044c20 0x7f886c04b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.541+0000 7f887bef6640 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f886c044c20 0x7f886c04b860 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.541+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.541+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.541+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.541+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f886c044c20 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f886c044c20 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.542+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f886c044c20 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.543+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.544+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.545+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f886c044c20 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.546+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f886c044c20 2024-09-19T11:25:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.547+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.547+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.547+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.547+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f886c044c20 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.547+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.547+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.547+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80062f0 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x5560c80018e0 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5560c800a620 con 0x7f8874020800 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.548+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c80096b0 con 0x5560c80018e0 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c8009f00 con 0x5560c80018e0 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c800aaa0 con 0x5560c80062f0 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 zone default found 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 4 Realm: () 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 zonegroup default 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.549+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c800ae40 con 0x7f886c044c20 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c800cc50 con 0x5560c80062f0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c800d1f0 con 0x5560c80062f0 2024-09-19T11:25:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 20 started sync module instance, tier type = 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.550+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800c0c0 con 0x7f8874020800 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.551+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.551+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93874160712720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800d650 con 0x7f8874020800 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.551+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800dea0 con 0x5560c80018e0 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.552+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93874160712720] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.553+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.553+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93874160714976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800dea0 con 0x5560c80018e0 2024-09-19T11:25:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.553+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800e9c0 con 0x5560c80062f0 2024-09-19T11:25:09.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.554+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93874160714976] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.554+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.554+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93874160723088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800dea0 con 0x5560c80062f0 2024-09-19T11:25:09.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.554+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800f450 con 0x7f8874020800 2024-09-19T11:25:09.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.555+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93874160723088] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.555+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.555+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93874160720368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800de10 con 0x7f8874020800 2024-09-19T11:25:09.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.555+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8010310 con 0x5560c80018e0 2024-09-19T11:25:09.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.556+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.556+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93874160729696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8010770 con 0x5560c80018e0 2024-09-19T11:25:09.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.556+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8011280 con 0x7f886c044c20 2024-09-19T11:25:09.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.556+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93874160720368] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.557+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.557+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93874160729696] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.557+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93874160733488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8011240 con 0x7f886c044c20 2024-09-19T11:25:09.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.557+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c80102c0 con 0x7f886c044c20 2024-09-19T11:25:09.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.558+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.558+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93874160730816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8011cc0 con 0x7f886c044c20 2024-09-19T11:25:09.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.558+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c80126e0 con 0x7f886c044c20 2024-09-19T11:25:09.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.558+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93874160733488] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93874160730816] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93874160738768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c800f840 con 0x7f886c044c20 2024-09-19T11:25:09.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 20 add_watcher() i=0 2024-09-19T11:25:09.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 20 add_watcher() i=1 2024-09-19T11:25:09.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 20 add_watcher() i=2 2024-09-19T11:25:09.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 20 add_watcher() i=3 2024-09-19T11:25:09.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 20 add_watcher() i=4 2024-09-19T11:25:09.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 20 add_watcher() i=5 2024-09-19T11:25:09.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.559+0000 7f887ec0d800 20 add_watcher() i=6 2024-09-19T11:25:09.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.560+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93874160738768] v33'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.560+0000 7f887ec0d800 20 add_watcher() i=7 2024-09-19T11:25:09.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.560+0000 7f887ec0d800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:09.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.560+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c80156d0 con 0x5560c80018e0 2024-09-19T11:25:09.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.560+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.560+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93874160736272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8012b50 con 0x5560c80018e0 2024-09-19T11:25:09.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.561+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93874160736272] v33'250 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.561+0000 7f8732fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:09.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:09.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 5 note: GC not initialized 2024-09-19T11:25:09.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f8731fdb640 20 reqs_thread_entry: start 2024-09-19T11:25:09.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:09.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 20 Filter name: none 2024-09-19T11:25:09.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f86a37fe640 20 reqs_thread_entry: start 2024-09-19T11:25:09.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-09-19T11:25:09.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:62 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c8048460 con 0x5560c80062f0 2024-09-19T11:25:09.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.562+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(62 a_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 234+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x0 2024-09-19T11:25:09.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-09-19T11:25:09.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : miss 2024-09-19T11:25:09.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:63 8.15 8:afe09604:users.keys::true_admin_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046b00 con 0x5560c80062f0 2024-09-19T11:25:09.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(63 true_admin_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 201+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x0 2024-09-19T11:25:09.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 10 adding default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-09-19T11:25:09.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-09-19T11:25:09.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-09-19T11:25:09.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.563+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 8.14 8:2efb937b:users.uid::a_user:head [delete,create,call version.set in=58b,writefull 0~363 in=363b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c80473a0 con 0x5560c80062f0 2024-09-19T11:25:09.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(64 a_user [delete,create,call,writefull 0~363] v33'5 uv5 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887ec0d800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-09-19T11:25:09.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887ec0d800 10 moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-09-19T11:25:09.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887ec0d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-09-19T11:25:09.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 93874160955120 in=554b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c804b8f0 con 0x5560c80062f0 2024-09-19T11:25:09.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== watch-notify(notify (1) cookie 93874160723088 notify 141733920777 ret 0) ==== 584+0+0 (crc 0 0 0) 0x5560c8046c60 con 0x5560c80062f0 2024-09-19T11:25:09.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f87537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920777 cookie 93874160723088 notifier 4858 bl.length()=542 2024-09-19T11:25:09.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f87537fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-09-19T11:25:09.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f87537fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-09-19T11:25:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f87537fe640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f8748003ea0 con 0x5560c80062f0 2024-09-19T11:25:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(65 notify.2 [notify cookie 93874160955120 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.564+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.565+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== watch-notify(notify_complete (2) cookie 93874160955120 notify 141733920777 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5560c7ff6550 con 0x5560c80062f0 2024-09-19T11:25:09.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.565+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 8.15 8:afe09604:users.keys::true_admin_akey:head [delete,create,writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8047350 con 0x5560c80062f0 2024-09-19T11:25:09.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(67 true_admin_akey [delete,create,writefull 0~10] v33'3 uv3 ondisk = 0) ==== 243+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887ec0d800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-09-19T11:25:09.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887ec0d800 10 moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-09-19T11:25:09.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887ec0d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-09-19T11:25:09.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 93874160955040 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5560c80472c0 con 0x7f886c044c20 2024-09-19T11:25:09.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== watch-notify(notify (1) cookie 93874160733488 notify 141733920812 ret 0) ==== 217+0+0 (crc 0 0 0) 0x7f8870081040 con 0x7f886c044c20 2024-09-19T11:25:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f8752ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920812 cookie 93874160733488 notifier 4858 bl.length()=175 2024-09-19T11:25:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f8752ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-09-19T11:25:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f8752ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-09-19T11:25:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(68 notify.5 [notify cookie 93874160955040 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f8752ffd640 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f86b0003400 con 0x7f886c044c20 2024-09-19T11:25:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.566+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.567+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== watch-notify(notify_complete (2) cookie 93874160955040 notify 141733920812 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5560c7ecf060 con 0x7f886c044c20 2024-09-19T11:25:09.592 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"a_user","display_name":"True Admin User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"a_user","access_key":"true_admin_akey","secret_key":"true_admin_skey","active":true,"create_date":"2024-09-19T11:25:09.564069Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","admin":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:09.564057Z","tags":[],"group_ids":[]} 2024-09-19T11:25:09.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.568+0000 7f8732fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.568+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93874160736272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x5560c80018e0 2024-09-19T11:25:09.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.569+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93874160736272] v33'251 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.569+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93874160712720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x7f8874020800 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.570+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93874160712720] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.570+0000 7f887ec0d800 20 remove_watcher() i=0 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.570+0000 7f887ec0d800 2 removed watcher, disabling cache 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.570+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93874160714976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x5560c80018e0 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.571+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93874160714976] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.571+0000 7f887ec0d800 20 remove_watcher() i=1 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.571+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93874160723088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x5560c80062f0 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.572+0000 7f887cef8640 1 -- 172.21.15.28:0/1353983139 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93874160723088] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f887400a4d0 con 0x5560c80062f0 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.572+0000 7f887ec0d800 20 remove_watcher() i=2 2024-09-19T11:25:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.572+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93874160720368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x7f8874020800 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.573+0000 7f887c6f7640 1 -- 172.21.15.28:0/1353983139 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93874160720368] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886c0187b0 con 0x7f8874020800 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.573+0000 7f887ec0d800 20 remove_watcher() i=3 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.573+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93874160729696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x5560c80018e0 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.574+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93874160729696] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x5560c80018e0 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.574+0000 7f887ec0d800 20 remove_watcher() i=4 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.574+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93874160733488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x7f886c044c20 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.575+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93874160733488] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.575+0000 7f887ec0d800 20 remove_watcher() i=5 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.575+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93874160730816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x7f886c044c20 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.576+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93874160730816] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.576+0000 7f887ec0d800 20 remove_watcher() i=6 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.576+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93874160738768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5560c8046290 con 0x7f886c044c20 2024-09-19T11:25:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93874160738768] v33'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8870051070 con 0x7f886c044c20 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 20 remove_watcher() i=7 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f886c044c20 msgr2=0x7f886c04b860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f886c044c20 0x7f886c04b860 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c80018e0 msgr2=0x5560c8003da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c80018e0 0x5560c8003da0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8874020800 msgr2=0x7f8874022840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8874020800 0x7f8874022840 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c80062f0 msgr2=0x5560c80086e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c80062f0 0x5560c80086e0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 msgr2=0x7f874403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.577+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f886c018ac0 tx=0x7f886c047000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887bef6640 1 -- 172.21.15.28:0/1353983139 reap_dead start 2024-09-19T11:25:09.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 msgr2=0x5560c7ff5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7ff5ff0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8874022ff0 tx=0x7f8874054770 comp rx=0 tx=0).stop 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 shutdown_connections 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/1353983139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7feea80 0x5560c7ff5ff0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 >> 172.21.15.28:0/1353983139 conn(0x5560c7feae80 msgr2=0x5560c7fe9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 shutdown_connections 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/1353983139 wait complete. 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875403d000 msgr2=0x7f875403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875403d000 0x7f875403f4c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c720 tx=0x7f886c052990 comp rx=0 tx=0).stop 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7fe2000 msgr2=0x5560c7fe2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7fe2000 0x5560c7fe2420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x5560c7fc7fa0 tx=0x7f88740167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 shutdown_connections 2024-09-19T11:25:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875403d000 0x7f875403f4c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/279146510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7fe2000 0x5560c7fe2420 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 >> 172.21.15.28:0/279146510 conn(0x5560c7fd63b0 msgr2=0x5560c7fd6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 shutdown_connections 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/279146510 wait complete. 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c7ecebb0 msgr2=0x5560c7fce760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c7ecebb0 0x5560c7fce760 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c7fd0bd0 msgr2=0x5560c7fd2fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.578+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c7fd0bd0 0x5560c7fd2fc0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f876403d000 msgr2=0x7f876403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f876403d000 0x7f876403f4c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8870057000 tx=0x7f8870052000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 msgr2=0x5560c7fc7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7fc7110 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c010 tx=0x7f886c000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 shutdown_connections 2024-09-19T11:25:09.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5560c7fd0bd0 0x5560c7fd2fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f876403d000 0x7f876403f4c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5560c7ecebb0 0x5560c7fce760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 --2- 172.21.15.28:0/3854757129 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5560c7ecd8e0 0x5560c7fc7110 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 >> 172.21.15.28:0/3854757129 conn(0x5560c7ec8cb0 msgr2=0x5560c7fb8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 shutdown_connections 2024-09-19T11:25:09.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.579+0000 7f887ec0d800 1 -- 172.21.15.28:0/3854757129 wait complete. 2024-09-19T11:25:09.612 DEBUG:tasks.util.rgw: json result: {'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-09-19T11:25:09.564069Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:09.564057Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.612 ERROR:root:{'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-09-19T11:25:09.564069Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:09.564057Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.613 ERROR:root:0 2024-09-19T11:25:09.613 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:09.626 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:09.626 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'a_user'] 2024-09-19T11:25:09.627 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'a_user'] 2024-09-19T11:25:09.627 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid a_user 2024-09-19T11:25:09.673 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:09.673 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:09.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.672+0000 7fa32d725800 1 Processor -- start 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.672+0000 7fa32d725800 1 -- start start 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa32d725800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55ec4ccd5c90 0x55ec4ccd6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa32d725800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ec4ccd7200 con 0x55ec4ccd65d0 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa32d725800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55ec4cb627d0 con 0x55ec4ccd5c90 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa329333640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55ec4ccd5c90 0x55ec4ccd6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa328b32640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55ec4ccd65d0 0x55ec4ccd6a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48326/0 (socket says 172.21.15.28:48326) 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa328b32640 1 -- 172.21.15.28:0/1210625775 learned_addr learned my addr 172.21.15.28:0/1210625775 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa329333640 1 -- 172.21.15.28:0/1210625775 >> v1:172.21.15.22:6789/0 conn(0x55ec4ccd65d0 legacy=0x55ec4ccd6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa31bfff640 1 -- 172.21.15.28:0/1210625775 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511648882 0 0) 0x55ec4ccd7200 con 0x55ec4ccd65d0 2024-09-19T11:25:09.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.673+0000 7fa329333640 1 -- 172.21.15.28:0/1210625775 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4ccd73d0 con 0x55ec4ccd5c90 2024-09-19T11:25:09.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.675+0000 7fa329333640 1 --2- 172.21.15.28:0/1210625775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4ccd6090 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00a470 tx=0x7fa31c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8775a1a059eb7a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.675+0000 7fa31bfff640 1 -- 172.21.15.28:0/1210625775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa31c053570 con 0x55ec4ccd5c90 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.675+0000 7fa31bfff640 1 -- 172.21.15.28:0/1210625775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c058070 con 0x55ec4ccd5c90 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.675+0000 7fa31bfff640 1 -- 172.21.15.28:0/1210625775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa31c053880 con 0x55ec4ccd5c90 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.676+0000 7fa32d725800 1 -- 172.21.15.28:0/1210625775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 msgr2=0x55ec4ccd6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.676+0000 7fa32d725800 1 --2- 172.21.15.28:0/1210625775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4ccd6090 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00a470 tx=0x7fa31c045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.676+0000 7fa32d725800 1 -- 172.21.15.28:0/1210625775 shutdown_connections 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.676+0000 7fa32d725800 1 --2- 172.21.15.28:0/1210625775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4ccd6090 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.676+0000 7fa32d725800 1 -- 172.21.15.28:0/1210625775 >> 172.21.15.28:0/1210625775 conn(0x55ec4ccd1a60 msgr2=0x55ec4ccd3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.676+0000 7fa32d725800 1 -- 172.21.15.28:0/1210625775 shutdown_connections 2024-09-19T11:25:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.676+0000 7fa32d725800 1 -- 172.21.15.28:0/1210625775 wait complete. 2024-09-19T11:25:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.691+0000 7fa32d725800 1 Processor -- start 2024-09-19T11:25:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.691+0000 7fa32d725800 1 -- start start 2024-09-19T11:25:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.691+0000 7fa32d725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdc9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.691+0000 7fa32d725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ec4cb6fdd0 con 0x55ec4ccd5c90 2024-09-19T11:25:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.691+0000 7fa329333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdc9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.691+0000 7fa329333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdc9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45106/0 (socket says 172.21.15.28:45106) 2024-09-19T11:25:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.691+0000 7fa329333640 1 -- 172.21.15.28:0/3713035215 learned_addr learned my addr 172.21.15.28:0/3713035215 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.692+0000 7fa329333640 1 -- 172.21.15.28:0/3713035215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4cdc98f0 con 0x55ec4ccd5c90 2024-09-19T11:25:09.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.692+0000 7fa329333640 1 --2- 172.21.15.28:0/3713035215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdc9300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa31c025c20 tx=0x7fa31c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9ba62a8b0678d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.692+0000 7fa217606640 1 -- 172.21.15.28:0/3713035215 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa31c053440 con 0x55ec4ccd5c90 2024-09-19T11:25:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.692+0000 7fa217606640 1 -- 172.21.15.28:0/3713035215 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c063070 con 0x55ec4ccd5c90 2024-09-19T11:25:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.692+0000 7fa217606640 1 -- 172.21.15.28:0/3713035215 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa31c053730 con 0x55ec4ccd5c90 2024-09-19T11:25:09.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 -- 172.21.15.28:0/3713035215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 msgr2=0x55ec4cdc9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 --2- 172.21.15.28:0/3713035215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdc9300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa31c025c20 tx=0x7fa31c051000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 -- 172.21.15.28:0/3713035215 shutdown_connections 2024-09-19T11:25:09.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 --2- 172.21.15.28:0/3713035215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdc9300 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 -- 172.21.15.28:0/3713035215 >> 172.21.15.28:0/3713035215 conn(0x55ec4ccd1a60 msgr2=0x55ec4cdc4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 -- 172.21.15.28:0/3713035215 shutdown_connections 2024-09-19T11:25:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 -- 172.21.15.28:0/3713035215 wait complete. 2024-09-19T11:25:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 Processor -- start 2024-09-19T11:25:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 -- start start 2024-09-19T11:25:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdd0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.693+0000 7fa32d725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ec4ccd7960 con 0x55ec4ccd5c90 2024-09-19T11:25:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa329333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdd0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa329333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdd0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45110/0 (socket says 172.21.15.28:45110) 2024-09-19T11:25:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa329333640 1 -- 172.21.15.28:0/827584338 learned_addr learned my addr 172.21.15.28:0/827584338 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa329333640 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4cdd1370 con 0x55ec4ccd5c90 2024-09-19T11:25:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa329333640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdd0e10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00a470 tx=0x7fa31c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa215e03640 1 -- 172.21.15.28:0/827584338 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa31c068470 con 0x55ec4ccd5c90 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa215e03640 1 -- 172.21.15.28:0/827584338 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c068610 con 0x55ec4ccd5c90 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec4cdd1600 con 0x55ec4ccd5c90 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa215e03640 1 -- 172.21.15.28:0/827584338 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa31c0688f0 con 0x55ec4ccd5c90 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.694+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4cdd1f80 con 0x55ec4ccd5c90 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.695+0000 7fa215e03640 1 -- 172.21.15.28:0/827584338 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa31c07a060 con 0x55ec4ccd5c90 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.695+0000 7fa215e03640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa21003cfb0 0x7fa21003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.695+0000 7fa215e03640 1 -- 172.21.15.28:0/827584338 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa31c098850 con 0x55ec4ccd5c90 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.695+0000 7fa328b32640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa21003cfb0 0x7fa21003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.695+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ccd65d0 0x55ec4cdd8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.695+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4cdd89a0 con 0x55ec4ccd65d0 2024-09-19T11:25:09.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.696+0000 7fa329b34640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ccd65d0 0x55ec4cdd8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.696+0000 7fa328b32640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa21003cfb0 0x7fa21003f470 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa320000c00 tx=0x7fa3200013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.696+0000 7fa329b34640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ccd65d0 0x55ec4cdd8460 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.696+0000 7fa329b34640 1 -- 172.21.15.28:0/827584338 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa32400a4d0 con 0x55ec4ccd65d0 2024-09-19T11:25:09.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.696+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4cdd8d40 con 0x55ec4ccd65d0 2024-09-19T11:25:09.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.696+0000 7fa329b34640 1 -- 172.21.15.28:0/827584338 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa32400a4d0 con 0x55ec4ccd65d0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4cdda7c0 0x55ec4cddcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4cddd3c0 con 0x55ec4cdda7c0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa329333640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4cdda7c0 0x55ec4cddcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa329333640 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4cdda7c0 0x55ec4cddcc30 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa329333640 1 -- 172.21.15.28:0/827584338 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa31c095040 con 0x55ec4cdda7c0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4cdd9590 con 0x55ec4ccd65d0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa329b34640 1 -- 172.21.15.28:0/827584338 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa32400a4d0 con 0x55ec4ccd65d0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.697+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4cdd9930 con 0x55ec4cdda7c0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa329333640 1 -- 172.21.15.28:0/827584338 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa31c095040 con 0x55ec4cdda7c0 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa32d725800 1 Processor -- start 2024-09-19T11:25:09.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa32d725800 1 -- start start 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa32d725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdebc30 0x55ec4cdec050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa32d725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ec4cbce4a0 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa329b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdebc30 0x55ec4cdec050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa329b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdebc30 0x55ec4cdec050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45116/0 (socket says 172.21.15.28:45116) 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.698+0000 7fa329b34640 1 -- 172.21.15.28:0/195930649 learned_addr learned my addr 172.21.15.28:0/195930649 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.699+0000 7fa329b34640 1 -- 172.21.15.28:0/195930649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4cdec690 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.699+0000 7fa329b34640 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdebc30 0x55ec4cdec050 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55ec4cdd1b60 tx=0x7fa3240167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.699+0000 7fa20effd640 1 -- 172.21.15.28:0/195930649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa324013640 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.699+0000 7fa20effd640 1 -- 172.21.15.28:0/195930649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3240137e0 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.699+0000 7fa20effd640 1 -- 172.21.15.28:0/195930649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa324013ae0 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.699+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec4cdecec0 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.699+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4cded460 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.700+0000 7fa20effd640 1 -- 172.21.15.28:0/195930649 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa324011040 con 0x55ec4cdebc30 2024-09-19T11:25:09.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.700+0000 7fa20effd640 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa20003cfb0 0x7fa20003f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.700+0000 7fa20effd640 1 -- 172.21.15.28:0/195930649 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa32400e040 con 0x55ec4cdebc30 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.700+0000 7fa329333640 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa20003cfb0 0x7fa20003f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.700+0000 7fa32d725800 1 Processor -- start 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa32d725800 1 -- start start 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa329333640 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa20003cfb0 0x7fa20003f470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55ec4cdd2e90 tx=0x7fa31c05d650 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa32d725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdf8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa32d725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ec4cdc9cc0 con 0x55ec4cdf8750 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa329b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdf8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa329b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdf8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45124/0 (socket says 172.21.15.28:45124) 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa329b34640 1 -- 172.21.15.28:0/1308834905 learned_addr learned my addr 172.21.15.28:0/1308834905 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa329b34640 1 -- 172.21.15.28:0/1308834905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4cdf9140 con 0x55ec4cdf8750 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa329b34640 1 --2- 172.21.15.28:0/1308834905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdf8b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa324054830 tx=0x7fa324013d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184f3a0c919b87b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa20cff9640 1 -- 172.21.15.28:0/1308834905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa324051020 con 0x55ec4cdf8750 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa20cff9640 1 -- 172.21.15.28:0/1308834905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa324021cf0 con 0x55ec4cdf8750 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.701+0000 7fa20cff9640 1 -- 172.21.15.28:0/1308834905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa32404de60 con 0x55ec4cdf8750 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 -- 172.21.15.28:0/1308834905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 msgr2=0x55ec4cdf8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 --2- 172.21.15.28:0/1308834905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdf8b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa324054830 tx=0x7fa324013d90 comp rx=0 tx=0).stop 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 -- 172.21.15.28:0/1308834905 shutdown_connections 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 --2- 172.21.15.28:0/1308834905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdf8b50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 -- 172.21.15.28:0/1308834905 >> 172.21.15.28:0/1308834905 conn(0x55ec4cdf4b50 msgr2=0x55ec4cdf2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 -- 172.21.15.28:0/1308834905 shutdown_connections 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 -- 172.21.15.28:0/1308834905 wait complete. 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 Processor -- start 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 -- start start 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdffd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.702+0000 7fa32d725800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ec4cdf9760 con 0x55ec4cdf8750 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa329b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa329b34640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdffd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45126/0 (socket says 172.21.15.28:45126) 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 learned_addr learned my addr 172.21.15.28:0/1654773186 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4ce00270 con 0x55ec4cdf8750 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa329b34640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdffd10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa324005a50 tx=0x7fa3240547c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa1feffd640 1 -- 172.21.15.28:0/1654773186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa324021440 con 0x55ec4cdf8750 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa1feffd640 1 -- 172.21.15.28:0/1654773186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3240215e0 con 0x55ec4cdf8750 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec4ce00500 con 0x55ec4cdf8750 2024-09-19T11:25:09.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa1feffd640 1 -- 172.21.15.28:0/1654773186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa324045b50 con 0x55ec4cdf8750 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.703+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4ce01090 con 0x55ec4cdf8750 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.704+0000 7fa1feffd640 1 -- 172.21.15.28:0/1654773186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fa324045cf0 con 0x55ec4cdf8750 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.704+0000 7fa1feffd640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1f403d000 0x7fa1f403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.704+0000 7fa1feffd640 1 -- 172.21.15.28:0/1654773186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fa32404c070 con 0x55ec4cdf8750 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.704+0000 7fa329333640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1f403d000 0x7fa1f403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.705+0000 7fa329333640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1f403d000 0x7fa1f403f4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa31c05d790 tx=0x7fa31c089000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.705+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.705+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ce0b6b0 0x55ec4ce0db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.705+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce0e2c0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.705+0000 7fa328b32640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ce0b6b0 0x55ec4ce0db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.705+0000 7fa328b32640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ce0b6b0 0x55ec4ce0db70 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa32000f5f0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 realm 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce0aa00 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa32000f5f0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce0edb0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa32000f5f0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce0f680 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa32000f5f0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4ce100a0 0x55ec4ce12490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce12c00 con 0x55ec4ce100a0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.706+0000 7fa329b34640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4ce100a0 0x55ec4ce12490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa329b34640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4ce100a0 0x55ec4ce12490 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa32d725800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa32d725800 20 searching for the correct realm 2024-09-19T11:25:09.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa329b34640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa324034c20 0x7fa324053900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa324034c20 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.707+0000 7fa329333640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa324034c20 0x7fa324053900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.708+0000 7fa329333640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa324034c20 0x7fa324053900 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.708+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.708+0000 7fa329333640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa31c0a5040 0x7fa31c08f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.708+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa31c0a5040 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.708+0000 7fa328b32640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa31c0a5040 0x7fa31c08f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.708+0000 7fa328b32640 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa31c0a5040 0x7fa31c08f930 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32000f5f0 con 0x7fa31c0a5040 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32000f5f0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa31c0a5040 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.709+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa31c0a5040 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa324034c20 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.710+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa31c0a5040 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa324034c20 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.711+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa324034c20 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.712+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa31c0a5040 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.713+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa324034c20 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa31c0a5040 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.714+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa31c0a5040 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.715+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32002b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.715+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa324034c20 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.715+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.715+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x55ec4ce100a0 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.715+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.715+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14530 con 0x7fa324034c20 2024-09-19T11:25:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.715+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7fa31c0b5040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x7fa31c0b5040 con 0x7fa324034c20 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce135c0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce13e60 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.716+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa32002b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce149b0 con 0x55ec4ce100a0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 zone default found 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 4 Realm: () 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 zonegroup default 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce14d50 con 0x7fa31c0a5040 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa32002b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce17030 con 0x55ec4ce100a0 2024-09-19T11:25:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.717+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce173d0 con 0x55ec4ce100a0 2024-09-19T11:25:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.718+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.718+0000 7fa32d725800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.718+0000 7fa32d725800 20 started sync module instance, tier type = 2024-09-19T11:25:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.718+0000 7fa32d725800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.718+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce164c0 con 0x7fa324034c20 2024-09-19T11:25:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.719+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.719+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94473390483632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce16120 con 0x7fa324034c20 2024-09-19T11:25:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.719+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce181c0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.720+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94473390483632] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.720+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.720+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94473390487552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce18180 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.720+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce18c70 con 0x55ec4ce100a0 2024-09-19T11:25:09.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.721+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94473390487552] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.722+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.722+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94473390491424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce18c30 con 0x55ec4ce100a0 2024-09-19T11:25:09.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.722+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce19720 con 0x7fa324034c20 2024-09-19T11:25:09.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.723+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94473390491424] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.723+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.723+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94473390494256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce19720 con 0x7fa324034c20 2024-09-19T11:25:09.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.723+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1a1d0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.724+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94473390494256] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.724+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.724+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94473390488608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1a1d0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.724+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1ac60 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.725+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94473390488608] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.725+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.725+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94473390499632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1ac20 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.725+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1b690 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.726+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94473390499632] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.726+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.726+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94473390502336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1b690 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.726+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1c050 con 0x7fa31c0a5040 2024-09-19T11:25:09.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94473390502336] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94473390496944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1c050 con 0x7fa31c0a5040 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 20 add_watcher() i=0 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 20 add_watcher() i=1 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 20 add_watcher() i=2 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 20 add_watcher() i=3 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 20 add_watcher() i=4 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 20 add_watcher() i=5 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.727+0000 7fa32d725800 20 add_watcher() i=6 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.728+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94473390496944] v33'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.728+0000 7fa32d725800 20 add_watcher() i=7 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.728+0000 7fa32d725800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.728+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1f4b0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.728+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.728+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94473390505328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce1c9d0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.729+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94473390505328] v33'252 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.730+0000 7fa1df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.730+0000 7fa32d725800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.730+0000 7fa32d725800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.730+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce27d50 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.731+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.731+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce27d50 con 0x7fa31c0a5040 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.732+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.732+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce287f0 con 0x55ec4ce100a0 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.733+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.733+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce28880 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.733+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'253 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.734+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce28d30 con 0x7fa324034c20 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.734+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'80 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.735+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce28dc0 con 0x55ec4ce100a0 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.735+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'53 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.735+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29390 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.736+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.737+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce27d50 con 0x7fa31c0a5040 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.738+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'53 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.738+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce27d50 con 0x7fa324034c20 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.739+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'31 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.739+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce27d50 con 0x55ec4ce100a0 2024-09-19T11:25:09.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.740+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.740+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce27d50 con 0x7fa324034c20 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.741+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'81 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.741+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x7fa31c0a5040 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.742+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'36 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.742+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x7fa324034c20 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.743+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.743+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x7fa31c0a5040 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.744+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'60 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.744+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.745+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'254 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.745+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce100a0 2024-09-19T11:25:09.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.746+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'74 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.746+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x7fa324034c20 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.747+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'82 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.747+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce100a0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.748+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.748+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.749+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'38 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.749+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.750+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'255 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.750+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce100a0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.751+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'75 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.751+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x7fa324034c20 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.752+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'40 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.752+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce100a0 2024-09-19T11:25:09.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.753+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.753+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29530 con 0x55ec4ce100a0 2024-09-19T11:25:09.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.754+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.754+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29a30 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.755+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'26 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.755+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29a10 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.756+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'54 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.756+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29a10 con 0x7fa31c0a5040 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.757+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'34 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.757+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce29a10 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.758+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.758+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce28880 con 0x7fa31c0a5040 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.759+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'54 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.759+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce28880 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.760+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'76 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.760+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce28880 con 0x7fa31c0a5040 2024-09-19T11:25:09.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.761+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'61 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.761+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce28880 con 0x7fa324034c20 2024-09-19T11:25:09.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.762+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa1de7e4640 20 reqs_thread_entry: start 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 20 Filter name: none 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa14ffff640 20 reqs_thread_entry: start 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:94 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce54190 con 0x55ec4ce100a0 2024-09-19T11:25:09.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(94 a_user [call out=48b,read 0~363 out=363b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 234+0+486 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 20 rados_obj.operate() r=0 bl.length=363 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x13 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.763+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:95 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce56ac0 con 0x55ec4ce100a0 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.764+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(95 a_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.764+0000 7fa32d725800 10 removing key index: true_admin_akey 2024-09-19T11:25:09.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.764+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:96 8.15 8:afe09604:users.keys::true_admin_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce54aa0 con 0x55ec4ce100a0 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(96 true_admin_akey [delete] v33'4 uv4 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa32d725800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify cookie 94473390740912 in=177b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce57070 con 0x7fa31c0a5040 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== watch-notify(notify (1) cookie 94473390499632 notify 141733920813 ret 0) ==== 207+0+0 (crc 0 0 0) 0x55ec4ce28560 con 0x7fa31c0a5040 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa20cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920813 cookie 94473390499632 notifier 4865 bl.length()=165 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(97 notify.5 [notify cookie 94473390740912 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa20cff9640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fa1ec003d60 con 0x7fa31c0a5040 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(98 notify.5 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== watch-notify(notify_complete (2) cookie 94473390740912 notify 141733920813 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55ec4ce00500 con 0x7fa31c0a5040 2024-09-19T11:25:09.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa32d725800 10 removing email index: 2024-09-19T11:25:09.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa32d725800 10 removing user buckets index 2024-09-19T11:25:09.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.765+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:99 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce56fd0 con 0x55ec4ce100a0 2024-09-19T11:25:09.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.766+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(99 a_user.buckets [delete] v33'1 uv0 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.766+0000 7fa32d725800 10 removing user index: a_user 2024-09-19T11:25:09.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.766+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:100 8.14 8:2efb937b:users.uid::a_user:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce56fd0 con 0x55ec4ce100a0 2024-09-19T11:25:09.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.767+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(100 a_user [call,call,delete] v33'6 uv6 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.767+0000 7fa32d725800 10 removing default.rgw.meta+users.uid+a_user from cache 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.767+0000 7fa32d725800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.767+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:101 7.15 7:a93a5511:::notify.2:head [notify cookie 94473390747600 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce555b0 con 0x55ec4ce100a0 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.768+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== watch-notify(notify (1) cookie 94473390491424 notify 141733920778 ret 0) ==== 197+0+0 (crc 0 0 0) 0x55ec4ce28340 con 0x55ec4ce100a0 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.768+0000 7fa20cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920778 cookie 94473390491424 notifier 4865 bl.length()=155 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.768+0000 7fa20cff9640 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:102 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fa1ec003d60 con 0x55ec4ce100a0 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.768+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(101 notify.2 [notify cookie 94473390747600 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.768+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(102 notify.2 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.768+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== watch-notify(notify_complete (2) cookie 94473390747600 notify 141733920778 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55ec4ce00270 con 0x55ec4ce100a0 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.769+0000 7fa1df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.769+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94473390505328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce56fd0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.770+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94473390505328] v33'256 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.770+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94473390483632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce56fd0 con 0x7fa324034c20 2024-09-19T11:25:09.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.771+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94473390483632] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.771+0000 7fa32d725800 20 remove_watcher() i=0 2024-09-19T11:25:09.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.771+0000 7fa32d725800 2 removed watcher, disabling cache 2024-09-19T11:25:09.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.771+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94473390487552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce56fd0 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.772+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94473390487552] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.772+0000 7fa32d725800 20 remove_watcher() i=1 2024-09-19T11:25:09.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.772+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94473390491424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce17800 con 0x55ec4ce100a0 2024-09-19T11:25:09.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.773+0000 7fa329b34640 1 -- 172.21.15.28:0/1654773186 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94473390491424] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32403a070 con 0x55ec4ce100a0 2024-09-19T11:25:09.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.773+0000 7fa32d725800 20 remove_watcher() i=2 2024-09-19T11:25:09.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.773+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94473390494256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce17800 con 0x7fa324034c20 2024-09-19T11:25:09.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.774+0000 7fa329333640 1 -- 172.21.15.28:0/1654773186 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94473390494256] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa31c095040 con 0x7fa324034c20 2024-09-19T11:25:09.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.774+0000 7fa32d725800 20 remove_watcher() i=3 2024-09-19T11:25:09.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.774+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94473390488608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce19230 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.775+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94473390488608] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x55ec4ce0b6b0 2024-09-19T11:25:09.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.775+0000 7fa32d725800 20 remove_watcher() i=4 2024-09-19T11:25:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.775+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94473390499632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce19230 con 0x7fa31c0a5040 2024-09-19T11:25:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.776+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94473390499632] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.776+0000 7fa32d725800 20 remove_watcher() i=5 2024-09-19T11:25:09.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.776+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94473390502336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce19230 con 0x7fa31c0a5040 2024-09-19T11:25:09.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.777+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94473390502336] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.777+0000 7fa32d725800 20 remove_watcher() i=6 2024-09-19T11:25:09.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.777+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94473390496944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55ec4ce17cb0 con 0x7fa31c0a5040 2024-09-19T11:25:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94473390496944] v33'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa32003b040 con 0x7fa31c0a5040 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 20 remove_watcher() i=7 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa31c0a5040 msgr2=0x7fa31c08f930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fa31c0a5040 0x7fa31c08f930 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ce0b6b0 msgr2=0x55ec4ce0db70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ce0b6b0 0x55ec4ce0db70 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa324034c20 msgr2=0x7fa324053900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fa324034c20 0x7fa324053900 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4ce100a0 msgr2=0x55ec4ce12490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4ce100a0 0x55ec4ce12490 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1f403d000 msgr2=0x7fa1f403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa1f403d000 0x7fa1f403f4c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa31c05d790 tx=0x7fa31c089000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 msgr2=0x55ec4cdffd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa328b32640 1 -- 172.21.15.28:0/1654773186 reap_dead start 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdffd10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa324005a50 tx=0x7fa3240547c0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 shutdown_connections 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.778+0000 7fa32d725800 1 --2- 172.21.15.28:0/1654773186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdf8750 0x55ec4cdffd10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 >> 172.21.15.28:0/1654773186 conn(0x55ec4cdf4b50 msgr2=0x55ec4cdf2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 shutdown_connections 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/1654773186 wait complete. 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa20003cfb0 msgr2=0x7fa20003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa20003cfb0 0x7fa20003f470 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55ec4cdd2e90 tx=0x7fa31c05d650 comp rx=0 tx=0).stop 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdebc30 msgr2=0x55ec4cdec050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdebc30 0x55ec4cdec050 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x55ec4cdd1b60 tx=0x7fa3240167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 shutdown_connections 2024-09-19T11:25:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa20003cfb0 0x7fa20003f470 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/195930649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4cdebc30 0x55ec4cdec050 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 >> 172.21.15.28:0/195930649 conn(0x55ec4cddffe0 msgr2=0x55ec4cde03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 shutdown_connections 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/195930649 wait complete. 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ccd65d0 msgr2=0x55ec4cdd8460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ccd65d0 0x55ec4cdd8460 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4cdda7c0 msgr2=0x55ec4cddcc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4cdda7c0 0x55ec4cddcc30 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa21003cfb0 msgr2=0x7fa21003f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa21003cfb0 0x7fa21003f470 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa320000c00 tx=0x7fa3200013b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 msgr2=0x55ec4cdd0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.779+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdd0e10 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00a470 tx=0x7fa31c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 shutdown_connections 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55ec4cdda7c0 0x55ec4cddcc30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fa21003cfb0 0x7fa21003f470 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55ec4ccd65d0 0x55ec4cdd8460 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 --2- 172.21.15.28:0/827584338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ec4ccd5c90 0x55ec4cdd0e10 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 >> 172.21.15.28:0/827584338 conn(0x55ec4ccd1a60 msgr2=0x55ec4cdca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 shutdown_connections 2024-09-19T11:25:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.780+0000 7fa32d725800 1 -- 172.21.15.28:0/827584338 wait complete. 2024-09-19T11:25:09.790 ERROR:root:None 2024-09-19T11:25:09.791 ERROR:root:0 2024-09-19T11:25:09.791 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-09-19T11:25:09.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-09-19T11:25:09.791 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=read 2024-09-19T11:25:09.839 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:09.839 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.838+0000 7ff833740800 1 Processor -- start 2024-09-19T11:25:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.838+0000 7ff833740800 1 -- start start 2024-09-19T11:25:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff833740800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5625b6d058a0 0x5625b6d06460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff833740800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5625b6d05f50 con 0x5625b6d069a0 2024-09-19T11:25:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff833740800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5625b6b907d0 con 0x5625b6d058a0 2024-09-19T11:25:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff82f486640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5625b6d058a0 0x5625b6d06460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff82ec85640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5625b6d069a0 0x5625b6d0b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48330/0 (socket says 172.21.15.28:48330) 2024-09-19T11:25:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff82ec85640 1 -- 172.21.15.28:0/1757460011 learned_addr learned my addr 172.21.15.28:0/1757460011 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff82f486640 1 -- 172.21.15.28:0/1757460011 >> v1:172.21.15.22:6789/0 conn(0x5625b6d069a0 legacy=0x5625b6d0b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff82e484640 1 -- 172.21.15.28:0/1757460011 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3823682517 0 0) 0x5625b6d05f50 con 0x5625b6d069a0 2024-09-19T11:25:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.839+0000 7ff82f486640 1 -- 172.21.15.28:0/1757460011 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b6d06e50 con 0x5625b6d058a0 2024-09-19T11:25:09.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.841+0000 7ff82f486640 1 --2- 172.21.15.28:0/1757460011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6d06460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff82000a470 tx=0x7ff820045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee536a2c1374fb51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.841+0000 7ff82e484640 1 -- 172.21.15.28:0/1757460011 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820053570 con 0x5625b6d058a0 2024-09-19T11:25:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.841+0000 7ff82e484640 1 -- 172.21.15.28:0/1757460011 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff820058070 con 0x5625b6d058a0 2024-09-19T11:25:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.841+0000 7ff82e484640 1 -- 172.21.15.28:0/1757460011 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820053880 con 0x5625b6d058a0 2024-09-19T11:25:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.841+0000 7ff833740800 1 -- 172.21.15.28:0/1757460011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 msgr2=0x5625b6d06460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.841+0000 7ff833740800 1 --2- 172.21.15.28:0/1757460011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6d06460 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff82000a470 tx=0x7ff820045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.842+0000 7ff833740800 1 -- 172.21.15.28:0/1757460011 shutdown_connections 2024-09-19T11:25:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.842+0000 7ff833740800 1 --2- 172.21.15.28:0/1757460011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6d06460 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.842+0000 7ff833740800 1 -- 172.21.15.28:0/1757460011 >> 172.21.15.28:0/1757460011 conn(0x5625b6d00c70 msgr2=0x5625b6d030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.842+0000 7ff833740800 1 -- 172.21.15.28:0/1757460011 shutdown_connections 2024-09-19T11:25:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.842+0000 7ff833740800 1 -- 172.21.15.28:0/1757460011 wait complete. 2024-09-19T11:25:09.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.857+0000 7ff833740800 1 Processor -- start 2024-09-19T11:25:09.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.857+0000 7ff833740800 1 -- start start 2024-09-19T11:25:09.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.857+0000 7ff833740800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6df5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.857+0000 7ff833740800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625b6b9ddd0 con 0x5625b6d058a0 2024-09-19T11:25:09.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.857+0000 7ff82f486640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6df5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.857+0000 7ff82f486640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6df5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45142/0 (socket says 172.21.15.28:45142) 2024-09-19T11:25:09.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.857+0000 7ff82f486640 1 -- 172.21.15.28:0/3365760547 learned_addr learned my addr 172.21.15.28:0/3365760547 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.858+0000 7ff82f486640 1 -- 172.21.15.28:0/3365760547 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b6df5740 con 0x5625b6d058a0 2024-09-19T11:25:09.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.858+0000 7ff82f486640 1 --2- 172.21.15.28:0/3365760547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6df5150 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff820045f50 tx=0x7ff820051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b8a68064944afd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.858+0000 7ff71d60a640 1 -- 172.21.15.28:0/3365760547 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820053c20 con 0x5625b6d058a0 2024-09-19T11:25:09.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.858+0000 7ff71d60a640 1 -- 172.21.15.28:0/3365760547 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff820063030 con 0x5625b6d058a0 2024-09-19T11:25:09.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.858+0000 7ff71d60a640 1 -- 172.21.15.28:0/3365760547 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820053440 con 0x5625b6d058a0 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 -- 172.21.15.28:0/3365760547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 msgr2=0x5625b6df5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 --2- 172.21.15.28:0/3365760547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6df5150 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff820045f50 tx=0x7ff820051000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 -- 172.21.15.28:0/3365760547 shutdown_connections 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 --2- 172.21.15.28:0/3365760547 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6df5150 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 -- 172.21.15.28:0/3365760547 >> 172.21.15.28:0/3365760547 conn(0x5625b6d00c70 msgr2=0x5625b6df0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 -- 172.21.15.28:0/3365760547 shutdown_connections 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 -- 172.21.15.28:0/3365760547 wait complete. 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 Processor -- start 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 -- start start 2024-09-19T11:25:09.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6c7b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff833740800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625b6d0ce20 con 0x5625b6d058a0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff82f486640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6c7b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff82f486640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6c7b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45158/0 (socket says 172.21.15.28:45158) 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.859+0000 7ff82f486640 1 -- 172.21.15.28:0/2196563726 learned_addr learned my addr 172.21.15.28:0/2196563726 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.860+0000 7ff82f486640 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b6c7b660 con 0x5625b6d058a0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.860+0000 7ff82f486640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6c7b100 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff820045f20 tx=0x7ff820002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.860+0000 7ff71be07640 1 -- 172.21.15.28:0/2196563726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820053570 con 0x5625b6d058a0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.860+0000 7ff71be07640 1 -- 172.21.15.28:0/2196563726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff820053710 con 0x5625b6d058a0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.860+0000 7ff71be07640 1 -- 172.21.15.28:0/2196563726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820068420 con 0x5625b6d058a0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.860+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5625b6c79a10 con 0x5625b6d058a0 2024-09-19T11:25:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.860+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b6c79f50 con 0x5625b6d058a0 2024-09-19T11:25:09.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.861+0000 7ff71be07640 1 -- 172.21.15.28:0/2196563726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff8200685c0 con 0x5625b6d058a0 2024-09-19T11:25:09.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.861+0000 7ff71be07640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff71403d000 0x7ff71403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.861+0000 7ff71be07640 1 -- 172.21.15.28:0/2196563726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff820070070 con 0x5625b6d058a0 2024-09-19T11:25:09.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.861+0000 7ff82ec85640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff71403d000 0x7ff71403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.862+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6d069a0 0x5625b6e05fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.862+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e06570 con 0x5625b6d069a0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.862+0000 7ff82fc87640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6d069a0 0x5625b6e05fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.862+0000 7ff82ec85640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff71403d000 0x7ff71403f4c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff824000c00 tx=0x7ff8240013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.862+0000 7ff82fc87640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6d069a0 0x5625b6e05fa0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.862+0000 7ff82fc87640 1 -- 172.21.15.28:0/2196563726 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6d069a0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.862+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e06e50 con 0x5625b6d069a0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.863+0000 7ff82fc87640 1 -- 172.21.15.28:0/2196563726 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6d069a0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.863+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e08880 0x5625b6e0acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.863+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e0b440 con 0x5625b6e08880 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.863+0000 7ff82f486640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e08880 0x5625b6e0acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.863+0000 7ff82f486640 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e08880 0x5625b6e0acb0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.863+0000 7ff82f486640 1 -- 172.21.15.28:0/2196563726 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff820051070 con 0x5625b6e08880 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.863+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e076a0 con 0x5625b6d069a0 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.864+0000 7ff82fc87640 1 -- 172.21.15.28:0/2196563726 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6d069a0 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.864+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e07a40 con 0x5625b6e08880 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.864+0000 7ff82f486640 1 -- 172.21.15.28:0/2196563726 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff820051070 con 0x5625b6e08880 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.864+0000 7ff833740800 1 Processor -- start 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.864+0000 7ff833740800 1 -- start start 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.864+0000 7ff833740800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e19be0 0x5625b6e1a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.864+0000 7ff833740800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625b6bfc4a0 con 0x5625b6e19be0 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff82fc87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e19be0 0x5625b6e1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff82fc87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e19be0 0x5625b6e1a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45168/0 (socket says 172.21.15.28:45168) 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff82fc87640 1 -- 172.21.15.28:0/1079109062 learned_addr learned my addr 172.21.15.28:0/1079109062 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff82fc87640 1 -- 172.21.15.28:0/1079109062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b6e1a640 con 0x5625b6e19be0 2024-09-19T11:25:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff82fc87640 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e19be0 0x5625b6e1a000 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x5625b6c7a170 tx=0x7ff8280167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff719602640 1 -- 172.21.15.28:0/1079109062 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828013640 con 0x5625b6e19be0 2024-09-19T11:25:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff719602640 1 -- 172.21.15.28:0/1079109062 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8280137e0 con 0x5625b6e19be0 2024-09-19T11:25:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5625b6e1ae70 con 0x5625b6e19be0 2024-09-19T11:25:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.865+0000 7ff719602640 1 -- 172.21.15.28:0/1079109062 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828013ae0 con 0x5625b6e19be0 2024-09-19T11:25:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.866+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b6e1b410 con 0x5625b6e19be0 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.866+0000 7ff719602640 1 -- 172.21.15.28:0/1079109062 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff828011040 con 0x5625b6e19be0 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.866+0000 7ff719602640 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff70803d050 0x7ff70803f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff719602640 1 -- 172.21.15.28:0/1079109062 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff82800e040 con 0x5625b6e19be0 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff82f486640 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff70803d050 0x7ff70803f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff833740800 1 Processor -- start 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff833740800 1 -- start start 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff82f486640 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff70803d050 0x7ff70803f510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5625b6c7a700 tx=0x7ff82005f890 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff833740800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e26a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff833740800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625b6df5b10 con 0x5625b6e26660 2024-09-19T11:25:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff82fc87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e26a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff82fc87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e26a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45172/0 (socket says 172.21.15.28:45172) 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.867+0000 7ff82fc87640 1 -- 172.21.15.28:0/1927614777 learned_addr learned my addr 172.21.15.28:0/1927614777 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff82fc87640 1 -- 172.21.15.28:0/1927614777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b6e27050 con 0x5625b6e26660 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff82fc87640 1 --2- 172.21.15.28:0/1927614777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e26a60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff828016cd0 tx=0x7ff828020460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc8d7ebf88e8a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff712ffd640 1 -- 172.21.15.28:0/1927614777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828051020 con 0x5625b6e26660 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff712ffd640 1 -- 172.21.15.28:0/1927614777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82801fdb0 con 0x5625b6e26660 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff712ffd640 1 -- 172.21.15.28:0/1927614777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff828021a40 con 0x5625b6e26660 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff833740800 1 -- 172.21.15.28:0/1927614777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 msgr2=0x5625b6e26a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff833740800 1 --2- 172.21.15.28:0/1927614777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e26a60 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff828016cd0 tx=0x7ff828020460 comp rx=0 tx=0).stop 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff833740800 1 -- 172.21.15.28:0/1927614777 shutdown_connections 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff833740800 1 --2- 172.21.15.28:0/1927614777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e26a60 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff833740800 1 -- 172.21.15.28:0/1927614777 >> 172.21.15.28:0/1927614777 conn(0x5625b6e22a60 msgr2=0x5625b6e20ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff833740800 1 -- 172.21.15.28:0/1927614777 shutdown_connections 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.868+0000 7ff833740800 1 -- 172.21.15.28:0/1927614777 wait complete. 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff833740800 1 Processor -- start 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff833740800 1 -- start start 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff833740800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e2dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff833740800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5625b6e27670 con 0x5625b6e26660 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff82fc87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e2dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff82fc87640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e2dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45174/0 (socket says 172.21.15.28:45174) 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 learned_addr learned my addr 172.21.15.28:0/234849232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.869+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b6e2e1d0 con 0x5625b6e26660 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.870+0000 7ff82fc87640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e2dc70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff828022ff0 tx=0x7ff828054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.870+0000 7ff7117fa640 1 -- 172.21.15.28:0/234849232 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8280143c0 con 0x5625b6e26660 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.870+0000 7ff7117fa640 1 -- 172.21.15.28:0/234849232 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8280532c0 con 0x5625b6e26660 2024-09-19T11:25:09.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.870+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5625b6e2e460 con 0x5625b6e26660 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.870+0000 7ff7117fa640 1 -- 172.21.15.28:0/234849232 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8280459d0 con 0x5625b6e26660 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.870+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b6e2e460 con 0x5625b6e26660 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.870+0000 7ff7117fa640 1 -- 172.21.15.28:0/234849232 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff828045b70 con 0x5625b6e26660 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.871+0000 7ff7117fa640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff6f803d000 0x7ff6f803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.871+0000 7ff7117fa640 1 -- 172.21.15.28:0/234849232 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff828034d50 con 0x5625b6e26660 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.871+0000 7ff82f486640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff6f803d000 0x7ff6f803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.871+0000 7ff82f486640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff6f803d000 0x7ff6f803f4c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff82000a330 tx=0x7ff82008a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6e39390 0x5625b6e3b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e3bfa0 con 0x5625b6e39390 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff82ec85640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6e39390 0x5625b6e3b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff82ec85640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6e39390 0x5625b6e3b850 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82400f5f0 con 0x5625b6e39390 2024-09-19T11:25:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff833740800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff833740800 20 realm 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.872+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e386e0 con 0x5625b6e39390 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82400f5f0 con 0x5625b6e39390 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e3ca90 con 0x5625b6e39390 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82400f5f0 con 0x5625b6e39390 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e3d360 con 0x5625b6e39390 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff82400f5f0 con 0x5625b6e39390 2024-09-19T11:25:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e3dd80 0x5625b6e40170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e408e0 con 0x5625b6e3dd80 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.873+0000 7ff82fc87640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e3dd80 0x5625b6e40170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff82fc87640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e3dd80 0x5625b6e40170 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff833740800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff833740800 20 searching for the correct realm 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff82fc87640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff82804d2d0 0x7ff8280147b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.874+0000 7ff82f486640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff82804d2d0 0x7ff8280147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.875+0000 7ff82f486640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff82804d2d0 0x7ff8280147b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.875+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.875+0000 7ff82f486640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff8200a5040 0x7ff82008f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.875+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff8200a5040 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.875+0000 7ff82ec85640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff8200a5040 0x7ff82008f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.875+0000 7ff82ec85640 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff8200a5040 0x7ff82008f6b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82400f5f0 con 0x7ff8200a5040 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82400f5f0 con 0x5625b6e39390 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff8200a5040 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x7ff8200a5040 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.876+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff8200a5040 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x7ff8200a5040 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.877+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff8200a5040 2024-09-19T11:25:09.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.878+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x7ff8200a5040 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.878+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.878+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.878+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.878+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.878+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.879+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff8200a5040 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x7ff8200a5040 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.880+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff8200a5040 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x7ff8200a5040 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.881+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff8200a5040 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82402b040 con 0x7ff8200a5040 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e3dd80 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.882+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x5625b6e39390 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42210 con 0x7ff82804d2d0 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e412d0 con 0x5625b6e39390 2024-09-19T11:25:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.883+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e41b70 con 0x5625b6e39390 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff82402b040 con 0x5625b6e39390 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42690 con 0x5625b6e3dd80 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 zone default found 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 4 Realm: () 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 zonegroup default 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e42a30 con 0x7ff8200a5040 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff82402b040 con 0x7ff8200a5040 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e44cc0 con 0x5625b6e3dd80 2024-09-19T11:25:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.884+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff833740800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e45060 con 0x5625b6e3dd80 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff833740800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff833740800 20 started sync module instance, tier type = 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff833740800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.885+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e44150 con 0x7ff82804d2d0 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.886+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.886+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94719982191984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e43db0 con 0x7ff82804d2d0 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.886+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e45f40 con 0x5625b6e39390 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.887+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94719982191984] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.888+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.888+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94719982195824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e45f40 con 0x5625b6e39390 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.888+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e46940 con 0x5625b6e3dd80 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.889+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94719982195824] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.889+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.889+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94719982199888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e45f40 con 0x5625b6e3dd80 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.889+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e473b0 con 0x7ff82804d2d0 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.890+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.890+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94719982199888] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.890+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94719982202528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e47370 con 0x7ff82804d2d0 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.890+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e45af0 con 0x5625b6e39390 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.891+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.891+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94719982206528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e45af0 con 0x5625b6e39390 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.891+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e48de0 con 0x7ff8200a5040 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.891+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94719982202528] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.892+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.892+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94719982206528] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.892+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94719982209200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e48da0 con 0x7ff8200a5040 2024-09-19T11:25:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.892+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e48280 con 0x7ff8200a5040 2024-09-19T11:25:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.893+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94719982209200] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.893+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.893+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94719982207616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e49340 con 0x7ff8200a5040 2024-09-19T11:25:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.893+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e49d00 con 0x7ff8200a5040 2024-09-19T11:25:09.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94719982207616] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94719982204832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e49d00 con 0x7ff8200a5040 2024-09-19T11:25:09.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 20 add_watcher() i=0 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 20 add_watcher() i=1 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 20 add_watcher() i=2 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 20 add_watcher() i=3 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 20 add_watcher() i=4 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 20 add_watcher() i=5 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.894+0000 7ff833740800 20 add_watcher() i=6 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.895+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94719982204832] v33'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.895+0000 7ff833740800 20 add_watcher() i=7 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.895+0000 7ff833740800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.895+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e4d010 con 0x5625b6e39390 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.895+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.895+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94719982213568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e4a290 con 0x5625b6e39390 2024-09-19T11:25:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.896+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94719982213568] v33'257 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff6e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 5 note: GC not initialized 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff6e4fe1640 20 reqs_thread_entry: start 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 20 Filter name: none 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff65e7fc640 20 reqs_thread_entry: start 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 20 rados->read ofs=0 len=0 2024-09-19T11:25:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.897+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7fff0 con 0x7ff8200a5040 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.898+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.898+0000 7ff833740800 20 rados_obj.operate() r=0 bl.length=460 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.898+0000 7ff833740800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.898+0000 7ff833740800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.898+0000 7ff833740800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.898+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e86490 con 0x7ff8200a5040 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v33'6 uv6 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff833740800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff833740800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff833740800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94719982460496 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e81ad0 con 0x5625b6e39390 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 94719982206528 notify 141733920796 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7ff824018040 con 0x5625b6e39390 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff712ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920796 cookie 94719982206528 notifier 4872 bl.length()=666 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 94719982460496 out=8b] v0'0 uv258 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff712ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff712ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff712ffd640 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7ff6f4003ea0 con 0x5625b6e39390 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.900+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 94719982460496 notify 141733920796 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff82401db60 con 0x5625b6e39390 2024-09-19T11:25:09.911 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"read"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.901+0000 7ff6e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.901+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94719982213568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x5625b6e39390 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.902+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94719982213568] v33'258 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.902+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94719982191984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x7ff82804d2d0 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.903+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94719982191984] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.903+0000 7ff833740800 20 remove_watcher() i=0 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.903+0000 7ff833740800 2 removed watcher, disabling cache 2024-09-19T11:25:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.903+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94719982195824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x5625b6e39390 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.905+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94719982195824] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.905+0000 7ff833740800 20 remove_watcher() i=1 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.905+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94719982199888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x5625b6e3dd80 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.906+0000 7ff82fc87640 1 -- 172.21.15.28:0/234849232 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94719982199888] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82800a4d0 con 0x5625b6e3dd80 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.906+0000 7ff833740800 20 remove_watcher() i=2 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.906+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94719982202528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x7ff82804d2d0 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.907+0000 7ff82f486640 1 -- 172.21.15.28:0/234849232 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94719982202528] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820051070 con 0x7ff82804d2d0 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.907+0000 7ff833740800 20 remove_watcher() i=3 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.907+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94719982206528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x5625b6e39390 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.908+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94719982206528] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x5625b6e39390 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.908+0000 7ff833740800 20 remove_watcher() i=4 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.908+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94719982209200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x7ff8200a5040 2024-09-19T11:25:09.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.909+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94719982209200] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.909+0000 7ff833740800 20 remove_watcher() i=5 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.909+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94719982207616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x7ff8200a5040 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.910+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94719982207616] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.910+0000 7ff833740800 20 remove_watcher() i=6 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.910+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94719982204832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5625b6e7de60 con 0x7ff8200a5040 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94719982204832] v33'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82403b040 con 0x7ff8200a5040 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 20 remove_watcher() i=7 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff8200a5040 msgr2=0x7ff82008f6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff8200a5040 0x7ff82008f6b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6e39390 msgr2=0x5625b6e3b850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6e39390 0x5625b6e3b850 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff82804d2d0 msgr2=0x7ff8280147b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff82804d2d0 0x7ff8280147b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e3dd80 msgr2=0x5625b6e40170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e3dd80 0x5625b6e40170 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff6f803d000 msgr2=0x7ff6f803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff6f803d000 0x7ff6f803f4c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff82000a330 tx=0x7ff82008a000 comp rx=0 tx=0).stop 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 msgr2=0x5625b6e2dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff82ec85640 1 -- 172.21.15.28:0/234849232 reap_dead start 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e2dc70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff828022ff0 tx=0x7ff828054790 comp rx=0 tx=0).stop 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 shutdown_connections 2024-09-19T11:25:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 --2- 172.21.15.28:0/234849232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e26660 0x5625b6e2dc70 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 >> 172.21.15.28:0/234849232 conn(0x5625b6e22a60 msgr2=0x5625b6e20c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 shutdown_connections 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/234849232 wait complete. 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.911+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff70803d050 msgr2=0x7ff70803f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff70803d050 0x7ff70803f510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x5625b6c7a700 tx=0x7ff82005f890 comp rx=0 tx=0).stop 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e19be0 msgr2=0x5625b6e1a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e19be0 0x5625b6e1a000 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x5625b6c7a170 tx=0x7ff8280167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 shutdown_connections 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff70803d050 0x7ff70803f510 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/1079109062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6e19be0 0x5625b6e1a000 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 >> 172.21.15.28:0/1079109062 conn(0x5625b6e0df90 msgr2=0x5625b6e0e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 shutdown_connections 2024-09-19T11:25:09.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/1079109062 wait complete. 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6d069a0 msgr2=0x5625b6e05fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6d069a0 0x5625b6e05fa0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e08880 msgr2=0x5625b6e0acb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e08880 0x5625b6e0acb0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff71403d000 msgr2=0x7ff71403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff71403d000 0x7ff71403f4c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff824000c00 tx=0x7ff8240013b0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 msgr2=0x5625b6c7b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6c7b100 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff820045f20 tx=0x7ff820002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 shutdown_connections 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5625b6e08880 0x5625b6e0acb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff71403d000 0x7ff71403f4c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5625b6d069a0 0x5625b6e05fa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 --2- 172.21.15.28:0/2196563726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5625b6d058a0 0x5625b6c7b100 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 >> 172.21.15.28:0/2196563726 conn(0x5625b6d00c70 msgr2=0x5625b6df6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 shutdown_connections 2024-09-19T11:25:09.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.912+0000 7ff833740800 1 -- 172.21.15.28:0/2196563726 wait complete. 2024-09-19T11:25:09.924 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.924 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:09.924 ERROR:root:0 2024-09-19T11:25:09.924 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:09.937 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:09.937 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-09-19T11:25:09.937 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-09-19T11:25:09.938 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'user-info-without-keys=*' 2024-09-19T11:25:09.985 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:09.985 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:09.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.984+0000 7ff55ed42800 1 Processor -- start 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55ed42800 1 -- start start 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55ed42800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5576a419aa90 0x5576a419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55ed42800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5576a419d5d0 con 0x5576a4196f20 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55ed42800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5576a40217d0 con 0x5576a419aa90 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55a285640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5576a419aa90 0x5576a419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55aa86640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5576a4196f20 0x5576a4199310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48336/0 (socket says 172.21.15.28:48336) 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55a285640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5576a419aa90 0x5576a419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45180/0 (socket says 172.21.15.28:45180) 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55aa86640 1 -- 172.21.15.28:0/2437526819 learned_addr learned my addr 172.21.15.28:0/2437526819 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55a285640 1 -- 172.21.15.28:0/2437526819 >> v1:172.21.15.22:6789/0 conn(0x5576a4196f20 legacy=0x5576a4199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.985+0000 7ff55a285640 1 -- 172.21.15.28:0/2437526819 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a419e800 con 0x5576a419aa90 2024-09-19T11:25:09.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.987+0000 7ff55a285640 1 --2- 172.21.15.28:0/2437526819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a419aa90 0x5576a419cf50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff55000a470 tx=0x7ff550045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8064e9bcc3019b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:09.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.987+0000 7ff559a84640 1 -- 172.21.15.28:0/2437526819 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff550053570 con 0x5576a419aa90 2024-09-19T11:25:09.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.987+0000 7ff559a84640 1 -- 172.21.15.28:0/2437526819 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff550058070 con 0x5576a419aa90 2024-09-19T11:25:09.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.987+0000 7ff559a84640 1 -- 172.21.15.28:0/2437526819 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff550053880 con 0x5576a419aa90 2024-09-19T11:25:09.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.988+0000 7ff55ed42800 1 -- 172.21.15.28:0/2437526819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a419aa90 msgr2=0x5576a419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:09.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.988+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2437526819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a419aa90 0x5576a419cf50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff55000a470 tx=0x7ff550045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:09.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.988+0000 7ff55ed42800 1 -- 172.21.15.28:0/2437526819 shutdown_connections 2024-09-19T11:25:09.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.988+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2437526819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a419aa90 0x5576a419cf50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:09.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.988+0000 7ff55ed42800 1 -- 172.21.15.28:0/2437526819 >> 172.21.15.28:0/2437526819 conn(0x5576a4190a60 msgr2=0x5576a4192ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:09.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.988+0000 7ff55ed42800 1 -- 172.21.15.28:0/2437526819 shutdown_connections 2024-09-19T11:25:09.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:09.988+0000 7ff55ed42800 1 -- 172.21.15.28:0/2437526819 wait complete. 2024-09-19T11:25:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.003+0000 7ff55ed42800 1 Processor -- start 2024-09-19T11:25:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.003+0000 7ff55ed42800 1 -- start start 2024-09-19T11:25:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.003+0000 7ff55ed42800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4196690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.004+0000 7ff55ed42800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5576a402edd0 con 0x5576a4196f20 2024-09-19T11:25:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.004+0000 7ff55aa86640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.004+0000 7ff55aa86640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4196690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45190/0 (socket says 172.21.15.28:45190) 2024-09-19T11:25:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.004+0000 7ff55aa86640 1 -- 172.21.15.28:0/1604883194 learned_addr learned my addr 172.21.15.28:0/1604883194 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.004+0000 7ff55aa86640 1 -- 172.21.15.28:0/1604883194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a4196c80 con 0x5576a4196f20 2024-09-19T11:25:10.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.004+0000 7ff55aa86640 1 --2- 172.21.15.28:0/1604883194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4196690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff54c001e40 tx=0x7ff54c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4290542518ff6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.004+0000 7ff448e09640 1 -- 172.21.15.28:0/1604883194 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c012020 con 0x5576a4196f20 2024-09-19T11:25:10.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff448e09640 1 -- 172.21.15.28:0/1604883194 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54c00e5a0 con 0x5576a4196f20 2024-09-19T11:25:10.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff448e09640 1 -- 172.21.15.28:0/1604883194 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c00e8a0 con 0x5576a4196f20 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff55ed42800 1 -- 172.21.15.28:0/1604883194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 msgr2=0x5576a4196690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff55ed42800 1 --2- 172.21.15.28:0/1604883194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4196690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff54c001e40 tx=0x7ff54c001e70 comp rx=0 tx=0).stop 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff55ed42800 1 -- 172.21.15.28:0/1604883194 shutdown_connections 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff55ed42800 1 --2- 172.21.15.28:0/1604883194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4196690 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff55ed42800 1 -- 172.21.15.28:0/1604883194 >> 172.21.15.28:0/1604883194 conn(0x5576a4190a60 msgr2=0x5576a4281d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff55ed42800 1 -- 172.21.15.28:0/1604883194 shutdown_connections 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.005+0000 7ff55ed42800 1 -- 172.21.15.28:0/1604883194 wait complete. 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55ed42800 1 Processor -- start 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55ed42800 1 -- start start 2024-09-19T11:25:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55ed42800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4290090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55ed42800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5576a419daa0 con 0x5576a4196f20 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55aa86640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4290090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55aa86640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4290090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45198/0 (socket says 172.21.15.28:45198) 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55aa86640 1 -- 172.21.15.28:0/2334993587 learned_addr learned my addr 172.21.15.28:0/2334993587 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55aa86640 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a42905f0 con 0x5576a4196f20 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff55aa86640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4290090 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c010 tx=0x7ff54c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.006+0000 7ff447606640 1 -- 172.21.15.28:0/2334993587 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c012030 con 0x5576a4196f20 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.007+0000 7ff447606640 1 -- 172.21.15.28:0/2334993587 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54c0014a0 con 0x5576a4196f20 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.007+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576a42908e0 con 0x5576a4196f20 2024-09-19T11:25:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.007+0000 7ff447606640 1 -- 172.21.15.28:0/2334993587 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff54c01a7a0 con 0x5576a4196f20 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.007+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a4291490 con 0x5576a4196f20 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.007+0000 7ff447606640 1 -- 172.21.15.28:0/2334993587 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff54c015020 con 0x5576a4196f20 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff447606640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff44003d000 0x7ff44003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff447606640 1 -- 172.21.15.28:0/2334993587 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff54c020030 con 0x5576a4196f20 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff55a285640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff44003d000 0x7ff44003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a419aa90 0x5576a42976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a4297c20 con 0x5576a419aa90 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff55b287640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a419aa90 0x5576a42976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff55a285640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff44003d000 0x7ff44003f4c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff550057000 tx=0x7ff550052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.008+0000 7ff55b287640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a419aa90 0x5576a42976e0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.009+0000 7ff55b287640 1 -- 172.21.15.28:0/2334993587 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff55400a4d0 con 0x5576a419aa90 2024-09-19T11:25:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.009+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a4297fc0 con 0x5576a419aa90 2024-09-19T11:25:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.009+0000 7ff55b287640 1 -- 172.21.15.28:0/2334993587 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff55400a4d0 con 0x5576a419aa90 2024-09-19T11:25:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.009+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a4299a80 0x5576a429bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.009+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a429c6b0 con 0x5576a4299a80 2024-09-19T11:25:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.009+0000 7ff55aa86640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a4299a80 0x5576a429bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.009+0000 7ff55aa86640 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a4299a80 0x5576a429bf20 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.010+0000 7ff55aa86640 1 -- 172.21.15.28:0/2334993587 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff54c0187b0 con 0x5576a4299a80 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.010+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a4298810 con 0x5576a419aa90 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.010+0000 7ff55b287640 1 -- 172.21.15.28:0/2334993587 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff55400a4d0 con 0x5576a419aa90 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.010+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a4298bb0 con 0x5576a4299a80 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.010+0000 7ff55aa86640 1 -- 172.21.15.28:0/2334993587 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff54c0187b0 con 0x5576a4299a80 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55ed42800 1 Processor -- start 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55ed42800 1 -- start start 2024-09-19T11:25:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55ed42800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42aaff0 0x5576a42ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55ed42800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5576a408d4a0 con 0x5576a42aaff0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55b287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42aaff0 0x5576a42ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55b287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42aaff0 0x5576a42ab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45204/0 (socket says 172.21.15.28:45204) 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55b287640 1 -- 172.21.15.28:0/2875983319 learned_addr learned my addr 172.21.15.28:0/2875983319 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.011+0000 7ff55b287640 1 -- 172.21.15.28:0/2875983319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a42aba50 con 0x5576a42aaff0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.012+0000 7ff55b287640 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42aaff0 0x5576a42ab410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x5576a4290f20 tx=0x7ff5540167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.012+0000 7ff444e01640 1 -- 172.21.15.28:0/2875983319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff554013640 con 0x5576a42aaff0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.012+0000 7ff444e01640 1 -- 172.21.15.28:0/2875983319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5540137e0 con 0x5576a42aaff0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.012+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576a42ac280 con 0x5576a42aaff0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.012+0000 7ff444e01640 1 -- 172.21.15.28:0/2875983319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff554013b00 con 0x5576a42aaff0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.012+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a42ac820 con 0x5576a42aaff0 2024-09-19T11:25:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.013+0000 7ff444e01640 1 -- 172.21.15.28:0/2875983319 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff554011040 con 0x5576a42aaff0 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.013+0000 7ff444e01640 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff43403d000 0x7ff43403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.013+0000 7ff444e01640 1 -- 172.21.15.28:0/2875983319 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff55400e040 con 0x5576a42aaff0 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.013+0000 7ff55aa86640 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff43403d000 0x7ff43403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.013+0000 7ff55aa86640 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff43403d000 0x7ff43403f4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c720 tx=0x7ff54c052920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.013+0000 7ff55ed42800 1 Processor -- start 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.013+0000 7ff55ed42800 1 -- start start 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff55ed42800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff55ed42800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5576a41950d0 con 0x5576a42b7a30 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff55b287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff55b287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45206/0 (socket says 172.21.15.28:45206) 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff55b287640 1 -- 172.21.15.28:0/769045556 learned_addr learned my addr 172.21.15.28:0/769045556 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff55b287640 1 -- 172.21.15.28:0/769045556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a42b8420 con 0x5576a42b7a30 2024-09-19T11:25:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff55b287640 1 --2- 172.21.15.28:0/769045556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42b7e30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff554016cd0 tx=0x7ff55401f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9430ce2f4fa04bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff43e7fc640 1 -- 172.21.15.28:0/769045556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff554051030 con 0x5576a42b7a30 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff43e7fc640 1 -- 172.21.15.28:0/769045556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55404de50 con 0x5576a42b7a30 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.014+0000 7ff43e7fc640 1 -- 172.21.15.28:0/769045556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff55402aa40 con 0x5576a42b7a30 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 -- 172.21.15.28:0/769045556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 msgr2=0x5576a42b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 --2- 172.21.15.28:0/769045556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42b7e30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff554016cd0 tx=0x7ff55401f400 comp rx=0 tx=0).stop 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 -- 172.21.15.28:0/769045556 shutdown_connections 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 --2- 172.21.15.28:0/769045556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42b7e30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 -- 172.21.15.28:0/769045556 >> 172.21.15.28:0/769045556 conn(0x5576a42b3e30 msgr2=0x5576a42b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 -- 172.21.15.28:0/769045556 shutdown_connections 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 -- 172.21.15.28:0/769045556 wait complete. 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 Processor -- start 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.015+0000 7ff55ed42800 1 -- start start 2024-09-19T11:25:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55ed42800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42bf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55ed42800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5576a42b8a40 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55b287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55b287640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42bf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45216/0 (socket says 172.21.15.28:45216) 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 learned_addr learned my addr 172.21.15.28:0/3319073337 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a42bf570 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55b287640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42bf010 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff554016cd0 tx=0x7ff5540547b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff43cff9640 1 -- 172.21.15.28:0/3319073337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff554051040 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff43cff9640 1 -- 172.21.15.28:0/3319073337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5540459f0 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576a42bf800 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff43cff9640 1 -- 172.21.15.28:0/3319073337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff554045cd0 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.016+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a42c0390 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.017+0000 7ff43cff9640 1 -- 172.21.15.28:0/3319073337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7ff5540451d0 con 0x5576a42b7a30 2024-09-19T11:25:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff43cff9640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff42403d000 0x7ff42403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff43cff9640 1 -- 172.21.15.28:0/3319073337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7ff55404c030 con 0x5576a42b7a30 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff55aa86640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff42403d000 0x7ff42403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff55aa86640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff42403d000 0x7ff42403f4c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff54c01af70 tx=0x7ff54c046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a42ca950 0x5576a42cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42cd560 con 0x5576a42ca950 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.018+0000 7ff55a285640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a42ca950 0x5576a42cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55a285640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a42ca950 0x5576a42cce10 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 20 realm 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42cd990 con 0x5576a42ca950 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.019+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42ce260 con 0x5576a42ca950 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55ed42800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42ce600 con 0x5576a42ca950 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55ed42800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a42cf360 0x5576a42d1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d1d60 con 0x5576a42cf360 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55b287640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a42cf360 0x5576a42d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.020+0000 7ff55b287640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a42cf360 0x5576a42d1750 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55ed42800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55ed42800 20 searching for the correct realm 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55b287640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff5540454c0 0x7ff554057610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55aa86640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff5540454c0 0x7ff554057610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.021+0000 7ff55aa86640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff5540454c0 0x7ff554057610 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.022+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.022+0000 7ff55aa86640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff54c017050 0x7ff54c04b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.022+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff54c017050 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.022+0000 7ff55a285640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff54c017050 0x7ff54c04b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.022+0000 7ff55a285640 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff54c017050 0x7ff54c04b7f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.022+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.022+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff54c017050 2024-09-19T11:25:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff54c017050 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.023+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff54c017050 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.024+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.025+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.026+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff54c017050 2024-09-19T11:25:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.027+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff54c017050 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff54c017050 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.028+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42cf360 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x5576a42ca950 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.029+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3690 con 0x7ff5540454c0 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:10.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d2720 con 0x5576a42ca950 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d2f70 con 0x5576a42ca950 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3b10 con 0x5576a42cf360 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 zone default found 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 4 Realm: () 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:10.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 zonegroup default 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.030+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d3f10 con 0x7ff54c017050 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d5d40 con 0x5576a42cf360 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d6200 con 0x5576a42cf360 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 20 started sync module instance, tier type = 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.031+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d51e0 con 0x7ff5540454c0 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.032+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.032+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93968048936448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d4e40 con 0x7ff5540454c0 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.032+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d7020 con 0x5576a42ca950 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.033+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93968048936448] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.033+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.033+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93968048940640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d7440 con 0x5576a42ca950 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.033+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d7e80 con 0x5576a42cf360 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.034+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93968048940640] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.035+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.035+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93968048945488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d7e40 con 0x5576a42cf360 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.035+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d89b0 con 0x7ff5540454c0 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.036+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93968048945488] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.036+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.036+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93968048948416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d8da0 con 0x7ff5540454c0 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.036+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d97c0 con 0x5576a42ca950 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.037+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93968048948416] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.037+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.037+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93968048951984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42d9be0 con 0x5576a42ca950 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.037+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42da600 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.038+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.038+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93968048951984] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.038+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93968048955632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42da5c0 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.038+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42db0f0 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.039+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 93968048955632] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.039+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.039+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93968048941696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42db0f0 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.039+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42dbab0 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 93968048941696] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93968048957680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42dbab0 con 0x7ff54c017050 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 20 add_watcher() i=0 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 20 add_watcher() i=1 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 20 add_watcher() i=2 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 20 add_watcher() i=3 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 20 add_watcher() i=4 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 20 add_watcher() i=5 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.040+0000 7ff55ed42800 20 add_watcher() i=6 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.041+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93968048957680] v33'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.041+0000 7ff55ed42800 20 add_watcher() i=7 2024-09-19T11:25:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.041+0000 7ff55ed42800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.041+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a42dede0 con 0x5576a42ca950 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.042+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.042+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93968048973840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a42dbc20 con 0x5576a42ca950 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93968048973840] v33'259 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff4117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 5 note: GC not initialized 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff38bfff640 20 reqs_thread_entry: start 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 20 Filter name: none 2024-09-19T11:25:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff389ffb640 20 reqs_thread_entry: start 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.043+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a4311cb0 con 0x7ff54c017050 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.044+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv6 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.044+0000 7ff55ed42800 20 rados_obj.operate() r=0 bl.length=490 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.044+0000 7ff55ed42800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.044+0000 7ff55ed42800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.044+0000 7ff55ed42800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.044+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a4318240 con 0x7ff54c017050 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v33'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff55ed42800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff55ed42800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff55ed42800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93968049208320 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5576a4313880 con 0x5576a42ca950 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 93968048951984 notify 141733920797 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7ff550081040 con 0x5576a42ca950 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff43e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920797 cookie 93968048951984 notifier 4879 bl.length()=666 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff43e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.045+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 93968049208320 out=8b] v0'0 uv261 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.046+0000 7ff43e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.046+0000 7ff43e7fc640 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7ff420003ea0 con 0x5576a42ca950 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.046+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.046+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 93968049208320 notify 141733920797 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5576a419e800 con 0x5576a42ca950 2024-09-19T11:25:10.063 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.047+0000 7ff4117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.047+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93968048973840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x5576a42ca950 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.048+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93968048973840] v33'260 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.048+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93968048936448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x7ff5540454c0 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.049+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93968048936448] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.049+0000 7ff55ed42800 20 remove_watcher() i=0 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.049+0000 7ff55ed42800 2 removed watcher, disabling cache 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.049+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93968048940640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x5576a42ca950 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.050+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93968048940640] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.050+0000 7ff55ed42800 20 remove_watcher() i=1 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.050+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93968048945488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x5576a42cf360 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.051+0000 7ff55b287640 1 -- 172.21.15.28:0/3319073337 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93968048945488] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff55403a070 con 0x5576a42cf360 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.051+0000 7ff55ed42800 20 remove_watcher() i=2 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.051+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93968048948416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x7ff5540454c0 2024-09-19T11:25:10.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.052+0000 7ff55aa86640 1 -- 172.21.15.28:0/3319073337 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93968048948416] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff54c0187b0 con 0x7ff5540454c0 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.052+0000 7ff55ed42800 20 remove_watcher() i=3 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.052+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93968048951984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x5576a42ca950 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.053+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93968048951984] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x5576a42ca950 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.053+0000 7ff55ed42800 20 remove_watcher() i=4 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.053+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93968048955632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x7ff54c017050 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.054+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93968048955632] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.054+0000 7ff55ed42800 20 remove_watcher() i=5 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.054+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93968048941696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x7ff54c017050 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.056+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93968048941696] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.056+0000 7ff55ed42800 20 remove_watcher() i=6 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.056+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93968048957680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5576a430fb40 con 0x7ff54c017050 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93968048957680] v33'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff550051070 con 0x7ff54c017050 2024-09-19T11:25:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 20 remove_watcher() i=7 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff54c017050 msgr2=0x7ff54c04b7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7ff54c017050 0x7ff54c04b7f0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a42ca950 msgr2=0x5576a42cce10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a42ca950 0x5576a42cce10 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff5540454c0 msgr2=0x7ff554057610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7ff5540454c0 0x7ff554057610 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a42cf360 msgr2=0x5576a42d1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a42cf360 0x5576a42d1750 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff42403d000 msgr2=0x7ff42403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff42403d000 0x7ff42403f4c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff54c01af70 tx=0x7ff54c046000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 msgr2=0x5576a42bf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42bf010 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff554016cd0 tx=0x7ff5540547b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55a285640 1 -- 172.21.15.28:0/3319073337 reap_dead start 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 shutdown_connections 2024-09-19T11:25:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/3319073337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42b7a30 0x5576a42bf010 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 >> 172.21.15.28:0/3319073337 conn(0x5576a42b3e30 msgr2=0x5576a42b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 shutdown_connections 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/3319073337 wait complete. 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff43403d000 msgr2=0x7ff43403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff43403d000 0x7ff43403f4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c720 tx=0x7ff54c052920 comp rx=0 tx=0).stop 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42aaff0 msgr2=0x5576a42ab410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42aaff0 0x5576a42ab410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x5576a4290f20 tx=0x7ff5540167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 shutdown_connections 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff43403d000 0x7ff43403f4c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2875983319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a42aaff0 0x5576a42ab410 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.057+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 >> 172.21.15.28:0/2875983319 conn(0x5576a429f3a0 msgr2=0x5576a429f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 shutdown_connections 2024-09-19T11:25:10.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2875983319 wait complete. 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a419aa90 msgr2=0x5576a42976e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a419aa90 0x5576a42976e0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a4299a80 msgr2=0x5576a429bf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a4299a80 0x5576a429bf20 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff44003d000 msgr2=0x7ff44003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff44003d000 0x7ff44003f4c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff550057000 tx=0x7ff550052000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 msgr2=0x5576a4290090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4290090 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c010 tx=0x7ff54c000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 shutdown_connections 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5576a4299a80 0x5576a429bf20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7ff44003d000 0x7ff44003f4c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5576a419aa90 0x5576a42976e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 --2- 172.21.15.28:0/2334993587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5576a4196f20 0x5576a4290090 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 >> 172.21.15.28:0/2334993587 conn(0x5576a4190a60 msgr2=0x5576a4281870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 shutdown_connections 2024-09-19T11:25:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.058+0000 7ff55ed42800 1 -- 172.21.15.28:0/2334993587 wait complete. 2024-09-19T11:25:10.074 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.074 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.074 ERROR:root:0 2024-09-19T11:25:10.074 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:10.087 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:10.087 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:10.087 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:10.088 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-09-19T11:25:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:10.135 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.134+0000 7f886ed2e800 1 Processor -- start 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.134+0000 7f886ed2e800 1 -- start start 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886ed2e800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5653c77866a0 0x5653c77873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886ed2e800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5653c7786d50 con 0x5653c77878e0 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886ed2e800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5653c76127d0 con 0x5653c77866a0 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886a964640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5653c77866a0 0x5653c77873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886a163640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5653c77878e0 0x5653c778c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48338/0 (socket says 172.21.15.28:48338) 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886a964640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5653c77866a0 0x5653c77873a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45228/0 (socket says 172.21.15.28:45228) 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886a163640 1 -- 172.21.15.28:0/2361756018 learned_addr learned my addr 172.21.15.28:0/2361756018 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:10.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886a964640 1 -- 172.21.15.28:0/2361756018 >> v1:172.21.15.22:6789/0 conn(0x5653c77878e0 legacy=0x5653c778c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f886a964640 1 -- 172.21.15.28:0/2361756018 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c7787d90 con 0x5653c77866a0 2024-09-19T11:25:10.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.135+0000 7f8869962640 1 -- 172.21.15.28:0/2361756018 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2224671703 0 0) 0x5653c7786d50 con 0x5653c77878e0 2024-09-19T11:25:10.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.137+0000 7f886a964640 1 --2- 172.21.15.28:0/2361756018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c77873a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f885c00a470 tx=0x7f885c045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9d46ad0ca34ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.137+0000 7f8869962640 1 -- 172.21.15.28:0/2361756018 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f885c053570 con 0x5653c77866a0 2024-09-19T11:25:10.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.137+0000 7f8869962640 1 -- 172.21.15.28:0/2361756018 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885c058070 con 0x5653c77866a0 2024-09-19T11:25:10.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.137+0000 7f8869962640 1 -- 172.21.15.28:0/2361756018 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f885c053880 con 0x5653c77866a0 2024-09-19T11:25:10.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.137+0000 7f886ed2e800 1 -- 172.21.15.28:0/2361756018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 msgr2=0x5653c77873a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.137+0000 7f886ed2e800 1 --2- 172.21.15.28:0/2361756018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c77873a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f885c00a470 tx=0x7f885c045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:10.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.138+0000 7f886ed2e800 1 -- 172.21.15.28:0/2361756018 shutdown_connections 2024-09-19T11:25:10.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.138+0000 7f886ed2e800 1 --2- 172.21.15.28:0/2361756018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c77873a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.138+0000 7f886ed2e800 1 -- 172.21.15.28:0/2361756018 >> 172.21.15.28:0/2361756018 conn(0x5653c7781a70 msgr2=0x5653c7783ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.138+0000 7f886ed2e800 1 -- 172.21.15.28:0/2361756018 shutdown_connections 2024-09-19T11:25:10.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.138+0000 7f886ed2e800 1 -- 172.21.15.28:0/2361756018 wait complete. 2024-09-19T11:25:10.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.153+0000 7f886ed2e800 1 Processor -- start 2024-09-19T11:25:10.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.153+0000 7f886ed2e800 1 -- start start 2024-09-19T11:25:10.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.153+0000 7f886ed2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7874fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.153+0000 7f886ed2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653c761fdd0 con 0x5653c77866a0 2024-09-19T11:25:10.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.153+0000 7f886a964640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7874fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.153+0000 7f886a964640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7874fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45234/0 (socket says 172.21.15.28:45234) 2024-09-19T11:25:10.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.153+0000 7f886a964640 1 -- 172.21.15.28:0/2895826261 learned_addr learned my addr 172.21.15.28:0/2895826261 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.154+0000 7f886a964640 1 -- 172.21.15.28:0/2895826261 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c78755c0 con 0x5653c77866a0 2024-09-19T11:25:10.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.154+0000 7f886a964640 1 --2- 172.21.15.28:0/2895826261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7874fd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f885c005870 tx=0x7f885c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e5aa64389758a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.154+0000 7f8758e09640 1 -- 172.21.15.28:0/2895826261 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f885c05c580 con 0x5653c77866a0 2024-09-19T11:25:10.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.154+0000 7f8758e09640 1 -- 172.21.15.28:0/2895826261 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885c063030 con 0x5653c77866a0 2024-09-19T11:25:10.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.154+0000 7f8758e09640 1 -- 172.21.15.28:0/2895826261 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f885c05c850 con 0x5653c77866a0 2024-09-19T11:25:10.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 -- 172.21.15.28:0/2895826261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 msgr2=0x5653c7874fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 --2- 172.21.15.28:0/2895826261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7874fd0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f885c005870 tx=0x7f885c051000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 -- 172.21.15.28:0/2895826261 shutdown_connections 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 --2- 172.21.15.28:0/2895826261 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7874fd0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 -- 172.21.15.28:0/2895826261 >> 172.21.15.28:0/2895826261 conn(0x5653c7781a70 msgr2=0x5653c7870950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 -- 172.21.15.28:0/2895826261 shutdown_connections 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 -- 172.21.15.28:0/2895826261 wait complete. 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 Processor -- start 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 -- start start 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7880e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886ed2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653c76f9ec0 con 0x5653c77866a0 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886a964640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7880e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886a964640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7880e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45240/0 (socket says 172.21.15.28:45240) 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.155+0000 7f886a964640 1 -- 172.21.15.28:0/4168957463 learned_addr learned my addr 172.21.15.28:0/4168957463 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.156+0000 7f886a964640 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c7881370 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.156+0000 7f886a964640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7880e10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f885c045f50 tx=0x7f885c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.156+0000 7f8757606640 1 -- 172.21.15.28:0/4168957463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f885c078070 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.156+0000 7f8757606640 1 -- 172.21.15.28:0/4168957463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f885c054a70 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.156+0000 7f8757606640 1 -- 172.21.15.28:0/4168957463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f885c054d50 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.156+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653c7881600 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.156+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c7882210 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.157+0000 7f8757606640 1 -- 172.21.15.28:0/4168957463 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f885c07b070 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.157+0000 7f8757606640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875003d000 0x7f875003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.157+0000 7f8757606640 1 -- 172.21.15.28:0/4168957463 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f885c070070 con 0x5653c77866a0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.157+0000 7f886a163640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875003d000 0x7f875003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.157+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c77878e0 0x5653c7888460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.157+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78889a0 con 0x5653c77878e0 2024-09-19T11:25:10.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.157+0000 7f886b165640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c77878e0 0x5653c7888460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886a163640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875003d000 0x7f875003f4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886b165640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c77878e0 0x5653c7888460 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886b165640 1 -- 172.21.15.28:0/4168957463 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c77878e0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c7888d40 con 0x5653c77878e0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886b165640 1 -- 172.21.15.28:0/4168957463 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c77878e0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c788a840 0x5653c788ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c788d440 con 0x5653c788a840 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.158+0000 7f886a964640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c788a840 0x5653c788ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.159+0000 7f886a964640 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c788a840 0x5653c788ccb0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.159+0000 7f886a964640 1 -- 172.21.15.28:0/4168957463 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f885c07a030 con 0x5653c788a840 2024-09-19T11:25:10.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.159+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c7889590 con 0x5653c77878e0 2024-09-19T11:25:10.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.159+0000 7f886b165640 1 -- 172.21.15.28:0/4168957463 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c77878e0 2024-09-19T11:25:10.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.159+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78899e0 con 0x5653c788a840 2024-09-19T11:25:10.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886a964640 1 -- 172.21.15.28:0/4168957463 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f885c07a030 con 0x5653c788a840 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886ed2e800 1 Processor -- start 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886ed2e800 1 -- start start 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886ed2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c789bdd0 0x5653c789c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886ed2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653c767e4a0 con 0x5653c789bdd0 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886b165640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c789bdd0 0x5653c789c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886b165640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c789bdd0 0x5653c789c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45244/0 (socket says 172.21.15.28:45244) 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.160+0000 7f886b165640 1 -- 172.21.15.28:0/4252531835 learned_addr learned my addr 172.21.15.28:0/4252531835 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.161+0000 7f886b165640 1 -- 172.21.15.28:0/4252531835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c789c830 con 0x5653c789bdd0 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.161+0000 7f886b165640 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c789bdd0 0x5653c789c1f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x5653c7882430 tx=0x7f88640167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.161+0000 7f8754e01640 1 -- 172.21.15.28:0/4252531835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8864013640 con 0x5653c789bdd0 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.161+0000 7f8754e01640 1 -- 172.21.15.28:0/4252531835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88640137e0 con 0x5653c789bdd0 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.161+0000 7f8754e01640 1 -- 172.21.15.28:0/4252531835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8864013ae0 con 0x5653c789bdd0 2024-09-19T11:25:10.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.161+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653c789d060 con 0x5653c789bdd0 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.161+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c789d600 con 0x5653c789bdd0 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.162+0000 7f8754e01640 1 -- 172.21.15.28:0/4252531835 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8864011040 con 0x5653c789bdd0 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.162+0000 7f8754e01640 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.162+0000 7f8754e01640 1 -- 172.21.15.28:0/4252531835 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f886400e040 con 0x5653c789bdd0 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.162+0000 7f886a964640 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.162+0000 7f886ed2e800 1 Processor -- start 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.162+0000 7f886ed2e800 1 -- start start 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.162+0000 7f886a964640 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f885c096760 tx=0x7f885c054ef0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f886ed2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f886ed2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653c7875990 con 0x5653c78a8870 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f886b165640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f886b165640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78a8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45246/0 (socket says 172.21.15.28:45246) 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f886b165640 1 -- 172.21.15.28:0/2807597777 learned_addr learned my addr 172.21.15.28:0/2807597777 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f886b165640 1 -- 172.21.15.28:0/2807597777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c78a9240 con 0x5653c78a8870 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f886b165640 1 --2- 172.21.15.28:0/2807597777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78a8c50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f88640547f0 tx=0x7f886401fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b89ca4e985b5626 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f874e7fc640 1 -- 172.21.15.28:0/2807597777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8864051070 con 0x5653c78a8870 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f874e7fc640 1 -- 172.21.15.28:0/2807597777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8864053440 con 0x5653c78a8870 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.163+0000 7f874e7fc640 1 -- 172.21.15.28:0/2807597777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8864021ad0 con 0x5653c78a8870 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 -- 172.21.15.28:0/2807597777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 msgr2=0x5653c78a8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 --2- 172.21.15.28:0/2807597777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78a8c50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f88640547f0 tx=0x7f886401fe70 comp rx=0 tx=0).stop 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 -- 172.21.15.28:0/2807597777 shutdown_connections 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 --2- 172.21.15.28:0/2807597777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78a8c50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 -- 172.21.15.28:0/2807597777 >> 172.21.15.28:0/2807597777 conn(0x5653c78a4c70 msgr2=0x5653c78a2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 -- 172.21.15.28:0/2807597777 shutdown_connections 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 -- 172.21.15.28:0/2807597777 wait complete. 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 Processor -- start 2024-09-19T11:25:10.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 -- start start 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886ed2e800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5653c78a9880 con 0x5653c78a8870 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886b165640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886b165640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45260/0 (socket says 172.21.15.28:45260) 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.164+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 learned_addr learned my addr 172.21.15.28:0/1181253915 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.165+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c78b0400 con 0x5653c78a8870 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.165+0000 7f886b165640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78afea0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8864022ff0 tx=0x7f886401fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.165+0000 7f874cff9640 1 -- 172.21.15.28:0/1181253915 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8864021440 con 0x5653c78a8870 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.165+0000 7f874cff9640 1 -- 172.21.15.28:0/1181253915 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8864021620 con 0x5653c78a8870 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.165+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653c78b0690 con 0x5653c78a8870 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.165+0000 7f874cff9640 1 -- 172.21.15.28:0/1181253915 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8864045b30 con 0x5653c78a8870 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.165+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c78b0690 con 0x5653c78a8870 2024-09-19T11:25:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.166+0000 7f874cff9640 1 -- 172.21.15.28:0/1181253915 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f8864045cd0 con 0x5653c78a8870 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.166+0000 7f874cff9640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f873403d000 0x7f873403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.166+0000 7f874cff9640 1 -- 172.21.15.28:0/1181253915 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f8864034d50 con 0x5653c78a8870 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.166+0000 7f886a964640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f873403d000 0x7f873403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886a964640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f873403d000 0x7f873403f4c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f885c098390 tx=0x7f885c091000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c78bb5a0 0x5653c78bda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78be1b0 con 0x5653c78bb5a0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886a163640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c78bb5a0 0x5653c78bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886a163640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c78bb5a0 0x5653c78bda60 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f886000f5f0 con 0x5653c78bb5a0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886ed2e800 20 realm 2024-09-19T11:25:10.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.167+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78ba8f0 con 0x5653c78bb5a0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f886000f5f0 con 0x5653c78bb5a0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78beca0 con 0x5653c78bb5a0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f886000f5f0 con 0x5653c78bb5a0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78bf570 con 0x5653c78bb5a0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f886000f5f0 con 0x5653c78bb5a0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c78bff90 0x5653c78c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c2af0 con 0x5653c78bff90 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.168+0000 7f886b165640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c78bff90 0x5653c78c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886b165640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c78bff90 0x5653c78c2380 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886ed2e800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886ed2e800 20 searching for the correct realm 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886b165640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8864045130 0x7f8864047960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.169+0000 7f886a964640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8864045130 0x7f8864047960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.170+0000 7f886a964640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8864045130 0x7f8864047960 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.170+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.170+0000 7f886a964640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f885c0a6040 0x7f885c009410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.170+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f885c0a6040 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.170+0000 7f886a163640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f885c0a6040 0x7f885c009410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.170+0000 7f886a163640 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f885c0a6040 0x7f885c009410 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886000f5f0 con 0x7f885c0a6040 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886000f5f0 con 0x5653c78bb5a0 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f885c0a6040 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x7f885c0a6040 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.171+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f885c0a6040 2024-09-19T11:25:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x7f885c0a6040 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.172+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f885c0a6040 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x7f885c0a6040 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.173+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.174+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f885c0a6040 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x7f885c0a6040 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.175+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.176+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.176+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f885c0a6040 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.176+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x7f885c0a6040 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.176+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.176+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.176+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f885c0a6040 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886002b040 con 0x7f885c0a6040 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bff90 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.177+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x5653c78bb5a0 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4420 con 0x7f8864045130 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c34b0 con 0x5653c78bb5a0 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c3d50 con 0x5653c78bb5a0 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f886002b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.178+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c48a0 con 0x5653c78bff90 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 zone default found 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 4 Realm: () 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 zonegroup default 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c4c40 con 0x7f885c0a6040 2024-09-19T11:25:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f886002b040 con 0x7f885c0a6040 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c6f00 con 0x5653c78bff90 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.179+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c72a0 con 0x5653c78bff90 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.180+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.180+0000 7f886ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.180+0000 7f886ed2e800 20 started sync module instance, tier type = 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.180+0000 7f886ed2e800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.180+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c6390 con 0x7f8864045130 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.181+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.181+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94917830141872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c7b60 con 0x7f8864045130 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.181+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c83b0 con 0x5653c78bb5a0 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.182+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94917830141872] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.182+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.182+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94917830145712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c83f0 con 0x5653c78bb5a0 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.182+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c8ee0 con 0x5653c78bff90 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.183+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94917830145712] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.183+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.183+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94917830150576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c8ea0 con 0x5653c78bff90 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.183+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c9970 con 0x7f8864045130 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.184+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94917830150576] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.184+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.184+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94917830153280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78c9d60 con 0x7f8864045130 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.184+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78ca7a0 con 0x5653c78bb5a0 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.186+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94917830153280] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.186+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.186+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94917830156912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78ca760 con 0x5653c78bb5a0 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.186+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cb230 con 0x7f885c0a6040 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.187+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94917830156912] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.187+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.187+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94917830159616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cb1f0 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.187+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cbc80 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.188+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94917830159616] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.188+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.188+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94917830147840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cbc40 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.188+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cc690 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94917830147840] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94917830162800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cc690 con 0x7f885c0a6040 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 20 add_watcher() i=0 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 20 add_watcher() i=1 2024-09-19T11:25:10.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 20 add_watcher() i=2 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 20 add_watcher() i=3 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 20 add_watcher() i=4 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 20 add_watcher() i=5 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.189+0000 7f886ed2e800 20 add_watcher() i=6 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.190+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94917830162800] v33'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.190+0000 7f886ed2e800 20 add_watcher() i=7 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.190+0000 7f886ed2e800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.190+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cfa00 con 0x5653c78bb5a0 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.190+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.190+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94917830177776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c78cc800 con 0x5653c78bb5a0 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.191+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94917830177776] v33'261 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.191+0000 7f87217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 5 note: GC not initialized 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f869bfff640 20 reqs_thread_entry: start 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 20 Filter name: none 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f8699ffb640 20 reqs_thread_entry: start 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.192+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c7902920 con 0x7f885c0a6040 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.193+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.193+0000 7f886ed2e800 20 rados_obj.operate() r=0 bl.length=490 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.193+0000 7f886ed2e800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.193+0000 7f886ed2e800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.193+0000 7f886ed2e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.193+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~503 in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7908ec0 con 0x7f885c0a6040 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~503] v33'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f886ed2e800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f886ed2e800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f886ed2e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94917830412480 in=691b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5653c7904520 con 0x5653c78bb5a0 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 94917830156912 notify 141733920798 ret 0) ==== 721+0+0 (crc 0 0 0) 0x7f8860018040 con 0x5653c78bb5a0 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f874e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920798 cookie 94917830156912 notifier 4886 bl.length()=679 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f874e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 94917830412480 out=8b] v0'0 uv264 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f874e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.194+0000 7f874e7fc640 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f8730003ea0 con 0x5653c78bb5a0 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.195+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.195+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 94917830412480 notify 141733920798 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f886001db70 con 0x5653c78bb5a0 2024-09-19T11:25:10.208 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:10.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.196+0000 7f87217e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.196+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94917830177776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x5653c78bb5a0 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.197+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94917830177776] v33'262 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.197+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94917830141872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x7f8864045130 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.198+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94917830141872] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.198+0000 7f886ed2e800 20 remove_watcher() i=0 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.198+0000 7f886ed2e800 2 removed watcher, disabling cache 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.198+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94917830145712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x5653c78bb5a0 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.199+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94917830145712] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.199+0000 7f886ed2e800 20 remove_watcher() i=1 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.199+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94917830150576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x5653c78bff90 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.200+0000 7f886b165640 1 -- 172.21.15.28:0/1181253915 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94917830150576] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886400a4d0 con 0x5653c78bff90 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.200+0000 7f886ed2e800 20 remove_watcher() i=2 2024-09-19T11:25:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.200+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94917830153280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x7f8864045130 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.201+0000 7f886a964640 1 -- 172.21.15.28:0/1181253915 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94917830153280] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f885c07a030 con 0x7f8864045130 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.201+0000 7f886ed2e800 20 remove_watcher() i=3 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.201+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94917830156912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x5653c78bb5a0 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.202+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94917830156912] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x5653c78bb5a0 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.202+0000 7f886ed2e800 20 remove_watcher() i=4 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.202+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94917830159616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x7f885c0a6040 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.203+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94917830159616] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.203+0000 7f886ed2e800 20 remove_watcher() i=5 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.203+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94917830147840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x7f885c0a6040 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.204+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94917830147840] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.204+0000 7f886ed2e800 20 remove_watcher() i=6 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.204+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94917830162800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5653c7900770 con 0x7f885c0a6040 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94917830162800] v33'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f886003b040 con 0x7f885c0a6040 2024-09-19T11:25:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 20 remove_watcher() i=7 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f885c0a6040 msgr2=0x7f885c009410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f885c0a6040 0x7f885c009410 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c78bb5a0 msgr2=0x5653c78bda60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c78bb5a0 0x5653c78bda60 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8864045130 msgr2=0x7f8864047960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f8864045130 0x7f8864047960 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c78bff90 msgr2=0x5653c78c2380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.205+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c78bff90 0x5653c78c2380 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f873403d000 msgr2=0x7f873403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f873403d000 0x7f873403f4c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f885c098390 tx=0x7f885c091000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 msgr2=0x5653c78afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/1181253915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c78a8870 0x5653c78afea0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8864022ff0 tx=0x7f886401fee0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 reap_dead start 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886a163640 1 -- 172.21.15.28:0/1181253915 reap_dead start 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 shutdown_connections 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 >> 172.21.15.28:0/1181253915 conn(0x5653c78a4c70 msgr2=0x5653c78a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 shutdown_connections 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/1181253915 wait complete. 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 msgr2=0x7f874403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f885c096760 tx=0x7f885c054ef0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c789bdd0 msgr2=0x5653c789c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c789bdd0 0x5653c789c1f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x5653c7882430 tx=0x7f88640167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 shutdown_connections 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f874403d000 0x7f874403f4c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4252531835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c789bdd0 0x5653c789c1f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 >> 172.21.15.28:0/4252531835 conn(0x5653c7890180 msgr2=0x5653c7890560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 shutdown_connections 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4252531835 wait complete. 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c77878e0 msgr2=0x5653c7888460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c77878e0 0x5653c7888460 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c788a840 msgr2=0x5653c788ccb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.206+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c788a840 0x5653c788ccb0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875003d000 msgr2=0x7f875003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875003d000 0x7f875003f4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600013b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 msgr2=0x5653c7880e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7880e10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f885c045f50 tx=0x7f885c002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 shutdown_connections 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5653c788a840 0x5653c788ccb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f875003d000 0x7f875003f4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5653c77878e0 0x5653c7888460 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 --2- 172.21.15.28:0/4168957463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5653c77866a0 0x5653c7880e10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 >> 172.21.15.28:0/4168957463 conn(0x5653c7781a70 msgr2=0x5653c786f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 shutdown_connections 2024-09-19T11:25:10.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.207+0000 7f886ed2e800 1 -- 172.21.15.28:0/4168957463 wait complete. 2024-09-19T11:25:10.220 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.220 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.220 ERROR:root:0 2024-09-19T11:25:10.220 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:10.233 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:10.233 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-09-19T11:25:10.233 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-09-19T11:25:10.233 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write; user-info-without-keys=*' 2024-09-19T11:25:10.281 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:10.281 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.280+0000 7fed20d2a800 1 Processor -- start 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.280+0000 7fed20d2a800 1 -- start start 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed20d2a800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f92ef0c1b0 0x55f92ef14300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed20d2a800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f92ef14840 con 0x55f92ef0bdb0 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed20d2a800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55f92ed967d0 con 0x55f92ef0c1b0 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed1c216640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55f92ef0c1b0 0x55f92ef14300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed1ca17640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55f92ef0bdb0 0x55f92ef10bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48348/0 (socket says 172.21.15.28:48348) 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed1ca17640 1 -- 172.21.15.28:0/3567281997 learned_addr learned my addr 172.21.15.28:0/3567281997 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed1ba15640 1 -- 172.21.15.28:0/3567281997 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258111505 0 0) 0x55f92ef14840 con 0x55f92ef0bdb0 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed1ba15640 1 -- 172.21.15.28:0/3567281997 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed04001e10 con 0x55f92ef0bdb0 2024-09-19T11:25:10.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed1c216640 1 -- 172.21.15.28:0/3567281997 >> v1:172.21.15.22:6789/0 conn(0x55f92ef0bdb0 legacy=0x55f92ef10bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.281+0000 7fed1c216640 1 -- 172.21.15.28:0/3567281997 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92ef15a20 con 0x55f92ef0c1b0 2024-09-19T11:25:10.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.283+0000 7fed1c216640 1 --2- 172.21.15.28:0/3567281997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0c1b0 0x55f92ef14300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fed1000a470 tx=0x7fed10045920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca9d1376f29b5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.283+0000 7fed1ba15640 1 -- 172.21.15.28:0/3567281997 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed10059070 con 0x55f92ef0c1b0 2024-09-19T11:25:10.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.283+0000 7fed1ba15640 1 -- 172.21.15.28:0/3567281997 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed10054570 con 0x55f92ef0c1b0 2024-09-19T11:25:10.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.283+0000 7fed1ba15640 1 -- 172.21.15.28:0/3567281997 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed10054870 con 0x55f92ef0c1b0 2024-09-19T11:25:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.284+0000 7fed20d2a800 1 -- 172.21.15.28:0/3567281997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0c1b0 msgr2=0x55f92ef14300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.284+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3567281997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0c1b0 0x55f92ef14300 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fed1000a470 tx=0x7fed10045920 comp rx=0 tx=0).stop 2024-09-19T11:25:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.284+0000 7fed20d2a800 1 -- 172.21.15.28:0/3567281997 shutdown_connections 2024-09-19T11:25:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.284+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3567281997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0c1b0 0x55f92ef14300 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.284+0000 7fed20d2a800 1 -- 172.21.15.28:0/3567281997 >> 172.21.15.28:0/3567281997 conn(0x55f92ef051d0 msgr2=0x55f92ef07650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.284+0000 7fed20d2a800 1 -- 172.21.15.28:0/3567281997 shutdown_connections 2024-09-19T11:25:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.284+0000 7fed20d2a800 1 -- 172.21.15.28:0/3567281997 wait complete. 2024-09-19T11:25:10.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.299+0000 7fed20d2a800 1 Processor -- start 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed20d2a800 1 -- start start 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed20d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92effd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed20d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f92eda3dd0 con 0x55f92ef0bdb0 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed1ca17640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92effd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed1ca17640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92effd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45278/0 (socket says 172.21.15.28:45278) 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed1ca17640 1 -- 172.21.15.28:0/2941293942 learned_addr learned my addr 172.21.15.28:0/2941293942 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed1ca17640 1 -- 172.21.15.28:0/2941293942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92effda30 con 0x55f92ef0bdb0 2024-09-19T11:25:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.300+0000 7fed1ca17640 1 --2- 172.21.15.28:0/2941293942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92effd440 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0016c0 tx=0x7fed0c001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340c98a2fdc040ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fec0ae0d640 1 -- 172.21.15.28:0/2941293942 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed0c004520 con 0x55f92ef0bdb0 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fec0ae0d640 1 -- 172.21.15.28:0/2941293942 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c0046c0 con 0x55f92ef0bdb0 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fec0ae0d640 1 -- 172.21.15.28:0/2941293942 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed0c0049c0 con 0x55f92ef0bdb0 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fed20d2a800 1 -- 172.21.15.28:0/2941293942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 msgr2=0x55f92effd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2941293942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92effd440 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0016c0 tx=0x7fed0c001b80 comp rx=0 tx=0).stop 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fed20d2a800 1 -- 172.21.15.28:0/2941293942 shutdown_connections 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2941293942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92effd440 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fed20d2a800 1 -- 172.21.15.28:0/2941293942 >> 172.21.15.28:0/2941293942 conn(0x55f92ef051d0 msgr2=0x55f92eff8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fed20d2a800 1 -- 172.21.15.28:0/2941293942 shutdown_connections 2024-09-19T11:25:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.301+0000 7fed20d2a800 1 -- 172.21.15.28:0/2941293942 wait complete. 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.302+0000 7fed20d2a800 1 Processor -- start 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.302+0000 7fed20d2a800 1 -- start start 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.302+0000 7fed20d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92f004f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.302+0000 7fed20d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f92ef14d10 con 0x55f92ef0bdb0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.302+0000 7fed1ca17640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92f004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.302+0000 7fed1ca17640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92f004f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45290/0 (socket says 172.21.15.28:45290) 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.302+0000 7fed1ca17640 1 -- 172.21.15.28:0/2537418242 learned_addr learned my addr 172.21.15.28:0/2537418242 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.303+0000 7fed1ca17640 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92f0054c0 con 0x55f92ef0bdb0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.303+0000 7fed1ca17640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92f004f60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007fd0 tx=0x7fed0c004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.303+0000 7fec0960a640 1 -- 172.21.15.28:0/2537418242 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed0c022020 con 0x55f92ef0bdb0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.303+0000 7fec0960a640 1 -- 172.21.15.28:0/2537418242 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c019040 con 0x55f92ef0bdb0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.303+0000 7fec0960a640 1 -- 172.21.15.28:0/2537418242 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed0c014a20 con 0x55f92ef0bdb0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.303+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f92f005750 con 0x55f92ef0bdb0 2024-09-19T11:25:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.303+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92f006150 con 0x55f92ef0bdb0 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.304+0000 7fec0960a640 1 -- 172.21.15.28:0/2537418242 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fed0c025020 con 0x55f92ef0bdb0 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.304+0000 7fec0960a640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fec0403cfb0 0x7fec0403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.304+0000 7fec0960a640 1 -- 172.21.15.28:0/2537418242 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fed0c0545f0 con 0x55f92ef0bdb0 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed1c216640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fec0403cfb0 0x7fec0403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92ef0c1b0 0x55f92f00c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f00caf0 con 0x55f92ef0c1b0 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed1d218640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92ef0c1b0 0x55f92f00c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed1c216640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fec0403cfb0 0x7fec0403f470 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55f92ee6eba0 tx=0x7fed10050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed1d218640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92ef0c1b0 0x55f92f00c5b0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed1d218640 1 -- 172.21.15.28:0/2537418242 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fed140083e0 con 0x55f92ef0c1b0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.305+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f00ce90 con 0x55f92ef0c1b0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.306+0000 7fed1d218640 1 -- 172.21.15.28:0/2537418242 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fed140083e0 con 0x55f92ef0c1b0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.306+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f00e950 0x55f92f010df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.306+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f011580 con 0x55f92f00e950 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.306+0000 7fed1ca17640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f00e950 0x55f92f010df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.306+0000 7fed1ca17640 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f00e950 0x55f92f010df0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.306+0000 7fed1ca17640 1 -- 172.21.15.28:0/2537418242 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fed0c00f030 con 0x55f92f00e950 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.307+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f00d6e0 con 0x55f92ef0c1b0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.307+0000 7fed1d218640 1 -- 172.21.15.28:0/2537418242 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fed140083e0 con 0x55f92ef0c1b0 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.307+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f00da80 con 0x55f92f00e950 2024-09-19T11:25:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.307+0000 7fed1ca17640 1 -- 172.21.15.28:0/2537418242 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fed0c00f030 con 0x55f92f00e950 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.307+0000 7fed20d2a800 1 Processor -- start 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.307+0000 7fed20d2a800 1 -- start start 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fed20d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f01ff10 0x55f92f020330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fed20d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f92ee024a0 con 0x55f92f01ff10 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fed1d218640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f01ff10 0x55f92f020330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fed1d218640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f01ff10 0x55f92f020330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45298/0 (socket says 172.21.15.28:45298) 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fed1d218640 1 -- 172.21.15.28:0/4248380320 learned_addr learned my addr 172.21.15.28:0/4248380320 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fed1d218640 1 -- 172.21.15.28:0/4248380320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92f020970 con 0x55f92f01ff10 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fed1d218640 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f01ff10 0x55f92f020330 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f92f006e70 tx=0x7fed1400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.308+0000 7fec02ffd640 1 -- 172.21.15.28:0/4248380320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1401c020 con 0x55f92f01ff10 2024-09-19T11:25:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.309+0000 7fec02ffd640 1 -- 172.21.15.28:0/4248380320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed14007c90 con 0x55f92f01ff10 2024-09-19T11:25:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.309+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f92f0211a0 con 0x55f92f01ff10 2024-09-19T11:25:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.309+0000 7fec02ffd640 1 -- 172.21.15.28:0/4248380320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14018440 con 0x55f92f01ff10 2024-09-19T11:25:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.309+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92f021740 con 0x55f92f01ff10 2024-09-19T11:25:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.309+0000 7fec02ffd640 1 -- 172.21.15.28:0/4248380320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fed1401f020 con 0x55f92f01ff10 2024-09-19T11:25:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fec02ffd640 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febf403d020 0x7febf403f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fec02ffd640 1 -- 172.21.15.28:0/4248380320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fed14055bb0 con 0x55f92f01ff10 2024-09-19T11:25:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed1ca17640 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febf403d020 0x7febf403f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed20d2a800 1 Processor -- start 2024-09-19T11:25:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed1ca17640 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febf403d020 0x7febf403f4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed0c010750 tx=0x7fed0c001b80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed20d2a800 1 -- start start 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed20d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f02cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed20d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f92effde60 con 0x55f92f02c950 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed1d218640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f02cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed1d218640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f02cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45310/0 (socket says 172.21.15.28:45310) 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.310+0000 7fed1d218640 1 -- 172.21.15.28:0/502129789 learned_addr learned my addr 172.21.15.28:0/502129789 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed1d218640 1 -- 172.21.15.28:0/502129789 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92f02d340 con 0x55f92f02c950 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed1d218640 1 --2- 172.21.15.28:0/502129789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f02cd50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fed14053e30 tx=0x7fed14015970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb9c6fc3c8929df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fec00ff9640 1 -- 172.21.15.28:0/502129789 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14028d50 con 0x55f92f02c950 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fec00ff9640 1 -- 172.21.15.28:0/502129789 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed14051070 con 0x55f92f02c950 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fec00ff9640 1 -- 172.21.15.28:0/502129789 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed140174d0 con 0x55f92f02c950 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed20d2a800 1 -- 172.21.15.28:0/502129789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 msgr2=0x55f92f02cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed20d2a800 1 --2- 172.21.15.28:0/502129789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f02cd50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fed14053e30 tx=0x7fed14015970 comp rx=0 tx=0).stop 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed20d2a800 1 -- 172.21.15.28:0/502129789 shutdown_connections 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed20d2a800 1 --2- 172.21.15.28:0/502129789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f02cd50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed20d2a800 1 -- 172.21.15.28:0/502129789 >> 172.21.15.28:0/502129789 conn(0x55f92f028d70 msgr2=0x55f92f026fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed20d2a800 1 -- 172.21.15.28:0/502129789 shutdown_connections 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.311+0000 7fed20d2a800 1 -- 172.21.15.28:0/502129789 wait complete. 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed20d2a800 1 Processor -- start 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed20d2a800 1 -- start start 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed20d2a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f033f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed20d2a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f92f02d960 con 0x55f92f02c950 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed1d218640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f033f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed1d218640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f033f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45318/0 (socket says 172.21.15.28:45318) 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 learned_addr learned my addr 172.21.15.28:0/3054998519 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92f0344e0 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.312+0000 7fed1d218640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f033f80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fed14018b10 tx=0x7fed14053dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.313+0000 7febfa7fc640 1 -- 172.21.15.28:0/3054998519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed14028300 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.313+0000 7febfa7fc640 1 -- 172.21.15.28:0/3054998519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed1402e070 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.313+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f92f034770 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.313+0000 7febfa7fc640 1 -- 172.21.15.28:0/3054998519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed1400b030 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.313+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92f034770 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.313+0000 7febfa7fc640 1 -- 172.21.15.28:0/3054998519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fed14016040 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7febfa7fc640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febe403cfb0 0x7febe403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7febfa7fc640 1 -- 172.21.15.28:0/3054998519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fed14004280 con 0x55f92f02c950 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7fed1ca17640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febe403cfb0 0x7febe403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7fed1ca17640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febe403cfb0 0x7febe403f470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed0c014be0 tx=0x7fed0c050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92f03f630 0x55f92f041af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f042240 con 0x55f92f03f630 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.314+0000 7fed1c216640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92f03f630 0x55f92f041af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed1c216640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92f03f630 0x55f92f041af0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 20 realm 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f042670 con 0x55f92f03f630 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.315+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f042f40 con 0x55f92f03f630 2024-09-19T11:25:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed20d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f0432e0 con 0x55f92f03f630 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed20d2a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f044040 0x55f92f046430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f046a40 con 0x55f92f044040 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed1d218640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f044040 0x55f92f046430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed1d218640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f044040 0x55f92f046430 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.316+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed20d2a800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed20d2a800 20 searching for the correct realm 2024-09-19T11:25:10.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1d218640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fed14021020 0x7fed140349b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1ca17640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fed14021020 0x7fed140349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1ca17640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fed14021020 0x7fed140349b0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1ca17640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fed0c03d040 0x7fed0c04b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.317+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed0c03d040 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.318+0000 7fed1c216640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fed0c03d040 0x7fed0c04b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.318+0000 7fed1c216640 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fed0c03d040 0x7fed0c04b6d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.318+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.318+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed0c03d040 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed0c03d040 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.319+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed0c03d040 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.320+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.321+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.322+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed0c03d040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.323+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed0c03d040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed0c03d040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.324+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f044040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x55f92f03f630 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55f92f048370 con 0x7fed14021020 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f047400 con 0x55f92f03f630 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.325+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f047c50 con 0x55f92f03f630 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f047040 con 0x55f92f044040 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 zone default found 2024-09-19T11:25:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 4 Realm: () 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 zonegroup default 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f0487f0 con 0x7fed0c03d040 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.326+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04a600 con 0x55f92f044040 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed20d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04aa90 con 0x55f92f044040 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed20d2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed20d2a800 20 started sync module instance, tier type = 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed20d2a800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.327+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f049aa0 con 0x7fed14021020 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.328+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.328+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94528724047344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04aa90 con 0x7fed14021020 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.328+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04b2e0 con 0x55f92f03f630 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.329+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94528724047344] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.329+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.329+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94528724053552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04b700 con 0x55f92f03f630 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.329+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04c0c0 con 0x55f92f044040 2024-09-19T11:25:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.330+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94528724053552] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.330+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.330+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94528724055088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04c0c0 con 0x55f92f044040 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.330+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04cac0 con 0x7fed14021020 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.331+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94528724055088] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.332+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.332+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94528724057104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04c030 con 0x7fed14021020 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.332+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04d980 con 0x55f92f03f630 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.333+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.333+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94528724057104] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.333+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94528724063440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04d930 con 0x55f92f03f630 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.333+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04cef0 con 0x7fed0c03d040 2024-09-19T11:25:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.334+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94528724063440] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.334+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.334+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94528724065568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04d8f0 con 0x7fed0c03d040 2024-09-19T11:25:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.334+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04eec0 con 0x7fed0c03d040 2024-09-19T11:25:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.335+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(56 notify.6 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.335+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94528724068880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04f2e0 con 0x7fed0c03d040 2024-09-19T11:25:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.335+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04fce0 con 0x7fed0c03d040 2024-09-19T11:25:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.335+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94528724065568] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94528724068880] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94528724067312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f04ee30 con 0x7fed0c03d040 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 20 add_watcher() i=0 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 20 add_watcher() i=1 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 20 add_watcher() i=2 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 20 add_watcher() i=3 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 20 add_watcher() i=4 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 20 add_watcher() i=5 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.336+0000 7fed20d2a800 20 add_watcher() i=6 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.337+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94528724067312] v33'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.337+0000 7fed20d2a800 20 add_watcher() i=7 2024-09-19T11:25:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.337+0000 7fed20d2a800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.337+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f052de0 con 0x55f92f03f630 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.338+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.338+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94528724072432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f050260 con 0x55f92f03f630 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.339+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94528724072432] v33'263 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.339+0000 7febd37de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.339+0000 7fed20d2a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.339+0000 7fed20d2a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.339+0000 7fed20d2a800 5 note: GC not initialized 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.339+0000 7febd27dc640 20 reqs_thread_entry: start 2024-09-19T11:25:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.339+0000 7fed20d2a800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 20 Filter name: none 2024-09-19T11:25:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7feb43fff640 20 reqs_thread_entry: start 2024-09-19T11:25:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f085a10 con 0x7fed0c03d040 2024-09-19T11:25:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~503 out=503b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+626 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 20 rados_obj.operate() r=0 bl.length=503 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.340+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f08c270 con 0x7fed0c03d040 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v33'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed20d2a800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed20d2a800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed20d2a800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94528724319344 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f92f087950 con 0x55f92f03f630 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 94528724063440 notify 141733920799 ret 0) ==== 678+0+0 (crc 0 0 0) 0x55f92ef15a20 con 0x55f92f03f630 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fec00ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920799 cookie 94528724063440 notifier 4893 bl.length()=636 2024-09-19T11:25:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fec00ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fec00ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fec00ff9640 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7febe8003ea0 con 0x55f92f03f630 2024-09-19T11:25:10.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 94528724319344 out=8b] v0'0 uv267 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.342+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 94528724319344 notify 141733920799 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fed10082ba0 con 0x55f92f03f630 2024-09-19T11:25:10.353 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:10.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.343+0000 7febd37de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.343+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94528724072432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x55f92f03f630 2024-09-19T11:25:10.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.344+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94528724072432] v33'264 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.344+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94528724047344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x7fed14021020 2024-09-19T11:25:10.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.345+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94528724047344] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.345+0000 7fed20d2a800 20 remove_watcher() i=0 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.345+0000 7fed20d2a800 2 removed watcher, disabling cache 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.345+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94528724053552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x55f92f03f630 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.346+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94528724053552] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.346+0000 7fed20d2a800 20 remove_watcher() i=1 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.346+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94528724055088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x55f92f044040 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.347+0000 7fed1d218640 1 -- 172.21.15.28:0/3054998519 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94528724055088] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed140083e0 con 0x55f92f044040 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.347+0000 7fed20d2a800 20 remove_watcher() i=2 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.347+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94528724057104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x7fed14021020 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.348+0000 7fed1ca17640 1 -- 172.21.15.28:0/3054998519 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94528724057104] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed0c00f030 con 0x7fed14021020 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.348+0000 7fed20d2a800 20 remove_watcher() i=3 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.348+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94528724063440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x55f92f03f630 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.349+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94528724063440] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x55f92f03f630 2024-09-19T11:25:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.349+0000 7fed20d2a800 20 remove_watcher() i=4 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.349+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94528724065568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x7fed0c03d040 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.350+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94528724065568] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.350+0000 7fed20d2a800 20 remove_watcher() i=5 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.350+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94528724068880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x7fed0c03d040 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.351+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94528724068880] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.351+0000 7fed20d2a800 20 remove_watcher() i=6 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.351+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94528724067312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55f92f083830 con 0x7fed0c03d040 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94528724067312] v33'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fed1005c040 con 0x7fed0c03d040 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 20 remove_watcher() i=7 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fed0c03d040 msgr2=0x7fed0c04b6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fed0c03d040 0x7fed0c04b6d0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92f03f630 msgr2=0x55f92f041af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92f03f630 0x55f92f041af0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fed14021020 msgr2=0x7fed140349b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fed14021020 0x7fed140349b0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f044040 msgr2=0x55f92f046430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f044040 0x55f92f046430 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febe403cfb0 msgr2=0x7febe403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febe403cfb0 0x7febe403f470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed0c014be0 tx=0x7fed0c050000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 msgr2=0x55f92f033f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/3054998519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f02c950 0x55f92f033f80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fed14018b10 tx=0x7fed14053dc0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed1c216640 1 -- 172.21.15.28:0/3054998519 reap_dead start 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 shutdown_connections 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 >> 172.21.15.28:0/3054998519 conn(0x55f92f028d70 msgr2=0x55f92f026f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 shutdown_connections 2024-09-19T11:25:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/3054998519 wait complete. 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febf403d020 msgr2=0x7febf403f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febf403d020 0x7febf403f4e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed0c010750 tx=0x7fed0c001b80 comp rx=0 tx=0).stop 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f01ff10 msgr2=0x55f92f020330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f01ff10 0x55f92f020330 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f92f006e70 tx=0x7fed1400bf30 comp rx=0 tx=0).stop 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 shutdown_connections 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7febf403d020 0x7febf403f4e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 --2- 172.21.15.28:0/4248380320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92f01ff10 0x55f92f020330 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 >> 172.21.15.28:0/4248380320 conn(0x55f92f0142c0 msgr2=0x55f92f0146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 shutdown_connections 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.353+0000 7fed20d2a800 1 -- 172.21.15.28:0/4248380320 wait complete. 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92ef0c1b0 msgr2=0x55f92f00c5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92ef0c1b0 0x55f92f00c5b0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f00e950 msgr2=0x55f92f010df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f00e950 0x55f92f010df0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fec0403cfb0 msgr2=0x7fec0403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fec0403cfb0 0x7fec0403f470 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x55f92ee6eba0 tx=0x7fed10050000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 msgr2=0x55f92f004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92f004f60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fed0c007fd0 tx=0x7fed0c004460 comp rx=0 tx=0).stop 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 shutdown_connections 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55f92f00e950 0x55f92f010df0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fec0403cfb0 0x7fec0403f470 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55f92ef0c1b0 0x55f92f00c5b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 --2- 172.21.15.28:0/2537418242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f92ef0bdb0 0x55f92f004f60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 >> 172.21.15.28:0/2537418242 conn(0x55f92ef051d0 msgr2=0x55f92effe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 shutdown_connections 2024-09-19T11:25:10.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.354+0000 7fed20d2a800 1 -- 172.21.15.28:0/2537418242 wait complete. 2024-09-19T11:25:10.364 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.365 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.365 ERROR:root:0 2024-09-19T11:25:10.365 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-09-19T11:25:10.365 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-09-19T11:25:10.365 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=write 2024-09-19T11:25:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.412+0000 7fbd3a81b800 1 Processor -- start 2024-09-19T11:25:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.412+0000 7fbd3a81b800 1 -- start start 2024-09-19T11:25:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.412+0000 7fbd3a81b800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x557515a1feb0 0x557515a222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.412+0000 7fbd3a81b800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557515a265f0 con 0x557515a228d0 2024-09-19T11:25:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.412+0000 7fbd3a81b800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5575158aa7d0 con 0x557515a1feb0 2024-09-19T11:25:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.412+0000 7fbd35feb640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x557515a1feb0 0x557515a222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.413+0000 7fbd2d7ea640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x557515a228d0 0x557515a24d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48354/0 (socket says 172.21.15.28:48354) 2024-09-19T11:25:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.413+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/1444503658 learned_addr learned my addr 172.21.15.28:0/1444503658 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:25:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.413+0000 7fbd35feb640 1 -- 172.21.15.28:0/1444503658 >> v1:172.21.15.22:6789/0 conn(0x557515a228d0 legacy=0x557515a24d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.413+0000 7fbd35feb640 1 -- 172.21.15.28:0/1444503658 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557515a27820 con 0x557515a1feb0 2024-09-19T11:25:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.413+0000 7fbd357ea640 1 -- 172.21.15.28:0/1444503658 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4263614722 0 0) 0x557515a265f0 con 0x557515a228d0 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd35feb640 1 --2- 172.21.15.28:0/1444503658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515a222a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800a470 tx=0x7fbd28045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21801cd63dca7d85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd357ea640 1 -- 172.21.15.28:0/1444503658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd28053570 con 0x557515a1feb0 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd357ea640 1 -- 172.21.15.28:0/1444503658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28058070 con 0x557515a1feb0 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd357ea640 1 -- 172.21.15.28:0/1444503658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd28053880 con 0x557515a1feb0 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd3a81b800 1 -- 172.21.15.28:0/1444503658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 msgr2=0x557515a222a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/1444503658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515a222a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800a470 tx=0x7fbd28045a20 comp rx=0 tx=0).stop 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd3a81b800 1 -- 172.21.15.28:0/1444503658 shutdown_connections 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/1444503658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515a222a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd3a81b800 1 -- 172.21.15.28:0/1444503658 >> 172.21.15.28:0/1444503658 conn(0x557515a19a70 msgr2=0x557515a1bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd3a81b800 1 -- 172.21.15.28:0/1444503658 shutdown_connections 2024-09-19T11:25:10.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.415+0000 7fbd3a81b800 1 -- 172.21.15.28:0/1444503658 wait complete. 2024-09-19T11:25:10.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.430+0000 7fbd3a81b800 1 Processor -- start 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd3a81b800 1 -- start start 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd3a81b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b0f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd3a81b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5575158b7dd0 con 0x557515a1feb0 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd35feb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b0f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd35feb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b0f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45340/0 (socket says 172.21.15.28:45340) 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd35feb640 1 -- 172.21.15.28:0/3394987509 learned_addr learned my addr 172.21.15.28:0/3394987509 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd35feb640 1 -- 172.21.15.28:0/3394987509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557515b0f7e0 con 0x557515a1feb0 2024-09-19T11:25:10.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.431+0000 7fbd35feb640 1 --2- 172.21.15.28:0/3394987509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b0f1f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005870 tx=0x7fbd28051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d056f896ba65c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbc23e07640 1 -- 172.21.15.28:0/3394987509 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2805c580 con 0x557515a1feb0 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbc23e07640 1 -- 172.21.15.28:0/3394987509 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28063030 con 0x557515a1feb0 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbc23e07640 1 -- 172.21.15.28:0/3394987509 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2805c850 con 0x557515a1feb0 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3394987509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 msgr2=0x557515b0f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/3394987509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b0f1f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005870 tx=0x7fbd28051000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3394987509 shutdown_connections 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/3394987509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b0f1f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3394987509 >> 172.21.15.28:0/3394987509 conn(0x557515a19a70 msgr2=0x557515b0ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3394987509 shutdown_connections 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.432+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3394987509 wait complete. 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.433+0000 7fbd3a81b800 1 Processor -- start 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.433+0000 7fbd3a81b800 1 -- start start 2024-09-19T11:25:10.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.433+0000 7fbd3a81b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b16c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.433+0000 7fbd3a81b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557515a26ac0 con 0x557515a1feb0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.433+0000 7fbd35feb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b16c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.433+0000 7fbd35feb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b16c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45354/0 (socket says 172.21.15.28:45354) 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.433+0000 7fbd35feb640 1 -- 172.21.15.28:0/685567636 learned_addr learned my addr 172.21.15.28:0/685567636 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.434+0000 7fbd35feb640 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557515b17180 con 0x557515a1feb0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.434+0000 7fbd35feb640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b16c20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbd28045f50 tx=0x7fbd28002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.434+0000 7fbc22604640 1 -- 172.21.15.28:0/685567636 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd28078070 con 0x557515a1feb0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.434+0000 7fbc22604640 1 -- 172.21.15.28:0/685567636 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd28054a70 con 0x557515a1feb0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.434+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557515b17410 con 0x557515a1feb0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.434+0000 7fbc22604640 1 -- 172.21.15.28:0/685567636 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd28054d50 con 0x557515a1feb0 2024-09-19T11:25:10.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.434+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557515b18020 con 0x557515a1feb0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.435+0000 7fbc22604640 1 -- 172.21.15.28:0/685567636 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbd2807b070 con 0x557515a1feb0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.435+0000 7fbc22604640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1c03cfb0 0x7fbc1c03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.435+0000 7fbc22604640 1 -- 172.21.15.28:0/685567636 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbd28070070 con 0x557515a1feb0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.435+0000 7fbd2d7ea640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1c03cfb0 0x7fbc1c03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.435+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515a228d0 0x557515b1e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.435+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b1e7b0 con 0x557515a228d0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.435+0000 7fbd367ec640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515a228d0 0x557515b1e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd2d7ea640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1c03cfb0 0x7fbc1c03f470 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd20000c00 tx=0x7fbd200013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd367ec640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515a228d0 0x557515b1e270 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd367ec640 1 -- 172.21.15.28:0/685567636 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515a228d0 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b1eb50 con 0x557515a228d0 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd367ec640 1 -- 172.21.15.28:0/685567636 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515a228d0 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b20690 0x557515b22b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b232c0 con 0x557515b20690 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.436+0000 7fbd35feb640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b20690 0x557515b22b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.437+0000 7fbd35feb640 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b20690 0x557515b22b30 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.437+0000 7fbd35feb640 1 -- 172.21.15.28:0/685567636 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbd2807a030 con 0x557515b20690 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.437+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b1f3a0 con 0x557515a228d0 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.437+0000 7fbd367ec640 1 -- 172.21.15.28:0/685567636 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515a228d0 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.437+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b1f7f0 con 0x557515b20690 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.438+0000 7fbd35feb640 1 -- 172.21.15.28:0/685567636 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbd2807a030 con 0x557515b20690 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.438+0000 7fbd3a81b800 1 Processor -- start 2024-09-19T11:25:10.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.438+0000 7fbd3a81b800 1 -- start start 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.438+0000 7fbd3a81b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1f650 0x557515a1dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.438+0000 7fbd3a81b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5575159164a0 con 0x557515a1f650 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbd367ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1f650 0x557515a1dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbd367ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1f650 0x557515a1dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45360/0 (socket says 172.21.15.28:45360) 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbd367ec640 1 -- 172.21.15.28:0/2216568826 learned_addr learned my addr 172.21.15.28:0/2216568826 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbd367ec640 1 -- 172.21.15.28:0/2216568826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557515a1fb70 con 0x557515a1f650 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbd367ec640 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1f650 0x557515a1dec0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x557515b18240 tx=0x7fbd300167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbc1b7fe640 1 -- 172.21.15.28:0/2216568826 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30013640 con 0x557515a1f650 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbc1b7fe640 1 -- 172.21.15.28:0/2216568826 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd300137e0 con 0x557515a1f650 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbc1b7fe640 1 -- 172.21.15.28:0/2216568826 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30013ae0 con 0x557515a1f650 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557515a1e400 con 0x557515a1f650 2024-09-19T11:25:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.439+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557515a1e950 con 0x557515a1f650 2024-09-19T11:25:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.440+0000 7fbc1b7fe640 1 -- 172.21.15.28:0/2216568826 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbd30011040 con 0x557515a1f650 2024-09-19T11:25:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.440+0000 7fbc1b7fe640 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1003d000 0x7fbc1003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.440+0000 7fbc1b7fe640 1 -- 172.21.15.28:0/2216568826 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbd3000e040 con 0x557515a1f650 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.440+0000 7fbd35feb640 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1003d000 0x7fbc1003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd3a81b800 1 Processor -- start 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd3a81b800 1 -- start start 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd35feb640 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1003d000 0x7fbc1003f4c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbd2807c700 tx=0x7fbd2807c690 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd3a81b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b40c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd3a81b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557515b0fbb0 con 0x557515b40860 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd367ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b40c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd367ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b40c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45372/0 (socket says 172.21.15.28:45372) 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd367ec640 1 -- 172.21.15.28:0/3442666922 learned_addr learned my addr 172.21.15.28:0/3442666922 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd367ec640 1 -- 172.21.15.28:0/3442666922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557515b41250 con 0x557515b40860 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.441+0000 7fbd367ec640 1 --2- 172.21.15.28:0/3442666922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b40c60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbd300547f0 tx=0x7fbd3001fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1850b73d5519cadf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbc197fa640 1 -- 172.21.15.28:0/3442666922 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30051070 con 0x557515b40860 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbc197fa640 1 -- 172.21.15.28:0/3442666922 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd30053440 con 0x557515b40860 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbc197fa640 1 -- 172.21.15.28:0/3442666922 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30021ad0 con 0x557515b40860 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3442666922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 msgr2=0x557515b40c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/3442666922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b40c60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbd300547f0 tx=0x7fbd3001fe70 comp rx=0 tx=0).stop 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3442666922 shutdown_connections 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/3442666922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b40c60 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3442666922 >> 172.21.15.28:0/3442666922 conn(0x557515b3cc60 msgr2=0x557515b3aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3442666922 shutdown_connections 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 -- 172.21.15.28:0/3442666922 wait complete. 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.442+0000 7fbd3a81b800 1 Processor -- start 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd3a81b800 1 -- start start 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd3a81b800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b47ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd3a81b800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557515b41870 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd367ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b47ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd367ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b47ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45376/0 (socket says 172.21.15.28:45376) 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 learned_addr learned my addr 172.21.15.28:0/2788854093 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557515b48420 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd367ec640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b47ec0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbd30022ff0 tx=0x7fbd3001fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbc077fe640 1 -- 172.21.15.28:0/2788854093 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30021440 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbc077fe640 1 -- 172.21.15.28:0/2788854093 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd30021620 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557515b486b0 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbc077fe640 1 -- 172.21.15.28:0/2788854093 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30045b30 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.443+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557515b486b0 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.444+0000 7fbc077fe640 1 -- 172.21.15.28:0/2788854093 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fbd30045cd0 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.444+0000 7fbc077fe640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc0003d000 0x7fbc0003f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.444+0000 7fbc077fe640 1 -- 172.21.15.28:0/2788854093 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fbd30034d90 con 0x557515b40860 2024-09-19T11:25:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.444+0000 7fbd35feb640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc0003d000 0x7fbc0003f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.445+0000 7fbd35feb640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc0003d000 0x7fbc0003f4c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbd28054800 tx=0x7fbd28086000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.445+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.445+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515b535c0 0x557515b55a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.445+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b561d0 con 0x557515b535c0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.445+0000 7fbd2d7ea640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515b535c0 0x557515b55a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd2d7ea640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515b535c0 0x557515b55a80 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbd2000f5f0 con 0x557515b535c0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 20 realm 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b52910 con 0x557515b535c0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbd2000f5f0 con 0x557515b535c0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b56cc0 con 0x557515b535c0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbd2000f5f0 con 0x557515b535c0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.446+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b57590 con 0x557515b535c0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbd2000f5f0 con 0x557515b535c0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd3a81b800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b57fb0 0x557515b5a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b5ab10 con 0x557515b57fb0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd367ec640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b57fb0 0x557515b5a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd367ec640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b57fb0 0x557515b5a3a0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd3a81b800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd3a81b800 20 searching for the correct realm 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.447+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd367ec640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbd3004d340 0x7fbd300119f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd35feb640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbd3004d340 0x7fbd300119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd35feb640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbd3004d340 0x7fbd300119f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd35feb640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbd2807d060 0x7fbd2805e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd2807d060 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.448+0000 7fbd2d7ea640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbd2807d060 0x7fbd2805e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.449+0000 7fbd2d7ea640 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbd2807d060 0x7fbd2805e810 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.449+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2000f5f0 con 0x7fbd2807d060 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.449+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.449+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2000f5f0 con 0x557515b535c0 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.449+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd2807d060 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x7fbd2807d060 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd2807d060 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x7fbd2807d060 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.450+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd2807d060 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x7fbd2807d060 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.451+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.452+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd2807d060 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x7fbd2807d060 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.453+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.454+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.454+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.454+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.454+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.454+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.454+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd2807d060 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x7fbd2807d060 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd2807d060 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2002b040 con 0x7fbd2807d060 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.455+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b57fb0 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x557515b535c0 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c440 con 0x7fbd3004d340 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.456+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b5b4d0 con 0x557515b535c0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b5bd70 con 0x557515b535c0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbd2002b040 con 0x557515b535c0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b5c8c0 con 0x557515b57fb0 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 zone default found 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 4 Realm: () 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 zonegroup default 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.457+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b5cc60 con 0x7fbd2807d060 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbd2002b040 con 0x7fbd2807d060 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b5ef40 con 0x557515b57fb0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b5f2e0 con 0x557515b57fb0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 20 started sync module instance, tier type = 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.458+0000 7fbd3a81b800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.459+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b5e3d0 con 0x7fbd3004d340 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.460+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.460+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93961363777472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b5e030 con 0x7fbd3004d340 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.460+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b600b0 con 0x557515b535c0 2024-09-19T11:25:10.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.461+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 93961363777472] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.461+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.461+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93961363781392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b604d0 con 0x557515b535c0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.461+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b60ef0 con 0x557515b57fb0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.463+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93961363781392] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.463+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.463+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93961363786208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b60eb0 con 0x557515b57fb0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.463+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b619b0 con 0x7fbd3004d340 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.464+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 93961363786208] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.464+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.464+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93961363788928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b61970 con 0x7fbd3004d340 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.464+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b62470 con 0x557515b535c0 2024-09-19T11:25:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.465+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 93961363788928] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.465+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.465+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93961363791776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b62470 con 0x557515b535c0 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.465+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b62f20 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.466+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.466+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93961363791776] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.466+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93961363794432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b62f20 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.466+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b5fb70 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.467+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 93961363794432] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.467+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.467+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93961363782512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b63960 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.467+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b64320 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 93961363782512] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93961363796560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b64320 con 0x7fbd2807d060 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 20 add_watcher() i=0 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 20 add_watcher() i=1 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 20 add_watcher() i=2 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 20 add_watcher() i=3 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 20 add_watcher() i=4 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 20 add_watcher() i=5 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.468+0000 7fbd3a81b800 20 add_watcher() i=6 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.469+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93961363796560] v33'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.469+0000 7fbd3a81b800 20 add_watcher() i=7 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.469+0000 7fbd3a81b800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.469+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b67740 con 0x557515b535c0 2024-09-19T11:25:10.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.470+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.470+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93961363800128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b64c40 con 0x557515b535c0 2024-09-19T11:25:10.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93961363800128] v33'265 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbbebfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbd3a81b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbd3a81b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbd3a81b800 5 note: GC not initialized 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbbeafe5640 20 reqs_thread_entry: start 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbd3a81b800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.471+0000 7fbd3a81b800 20 Filter name: none 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbbe8fe1640 20 reqs_thread_entry: start 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b9a5c0 con 0x7fbd2807d060 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 20 rados_obj.operate() r=0 bl.length=460 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.472+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515ba0a20 con 0x7fbd2807d060 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v33'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd3a81b800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd3a81b800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd3a81b800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93961364046816 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x557515b9c040 con 0x557515b535c0 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 93961363791776 notify 141733920800 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7fbd20018040 con 0x557515b535c0 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbc197fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920800 cookie 93961363791776 notifier 4900 bl.length()=666 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 93961364046816 out=8b] v0'0 uv270 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbc197fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbc197fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbc197fa640 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7fbbf8003ea0 con 0x557515b535c0 2024-09-19T11:25:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.474+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 93961364046816 notify 141733920800 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fbd2001db80 con 0x557515b535c0 2024-09-19T11:25:10.484 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"write"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:10.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.475+0000 7fbbebfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.475+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93961363800128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x557515b535c0 2024-09-19T11:25:10.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.476+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93961363800128] v33'266 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.476+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93961363777472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x7fbd3004d340 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.477+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93961363777472] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.477+0000 7fbd3a81b800 20 remove_watcher() i=0 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.477+0000 7fbd3a81b800 2 removed watcher, disabling cache 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.477+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93961363781392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x557515b535c0 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.478+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93961363781392] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.478+0000 7fbd3a81b800 20 remove_watcher() i=1 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.478+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93961363786208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x557515b57fb0 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.479+0000 7fbd367ec640 1 -- 172.21.15.28:0/2788854093 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93961363786208] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd3000a4d0 con 0x557515b57fb0 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.479+0000 7fbd3a81b800 20 remove_watcher() i=2 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.479+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93961363788928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x7fbd3004d340 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.480+0000 7fbd35feb640 1 -- 172.21.15.28:0/2788854093 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93961363788928] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2807a030 con 0x7fbd3004d340 2024-09-19T11:25:10.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.480+0000 7fbd3a81b800 20 remove_watcher() i=3 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.480+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93961363791776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x557515b535c0 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.481+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93961363791776] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x557515b535c0 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.481+0000 7fbd3a81b800 20 remove_watcher() i=4 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.481+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93961363794432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x7fbd2807d060 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.482+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93961363794432] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.482+0000 7fbd3a81b800 20 remove_watcher() i=5 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.482+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93961363782512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x7fbd2807d060 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.483+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93961363782512] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.483+0000 7fbd3a81b800 20 remove_watcher() i=6 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.483+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93961363796560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x557515b98450 con 0x7fbd2807d060 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.484+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93961363796560] v33'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbd2003b040 con 0x7fbd2807d060 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.484+0000 7fbd3a81b800 20 remove_watcher() i=7 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbd2807d060 msgr2=0x7fbd2805e810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7fbd2807d060 0x7fbd2805e810 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515b535c0 msgr2=0x557515b55a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515b535c0 0x557515b55a80 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbd3004d340 msgr2=0x7fbd300119f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fbd3004d340 0x7fbd300119f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b57fb0 msgr2=0x557515b5a3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b57fb0 0x557515b5a3a0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc0003d000 msgr2=0x7fbc0003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc0003d000 0x7fbc0003f4c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbd28054800 tx=0x7fbd28086000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 msgr2=0x557515b47ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd2d7ea640 1 -- 172.21.15.28:0/2788854093 reap_dead start 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b47ec0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbd30022ff0 tx=0x7fbd3001fee0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 shutdown_connections 2024-09-19T11:25:10.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2788854093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515b40860 0x557515b47ec0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 >> 172.21.15.28:0/2788854093 conn(0x557515b3cc60 msgr2=0x557515b3ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 shutdown_connections 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2788854093 wait complete. 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1003d000 msgr2=0x7fbc1003f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1003d000 0x7fbc1003f4c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbd2807c700 tx=0x7fbd2807c690 comp rx=0 tx=0).stop 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1f650 msgr2=0x557515a1dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1f650 0x557515a1dec0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x557515b18240 tx=0x7fbd300167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 shutdown_connections 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1003d000 0x7fbc1003f4c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/2216568826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1f650 0x557515a1dec0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 >> 172.21.15.28:0/2216568826 conn(0x557515b25fb0 msgr2=0x557515b26390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 shutdown_connections 2024-09-19T11:25:10.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.485+0000 7fbd3a81b800 1 -- 172.21.15.28:0/2216568826 wait complete. 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515a228d0 msgr2=0x557515b1e270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515a228d0 0x557515b1e270 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b20690 msgr2=0x557515b22b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b20690 0x557515b22b30 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1c03cfb0 msgr2=0x7fbc1c03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1c03cfb0 0x7fbc1c03f470 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd20000c00 tx=0x7fbd200013b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 msgr2=0x557515b16c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b16c20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbd28045f50 tx=0x7fbd28002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 shutdown_connections 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x557515b20690 0x557515b22b30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fbc1c03cfb0 0x7fbc1c03f470 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x557515a228d0 0x557515b1e270 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 --2- 172.21.15.28:0/685567636 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557515a1feb0 0x557515b16c20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 >> 172.21.15.28:0/685567636 conn(0x557515a19a70 msgr2=0x557515b09360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 shutdown_connections 2024-09-19T11:25:10.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.486+0000 7fbd3a81b800 1 -- 172.21.15.28:0/685567636 wait complete. 2024-09-19T11:25:10.497 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.497 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.497 ERROR:root:0 2024-09-19T11:25:10.497 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:10.508 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'AccessDenied', 'Message': '', 'RequestId': 'tx00000f2c7de34f47f257c-0066ec0a16-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:25:10.509 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-09-19T11:25:10.509 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-09-19T11:25:10.509 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps user-info-without-keys=write 2024-09-19T11:25:10.557 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:10.557 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:10.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.556+0000 7f6331e0f800 1 Processor -- start 2024-09-19T11:25:10.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.556+0000 7f6331e0f800 1 -- start start 2024-09-19T11:25:10.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.556+0000 7f6331e0f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5651fe3f0ab0 0x5651fe3f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.556+0000 7f6331e0f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5651fe3efd50 con 0x5651fe3ef6a0 2024-09-19T11:25:10.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.556+0000 7f6331e0f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5651fe27b7d0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.557+0000 7f632f0ff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5651fe3f0ab0 0x5651fe3f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.557+0000 7f632f0ff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5651fe3f0ab0 0x5651fe3f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45392/0 (socket says 172.21.15.28:45392) 2024-09-19T11:25:10.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.557+0000 7f632f900640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5651fe3ef6a0 0x5651fe3f03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48358/0 (socket says 172.21.15.28:48358) 2024-09-19T11:25:10.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.557+0000 7f632f0ff640 1 -- 172.21.15.28:0/49336508 learned_addr learned my addr 172.21.15.28:0/49336508 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.557+0000 7f632f0ff640 1 -- 172.21.15.28:0/49336508 >> v1:172.21.15.22:6789/0 conn(0x5651fe3ef6a0 legacy=0x5651fe3f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.557+0000 7f632f0ff640 1 -- 172.21.15.28:0/49336508 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fe3f0f60 con 0x5651fe3f0ab0 2024-09-19T11:25:10.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.559+0000 7f632f0ff640 1 --2- 172.21.15.28:0/49336508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3f0ab0 0x5651fe3f6650 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f632400a520 tx=0x7f6324045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190ac92dc2110ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.559+0000 7f632e8fe640 1 -- 172.21.15.28:0/49336508 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6324055070 con 0x5651fe3f0ab0 2024-09-19T11:25:10.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.559+0000 7f632e8fe640 1 -- 172.21.15.28:0/49336508 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63240515b0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.559+0000 7f632e8fe640 1 -- 172.21.15.28:0/49336508 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63240518d0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.559+0000 7f6331e0f800 1 -- 172.21.15.28:0/49336508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3f0ab0 msgr2=0x5651fe3f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.559+0000 7f6331e0f800 1 --2- 172.21.15.28:0/49336508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3f0ab0 0x5651fe3f6650 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f632400a520 tx=0x7f6324045ba0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.560+0000 7f6331e0f800 1 -- 172.21.15.28:0/49336508 shutdown_connections 2024-09-19T11:25:10.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.560+0000 7f6331e0f800 1 --2- 172.21.15.28:0/49336508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3f0ab0 0x5651fe3f6650 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.560+0000 7f6331e0f800 1 -- 172.21.15.28:0/49336508 >> 172.21.15.28:0/49336508 conn(0x5651fe3eaa70 msgr2=0x5651fe3eced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.560+0000 7f6331e0f800 1 -- 172.21.15.28:0/49336508 shutdown_connections 2024-09-19T11:25:10.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.560+0000 7f6331e0f800 1 -- 172.21.15.28:0/49336508 wait complete. 2024-09-19T11:25:10.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f6331e0f800 1 Processor -- start 2024-09-19T11:25:10.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f6331e0f800 1 -- start start 2024-09-19T11:25:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f6331e0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f6331e0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5651fe288dd0 con 0x5651fe3ef6a0 2024-09-19T11:25:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f632f900640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f632f900640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45404/0 (socket says 172.21.15.28:45404) 2024-09-19T11:25:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f632f900640 1 -- 172.21.15.28:0/3102777413 learned_addr learned my addr 172.21.15.28:0/3102777413 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.575+0000 7f632f900640 1 -- 172.21.15.28:0/3102777413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fe4e0990 con 0x5651fe3ef6a0 2024-09-19T11:25:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f632f900640 1 --2- 172.21.15.28:0/3102777413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e03a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f632000b260 tx=0x7f632000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6bde3449e5329c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f621de0b640 1 -- 172.21.15.28:0/3102777413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f632000be60 con 0x5651fe3ef6a0 2024-09-19T11:25:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f621de0b640 1 -- 172.21.15.28:0/3102777413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6320006070 con 0x5651fe3ef6a0 2024-09-19T11:25:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f621de0b640 1 -- 172.21.15.28:0/3102777413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f632000e710 con 0x5651fe3ef6a0 2024-09-19T11:25:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f6331e0f800 1 -- 172.21.15.28:0/3102777413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 msgr2=0x5651fe4e03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3102777413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e03a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f632000b260 tx=0x7f632000b660 comp rx=0 tx=0).stop 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f6331e0f800 1 -- 172.21.15.28:0/3102777413 shutdown_connections 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3102777413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e03a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.576+0000 7f6331e0f800 1 -- 172.21.15.28:0/3102777413 >> 172.21.15.28:0/3102777413 conn(0x5651fe3eaa70 msgr2=0x5651fe4de740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f6331e0f800 1 -- 172.21.15.28:0/3102777413 shutdown_connections 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f6331e0f800 1 -- 172.21.15.28:0/3102777413 wait complete. 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f6331e0f800 1 Processor -- start 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f6331e0f800 1 -- start start 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f6331e0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f6331e0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5651fe3f6da0 con 0x5651fe3ef6a0 2024-09-19T11:25:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f632f900640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f632f900640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45416/0 (socket says 172.21.15.28:45416) 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f632f900640 1 -- 172.21.15.28:0/3398437365 learned_addr learned my addr 172.21.15.28:0/3398437365 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.577+0000 7f632f900640 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fe4e8480 con 0x5651fe3ef6a0 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.578+0000 7f632f900640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e7f20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x5651fe4e1330 tx=0x7f6320000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.578+0000 7f6217fff640 1 -- 172.21.15.28:0/3398437365 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f632000ee10 con 0x5651fe3ef6a0 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.578+0000 7f6217fff640 1 -- 172.21.15.28:0/3398437365 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632000fa30 con 0x5651fe3ef6a0 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.578+0000 7f6217fff640 1 -- 172.21.15.28:0/3398437365 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f632000fd30 con 0x5651fe3ef6a0 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.578+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651fe4e8710 con 0x5651fe3ef6a0 2024-09-19T11:25:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.578+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fe4e9110 con 0x5651fe3ef6a0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f6217fff640 1 -- 172.21.15.28:0/3398437365 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6320017070 con 0x5651fe3ef6a0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f6217fff640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f621803cfb0 0x7f621803f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f6217fff640 1 -- 172.21.15.28:0/3398437365 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f6320015020 con 0x5651fe3ef6a0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f632f0ff640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f621803cfb0 0x7f621803f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe3f0ab0 0x5651fe4ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe4efab0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f6330101640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe3f0ab0 0x5651fe4ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.579+0000 7f632f0ff640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f621803cfb0 0x7f621803f470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6324057000 tx=0x7f6324052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.580+0000 7f6330101640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe3f0ab0 0x5651fe4ef570 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.580+0000 7f6330101640 1 -- 172.21.15.28:0/3398437365 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.580+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe4efe50 con 0x5651fe3f0ab0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.580+0000 7f6330101640 1 -- 172.21.15.28:0/3398437365 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.580+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe4f1950 0x5651fe4f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.580+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe4f4560 con 0x5651fe4f1950 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.580+0000 7f632f900640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe4f1950 0x5651fe4f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.581+0000 7f632f900640 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe4f1950 0x5651fe4f3dd0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.581+0000 7f632f900640 1 -- 172.21.15.28:0/3398437365 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f632002e070 con 0x5651fe4f1950 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.581+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe4f06a0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.581+0000 7f6330101640 1 -- 172.21.15.28:0/3398437365 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe3f0ab0 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.581+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe4f0a90 con 0x5651fe4f1950 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.581+0000 7f632f900640 1 -- 172.21.15.28:0/3398437365 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f632002e070 con 0x5651fe4f1950 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6331e0f800 1 Processor -- start 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6331e0f800 1 -- start start 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6331e0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe502eb0 0x5651fe5032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6331e0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5651fe2e74a0 con 0x5651fe502eb0 2024-09-19T11:25:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6330101640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe502eb0 0x5651fe5032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6330101640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe502eb0 0x5651fe5032d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45424/0 (socket says 172.21.15.28:45424) 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6330101640 1 -- 172.21.15.28:0/2256531978 learned_addr learned my addr 172.21.15.28:0/2256531978 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.582+0000 7f6330101640 1 -- 172.21.15.28:0/2256531978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fe503910 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.583+0000 7f6330101640 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe502eb0 0x5651fe5032d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x5651fe4e8f20 tx=0x7f63280167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.583+0000 7f6215ffb640 1 -- 172.21.15.28:0/2256531978 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6328013640 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.583+0000 7f6215ffb640 1 -- 172.21.15.28:0/2256531978 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63280137e0 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.583+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651fe504140 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.583+0000 7f6215ffb640 1 -- 172.21.15.28:0/2256531978 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6328013ae0 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.583+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fe5046e0 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f6215ffb640 1 -- 172.21.15.28:0/2256531978 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6328011040 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f6215ffb640 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620803d000 0x7f620803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f6215ffb640 1 -- 172.21.15.28:0/2256531978 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f632800e040 con 0x5651fe502eb0 2024-09-19T11:25:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f632f900640 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620803d000 0x7f620803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f6331e0f800 1 Processor -- start 2024-09-19T11:25:10.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f6331e0f800 1 -- start start 2024-09-19T11:25:10.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f632f900640 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620803d000 0x7f620803f4c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6320017ab0 tx=0x7f6320053160 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f6331e0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe50fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.584+0000 7f6331e0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5651fe4e0dc0 con 0x5651fe50f930 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f6330101640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe50fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f6330101640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe50fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45426/0 (socket says 172.21.15.28:45426) 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f6330101640 1 -- 172.21.15.28:0/3702331415 learned_addr learned my addr 172.21.15.28:0/3702331415 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f6330101640 1 -- 172.21.15.28:0/3702331415 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fe510320 con 0x5651fe50f930 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f6330101640 1 --2- 172.21.15.28:0/3702331415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe50fd30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f63280547f0 tx=0x7f632801fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461bc0f9f7c704c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f620effd640 1 -- 172.21.15.28:0/3702331415 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6328051070 con 0x5651fe50f930 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f620effd640 1 -- 172.21.15.28:0/3702331415 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6328053440 con 0x5651fe50f930 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.585+0000 7f620effd640 1 -- 172.21.15.28:0/3702331415 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6328021ad0 con 0x5651fe50f930 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 -- 172.21.15.28:0/3702331415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 msgr2=0x5651fe50fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3702331415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe50fd30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f63280547f0 tx=0x7f632801fe70 comp rx=0 tx=0).stop 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 -- 172.21.15.28:0/3702331415 shutdown_connections 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3702331415 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe50fd30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 -- 172.21.15.28:0/3702331415 >> 172.21.15.28:0/3702331415 conn(0x5651fe50bd30 msgr2=0x5651fe509f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 -- 172.21.15.28:0/3702331415 shutdown_connections 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 -- 172.21.15.28:0/3702331415 wait complete. 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 Processor -- start 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 -- start start 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe516f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6331e0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5651fe510940 con 0x5651fe50f930 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6330101640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe516f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6330101640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe516f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45438/0 (socket says 172.21.15.28:45438) 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.586+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 learned_addr learned my addr 172.21.15.28:0/3514581209 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.587+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fe5174c0 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.587+0000 7f6330101640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe516f60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6328022ff0 tx=0x7f632801fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.587+0000 7f620d7fa640 1 -- 172.21.15.28:0/3514581209 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6328021440 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.587+0000 7f620d7fa640 1 -- 172.21.15.28:0/3514581209 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6328021620 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.587+0000 7f620d7fa640 1 -- 172.21.15.28:0/3514581209 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6328045b30 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.587+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651fe517750 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.587+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fe517750 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.588+0000 7f620d7fa640 1 -- 172.21.15.28:0/3514581209 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6328045cd0 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.588+0000 7f620d7fa640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f61f803d000 0x7f61f803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.588+0000 7f620d7fa640 1 -- 172.21.15.28:0/3514581209 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f6328034d70 con 0x5651fe50f930 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.588+0000 7f632f900640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f61f803d000 0x7f61f803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.588+0000 7f632f900640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f61f803d000 0x7f61f803f4c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f632000f6a0 tx=0x7f6320048000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe522670 0x5651fe524b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe525280 con 0x5651fe522670 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f632f0ff640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe522670 0x5651fe524b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f632f0ff640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe522670 0x5651fe524b30 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f6331e0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f6331e0f800 20 realm 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.589+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5256b0 con 0x5651fe522670 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe525f80 con 0x5651fe522670 2024-09-19T11:25:10.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe526320 con 0x5651fe522670 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe527080 0x5651fe529470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe529a80 con 0x5651fe527080 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.590+0000 7f6330101640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe527080 0x5651fe529470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6330101640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe527080 0x5651fe529470 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6331e0f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:10.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6331e0f800 20 searching for the correct realm 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6330101640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6328045130 0x7f6328047960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.591+0000 7f632f900640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6328045130 0x7f6328047960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.592+0000 7f632f900640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6328045130 0x7f6328047960 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.592+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632002e070 con 0x7f6328045130 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.592+0000 7f632f900640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f632000f070 0x7f63200535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.592+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f632000f070 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.592+0000 7f632f0ff640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f632000f070 0x7f63200535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.593+0000 7f632f0ff640 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f632000f070 0x7f63200535a0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.593+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.593+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.593+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.593+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.593+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.593+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f632000f070 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f632000f070 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632002e070 con 0x7f6328045130 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.594+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f632000f070 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.595+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.596+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f632000f070 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.597+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f632000f070 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.598+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f632000f070 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe527080 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.599+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x5651fe522670 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b3d0 con 0x7f6328045130 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52a440 con 0x5651fe522670 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.600+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52ac90 con 0x5651fe522670 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52b850 con 0x5651fe527080 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 zone default found 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 4 Realm: () 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 zonegroup default 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52bbf0 con 0x7f632000f070 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.601+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52deb0 con 0x5651fe527080 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6331e0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52e270 con 0x5651fe527080 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6331e0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6331e0f800 20 started sync module instance, tier type = 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6331e0f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.602+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52d340 con 0x7f6328045130 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.603+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.603+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94910159181616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52eb80 con 0x7f6328045130 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.603+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52f3d0 con 0x5651fe522670 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.604+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.604+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94910159187744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52f910 con 0x5651fe522670 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.604+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe530430 con 0x5651fe527080 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.604+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94910159181616] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.605+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.605+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94910159187744] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.605+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94910159191808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5303f0 con 0x5651fe527080 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.605+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe52f3c0 con 0x7f6328045130 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.606+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94910159191808] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.606+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.606+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94910159185616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe530ea0 con 0x7f6328045130 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.606+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe531860 con 0x5651fe522670 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.607+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.607+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94910159197104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe531c80 con 0x5651fe522670 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.607+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe532740 con 0x7f632000f070 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.608+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94910159185616] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.608+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94910159197104] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.608+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.608+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94910159200784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5317d0 con 0x7f632000f070 2024-09-19T11:25:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.609+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe532fe0 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.610+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94910159200784] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.610+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.610+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94910159194608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5317d0 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.610+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5335b0 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94910159194608] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94910159202976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe533570 con 0x7f632000f070 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 20 add_watcher() i=0 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 20 add_watcher() i=1 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 20 add_watcher() i=2 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 20 add_watcher() i=3 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 20 add_watcher() i=4 2024-09-19T11:25:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 20 add_watcher() i=5 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.611+0000 7f6331e0f800 20 add_watcher() i=6 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.612+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94910159202976] v33'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.612+0000 7f6331e0f800 20 add_watcher() i=7 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.612+0000 7f6331e0f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.612+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5369d0 con 0x5651fe522670 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.612+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.612+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94910159205008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe533f20 con 0x5651fe522670 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.613+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94910159205008] v33'267 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.613+0000 7f61e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.613+0000 7f6331e0f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.613+0000 7f6331e0f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.613+0000 7f6331e0f800 5 note: GC not initialized 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.614+0000 7f61e4fe1640 20 reqs_thread_entry: start 2024-09-19T11:25:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.614+0000 7f6331e0f800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.614+0000 7f6331e0f800 20 Filter name: none 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.614+0000 7f615e7fc640 20 reqs_thread_entry: start 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.614+0000 7f6331e0f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.614+0000 7f6331e0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.614+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe569850 con 0x7f632000f070 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.615+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.615+0000 7f6331e0f800 20 rados_obj.operate() r=0 bl.length=490 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.615+0000 7f6331e0f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.615+0000 7f6331e0f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.615+0000 7f6331e0f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.615+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe56fdc0 con 0x7f632000f070 2024-09-19T11:25:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.616+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v33'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.616+0000 7f6331e0f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.616+0000 7f6331e0f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.616+0000 7f6331e0f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.616+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94910159452032 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5651fe56b420 con 0x5651fe522670 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 94910159197104 notify 141733920801 ret 0) ==== 678+0+0 (crc 0 0 0) 0x5651fe3f0f60 con 0x5651fe522670 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f620effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920801 cookie 94910159197104 notifier 4907 bl.length()=636 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f620effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f620effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 94910159452032 out=8b] v0'0 uv273 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f620effd640 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f61f4003ea0 con 0x5651fe522670 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.617+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 94910159452032 notify 141733920801 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f632405ba90 con 0x5651fe522670 2024-09-19T11:25:10.628 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:10.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.618+0000 7f61e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.618+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94910159205008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x5651fe522670 2024-09-19T11:25:10.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.619+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94910159205008] v33'268 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.619+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94910159181616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x7f6328045130 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.620+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94910159181616] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.620+0000 7f6331e0f800 20 remove_watcher() i=0 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.620+0000 7f6331e0f800 2 removed watcher, disabling cache 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.620+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94910159187744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x5651fe522670 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.621+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94910159187744] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.621+0000 7f6331e0f800 20 remove_watcher() i=1 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.621+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94910159191808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x5651fe527080 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.622+0000 7f6330101640 1 -- 172.21.15.28:0/3514581209 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94910159191808] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632800a4d0 con 0x5651fe527080 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.622+0000 7f6331e0f800 20 remove_watcher() i=2 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.622+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94910159185616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x7f6328045130 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.623+0000 7f632f900640 1 -- 172.21.15.28:0/3514581209 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94910159185616] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f632005e060 con 0x7f6328045130 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.623+0000 7f6331e0f800 20 remove_watcher() i=3 2024-09-19T11:25:10.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.623+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94910159197104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x5651fe522670 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.624+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94910159197104] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x5651fe522670 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.624+0000 7f6331e0f800 20 remove_watcher() i=4 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.624+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94910159200784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x7f632000f070 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.625+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94910159200784] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.625+0000 7f6331e0f800 20 remove_watcher() i=5 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.625+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94910159194608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x7f632000f070 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.626+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94910159194608] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.626+0000 7f6331e0f800 20 remove_watcher() i=6 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.626+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94910159202976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5651fe5676c0 con 0x7f632000f070 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94910159202976] v33'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6324057030 con 0x7f632000f070 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 20 remove_watcher() i=7 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f632000f070 msgr2=0x7f63200535a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f632000f070 0x7f63200535a0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe522670 msgr2=0x5651fe524b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe522670 0x5651fe524b30 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6328045130 msgr2=0x7f6328047960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f6328045130 0x7f6328047960 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe527080 msgr2=0x5651fe529470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe527080 0x5651fe529470 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f61f803d000 msgr2=0x7f61f803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f61f803d000 0x7f61f803f4c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f632000f6a0 tx=0x7f6320048000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 msgr2=0x5651fe516f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f632f0ff640 1 -- 172.21.15.28:0/3514581209 reap_dead start 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe516f60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6328022ff0 tx=0x7f632801fee0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 shutdown_connections 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3514581209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe50f930 0x5651fe516f60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 >> 172.21.15.28:0/3514581209 conn(0x5651fe50bd30 msgr2=0x5651fe509f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.627+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 shutdown_connections 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/3514581209 wait complete. 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620803d000 msgr2=0x7f620803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620803d000 0x7f620803f4c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6320017ab0 tx=0x7f6320053160 comp rx=0 tx=0).stop 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe502eb0 msgr2=0x5651fe5032d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe502eb0 0x5651fe5032d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x5651fe4e8f20 tx=0x7f63280167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 shutdown_connections 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f620803d000 0x7f620803f4c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/2256531978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe502eb0 0x5651fe5032d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 >> 172.21.15.28:0/2256531978 conn(0x5651fe4f7260 msgr2=0x5651fe4f7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 shutdown_connections 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/2256531978 wait complete. 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe3f0ab0 msgr2=0x5651fe4ef570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe3f0ab0 0x5651fe4ef570 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe4f1950 msgr2=0x5651fe4f3dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe4f1950 0x5651fe4f3dd0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f621803cfb0 msgr2=0x7f621803f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f621803cfb0 0x7f621803f470 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6324057000 tx=0x7f6324052000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 msgr2=0x5651fe4e7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e7f20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x5651fe4e1330 tx=0x7f6320000ed0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 shutdown_connections 2024-09-19T11:25:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5651fe4f1950 0x5651fe4f3dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f621803cfb0 0x7f621803f470 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5651fe3f0ab0 0x5651fe4ef570 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 --2- 172.21.15.28:0/3398437365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5651fe3ef6a0 0x5651fe4e7f20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.628+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 >> 172.21.15.28:0/3398437365 conn(0x5651fe3eaa70 msgr2=0x5651fe4d9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.629+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 shutdown_connections 2024-09-19T11:25:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.629+0000 7f6331e0f800 1 -- 172.21.15.28:0/3398437365 wait complete. 2024-09-19T11:25:10.642 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.642 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.642 ERROR:root:0 2024-09-19T11:25:10.642 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:10.642 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:10.642 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-09-19T11:25:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.689+0000 7f1b7032a800 1 Processor -- start 2024-09-19T11:25:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.689+0000 7f1b7032a800 1 -- start start 2024-09-19T11:25:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.689+0000 7f1b7032a800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55b8a8966080 0x55b8a896df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.689+0000 7f1b7032a800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b8a896e5e0 con 0x55b8a8965b60 2024-09-19T11:25:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.689+0000 7f1b7032a800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x55b8a87f27d0 con 0x55b8a8966080 2024-09-19T11:25:10.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.690+0000 7f1b6b84e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55b8a8966080 0x55b8a896df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.690+0000 7f1b6b84e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x55b8a8966080 0x55b8a896df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45442/0 (socket says 172.21.15.28:45442) 2024-09-19T11:25:10.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.690+0000 7f1b6c04f640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x55b8a8965b60 0x55b8a896a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48362/0 (socket says 172.21.15.28:48362) 2024-09-19T11:25:10.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.690+0000 7f1b6b84e640 1 -- 172.21.15.28:0/52350851 learned_addr learned my addr 172.21.15.28:0/52350851 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.690+0000 7f1b6b84e640 1 -- 172.21.15.28:0/52350851 >> v1:172.21.15.22:6789/0 conn(0x55b8a8965b60 legacy=0x55b8a896a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.690+0000 7f1b6b84e640 1 -- 172.21.15.28:0/52350851 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a896f810 con 0x55b8a8966080 2024-09-19T11:25:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.692+0000 7f1b6b84e640 1 --2- 172.21.15.28:0/52350851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8966080 0x55b8a896df60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000a520 tx=0x7f1b60045ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b9d12ff5c2be2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.692+0000 7f1b6b04d640 1 -- 172.21.15.28:0/52350851 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b60055070 con 0x55b8a8966080 2024-09-19T11:25:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.692+0000 7f1b6b04d640 1 -- 172.21.15.28:0/52350851 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b600515b0 con 0x55b8a8966080 2024-09-19T11:25:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.692+0000 7f1b6b04d640 1 -- 172.21.15.28:0/52350851 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b600518d0 con 0x55b8a8966080 2024-09-19T11:25:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.693+0000 7f1b7032a800 1 -- 172.21.15.28:0/52350851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8966080 msgr2=0x55b8a896df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.693+0000 7f1b7032a800 1 --2- 172.21.15.28:0/52350851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8966080 0x55b8a896df60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000a520 tx=0x7f1b60045ba0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.693+0000 7f1b7032a800 1 -- 172.21.15.28:0/52350851 shutdown_connections 2024-09-19T11:25:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.693+0000 7f1b7032a800 1 --2- 172.21.15.28:0/52350851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8966080 0x55b8a896df60 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.693+0000 7f1b7032a800 1 -- 172.21.15.28:0/52350851 >> 172.21.15.28:0/52350851 conn(0x55b8a89611d0 msgr2=0x55b8a8963650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.693+0000 7f1b7032a800 1 -- 172.21.15.28:0/52350851 shutdown_connections 2024-09-19T11:25:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.693+0000 7f1b7032a800 1 -- 172.21.15.28:0/52350851 wait complete. 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b7032a800 1 Processor -- start 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b7032a800 1 -- start start 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b7032a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b7032a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b8a87ffdd0 con 0x55b8a8965b60 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b6c04f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b6c04f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a55190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45458/0 (socket says 172.21.15.28:45458) 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1748416747 learned_addr learned my addr 172.21.15.28:0/1748416747 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.708+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1748416747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a8a55780 con 0x55b8a8965b60 2024-09-19T11:25:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.709+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/1748416747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a55190 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0016e0 tx=0x7f1b5c001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8824efb5c1598228 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.709+0000 7f1a5a60c640 1 -- 172.21.15.28:0/1748416747 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5c013420 con 0x55b8a8965b60 2024-09-19T11:25:10.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.709+0000 7f1a5a60c640 1 -- 172.21.15.28:0/1748416747 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c0135e0 con 0x55b8a8965b60 2024-09-19T11:25:10.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.709+0000 7f1a5a60c640 1 -- 172.21.15.28:0/1748416747 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5c013900 con 0x55b8a8965b60 2024-09-19T11:25:10.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.709+0000 7f1b7032a800 1 -- 172.21.15.28:0/1748416747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 msgr2=0x55b8a8a55190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.709+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1748416747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a55190 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0016e0 tx=0x7f1b5c001ba0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 -- 172.21.15.28:0/1748416747 shutdown_connections 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1748416747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a55190 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 -- 172.21.15.28:0/1748416747 >> 172.21.15.28:0/1748416747 conn(0x55b8a89611d0 msgr2=0x55b8a8a50b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 -- 172.21.15.28:0/1748416747 shutdown_connections 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 -- 172.21.15.28:0/1748416747 wait complete. 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 Processor -- start 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 -- start start 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b7032a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b8a896eab0 con 0x55b8a8965b60 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b6c04f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b6c04f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a5cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45470/0 (socket says 172.21.15.28:45470) 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b6c04f640 1 -- 172.21.15.28:0/3901310422 learned_addr learned my addr 172.21.15.28:0/3901310422 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.710+0000 7f1b6c04f640 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a8a5d240 con 0x55b8a8965b60 2024-09-19T11:25:10.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.711+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a5cce0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000f80 tx=0x7f1b5c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.711+0000 7f1a58e09640 1 -- 172.21.15.28:0/3901310422 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5c022020 con 0x55b8a8965b60 2024-09-19T11:25:10.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.711+0000 7f1a58e09640 1 -- 172.21.15.28:0/3901310422 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5c006810 con 0x55b8a8965b60 2024-09-19T11:25:10.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.711+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b8a8a5d4d0 con 0x55b8a8965b60 2024-09-19T11:25:10.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.711+0000 7f1a58e09640 1 -- 172.21.15.28:0/3901310422 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5c004ae0 con 0x55b8a8965b60 2024-09-19T11:25:10.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.711+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a8a5ded0 con 0x55b8a8965b60 2024-09-19T11:25:10.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.712+0000 7f1a58e09640 1 -- 172.21.15.28:0/3901310422 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1b5c025020 con 0x55b8a8965b60 2024-09-19T11:25:10.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.712+0000 7f1a58e09640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5403cfb0 0x7f1a5403f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.712+0000 7f1a58e09640 1 -- 172.21.15.28:0/3901310422 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f1b5c018030 con 0x55b8a8965b60 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.712+0000 7f1b6b84e640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5403cfb0 0x7f1a5403f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.712+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8966080 0x55b8a8a64330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.712+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a64870 con 0x55b8a8966080 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.712+0000 7f1b6c850640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8966080 0x55b8a8a64330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b6c850640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8966080 0x55b8a8a64330 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b6b84e640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5403cfb0 0x7f1a5403f470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b60057000 tx=0x7f1b60052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b6c850640 1 -- 172.21.15.28:0/3901310422 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b640083e0 con 0x55b8a8966080 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a64c10 con 0x55b8a8966080 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b6c850640 1 -- 172.21.15.28:0/3901310422 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b640083e0 con 0x55b8a8966080 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a66710 0x55b8a8a68b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a69320 con 0x55b8a8a66710 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.713+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a66710 0x55b8a8a68b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.714+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a66710 0x55b8a8a68b90 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.714+0000 7f1b6c04f640 1 -- 172.21.15.28:0/3901310422 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f1b5c01b070 con 0x55b8a8a66710 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.714+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a65460 con 0x55b8a8966080 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.714+0000 7f1b6c850640 1 -- 172.21.15.28:0/3901310422 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b640083e0 con 0x55b8a8966080 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.714+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a658b0 con 0x55b8a8a66710 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b6c04f640 1 -- 172.21.15.28:0/3901310422 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f1b5c01b070 con 0x55b8a8a66710 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b7032a800 1 Processor -- start 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b7032a800 1 -- start start 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b7032a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a77cf0 0x55b8a8a78110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b7032a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b8a885e4a0 con 0x55b8a8a77cf0 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b6c850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a77cf0 0x55b8a8a78110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b6c850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a77cf0 0x55b8a8a78110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45480/0 (socket says 172.21.15.28:45480) 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.715+0000 7f1b6c850640 1 -- 172.21.15.28:0/2429523597 learned_addr learned my addr 172.21.15.28:0/2429523597 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.716+0000 7f1b6c850640 1 -- 172.21.15.28:0/2429523597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a8a78750 con 0x55b8a8a77cf0 2024-09-19T11:25:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.716+0000 7f1b6c850640 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a77cf0 0x55b8a8a78110 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55b8a8a5dce0 tx=0x7f1b6400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.716+0000 7f1a51ffb640 1 -- 172.21.15.28:0/2429523597 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b6401c020 con 0x55b8a8a77cf0 2024-09-19T11:25:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.716+0000 7f1a51ffb640 1 -- 172.21.15.28:0/2429523597 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64007c90 con 0x55b8a8a77cf0 2024-09-19T11:25:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.716+0000 7f1a51ffb640 1 -- 172.21.15.28:0/2429523597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64018440 con 0x55b8a8a77cf0 2024-09-19T11:25:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.716+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b8a8a78f80 con 0x55b8a8a77cf0 2024-09-19T11:25:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.716+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a8a79520 con 0x55b8a8a77cf0 2024-09-19T11:25:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.717+0000 7f1a51ffb640 1 -- 172.21.15.28:0/2429523597 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1b6401f020 con 0x55b8a8a77cf0 2024-09-19T11:25:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.717+0000 7f1a51ffb640 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4403d000 0x7f1a4403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.717+0000 7f1a51ffb640 1 -- 172.21.15.28:0/2429523597 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f1b64055ad0 con 0x55b8a8a77cf0 2024-09-19T11:25:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.717+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4403d000 0x7f1a4403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.717+0000 7f1b7032a800 1 Processor -- start 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b7032a800 1 -- start start 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4403d000 0x7f1a4403f4c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c052b50 tx=0x7f1b5c004d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b7032a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a84b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b7032a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b8a8a55bb0 con 0x55b8a8a84790 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b6c850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a84b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b6c850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a84b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45484/0 (socket says 172.21.15.28:45484) 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b6c850640 1 -- 172.21.15.28:0/2056304145 learned_addr learned my addr 172.21.15.28:0/2056304145 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b6c850640 1 -- 172.21.15.28:0/2056304145 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a8a85180 con 0x55b8a8a84790 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1b6c850640 1 --2- 172.21.15.28:0/2056304145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a84b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1b640089f0 tx=0x7f1b64015630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff5c371600ff285 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1a437fe640 1 -- 172.21.15.28:0/2056304145 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b6404a070 con 0x55b8a8a84790 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1a437fe640 1 -- 172.21.15.28:0/2056304145 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b640174d0 con 0x55b8a8a84790 2024-09-19T11:25:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.718+0000 7f1a437fe640 1 -- 172.21.15.28:0/2056304145 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64052070 con 0x55b8a8a84790 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 -- 172.21.15.28:0/2056304145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 msgr2=0x55b8a8a84b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 --2- 172.21.15.28:0/2056304145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a84b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1b640089f0 tx=0x7f1b64015630 comp rx=0 tx=0).stop 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 -- 172.21.15.28:0/2056304145 shutdown_connections 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 --2- 172.21.15.28:0/2056304145 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a84b90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 -- 172.21.15.28:0/2056304145 >> 172.21.15.28:0/2056304145 conn(0x55b8a8a80b90 msgr2=0x55b8a8a7edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 -- 172.21.15.28:0/2056304145 shutdown_connections 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 -- 172.21.15.28:0/2056304145 wait complete. 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 Processor -- start 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 -- start start 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a8bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b7032a800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b8a8a857a0 con 0x55b8a8a84790 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b6c850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a8bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.719+0000 7f1b6c850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a8bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45492/0 (socket says 172.21.15.28:45492) 2024-09-19T11:25:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 learned_addr learned my addr 172.21.15.28:0/1459488933 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a8a8c300 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1b6c850640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a8bda0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1b64016010 tx=0x7f1b64053ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1a41ffb640 1 -- 172.21.15.28:0/1459488933 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b6404a040 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1a41ffb640 1 -- 172.21.15.28:0/1459488933 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64013320 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b8a8a8c590 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1a41ffb640 1 -- 172.21.15.28:0/1459488933 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64013640 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.720+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a8a8d120 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.721+0000 7f1a41ffb640 1 -- 172.21.15.28:0/1459488933 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f1b64016040 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.721+0000 7f1a41ffb640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a3403d000 0x7f1a3403f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.721+0000 7f1a41ffb640 1 -- 172.21.15.28:0/1459488933 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f1b64045850 con 0x55b8a8a84790 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.721+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a3403d000 0x7f1a3403f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a3403d000 0x7f1a3403f4c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c01d850 tx=0x7f1b5c046000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8a97680 0x55b8a8a99b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a9a290 con 0x55b8a8a97680 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b6b84e640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8a97680 0x55b8a8a99b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b6b84e640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8a97680 0x55b8a8a99b40 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b7032a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b7032a800 20 realm 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.722+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a9a6c0 con 0x55b8a8a97680 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a9af90 con 0x55b8a8a97680 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a9b330 con 0x55b8a8a97680 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a9c090 0x55b8a8a9e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a9ea90 con 0x55b8a8a9c090 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.723+0000 7f1b6c850640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a9c090 0x55b8a8a9e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b6c850640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a9c090 0x55b8a8a9e480 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b7032a800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:10.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b7032a800 20 searching for the correct realm 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b6c850640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b64021020 0x7f1b64044f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.724+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b64021020 0x7f1b64044f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.725+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b64021020 0x7f1b64044f20 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.725+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.725+0000 7f1b6c04f640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b5c027020 0x7f1b5c054050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.725+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b5c027020 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.725+0000 7f1b6b84e640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b5c027020 0x7f1b5c054050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.725+0000 7f1b6b84e640 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b5c027020 0x7f1b5c054050 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b5c027020 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.726+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b5c027020 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.727+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b5c027020 2024-09-19T11:25:10.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.728+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.729+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.730+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.730+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.730+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b5c027020 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.730+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.730+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.730+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.730+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b5c027020 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.731+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b5c027020 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a9c090 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x55b8a8a97680 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.732+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa03e0 con 0x7f1b64021020 2024-09-19T11:25:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a9f450 con 0x55b8a8a97680 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8a9fca0 con 0x55b8a8a97680 2024-09-19T11:25:10.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.733+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa0860 con 0x55b8a8a9c090 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 zone default found 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 4 Realm: () 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 zonegroup default 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa0c00 con 0x7f1b5c027020 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa2e90 con 0x55b8a8a9c090 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.734+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa2e90 con 0x55b8a8a9c090 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.735+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.735+0000 7f1b7032a800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.735+0000 7f1b7032a800 20 started sync module instance, tier type = 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.735+0000 7f1b7032a800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.735+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa2320 con 0x7f1b64021020 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.736+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.736+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94251592066880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa32c0 con 0x7f1b64021020 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.736+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa3b10 con 0x55b8a8a97680 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.737+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.737+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94251592066880] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.737+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94251592070752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa3f30 con 0x55b8a8a97680 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.737+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa4780 con 0x55b8a8a9c090 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.738+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94251592070752] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.738+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.738+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94251592072288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa4670 con 0x55b8a8a9c090 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.738+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa51d0 con 0x7f1b64021020 2024-09-19T11:25:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.739+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94251592072288] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.739+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.739+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94251592073408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa55c0 con 0x7f1b64021020 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.739+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa5fe0 con 0x55b8a8a97680 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.740+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.740+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94251592080080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa5fa0 con 0x55b8a8a97680 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.740+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94251592073408] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.740+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa6fb0 con 0x7f1b5c027020 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.741+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.741+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94251592076592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa6fb0 con 0x7f1b5c027020 2024-09-19T11:25:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.741+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa79b0 con 0x7f1b5c027020 2024-09-19T11:25:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.741+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94251592080080] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.742+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94251592076592] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.742+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.742+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94251592084080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa74c0 con 0x7f1b5c027020 2024-09-19T11:25:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.742+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa7e80 con 0x7f1b5c027020 2024-09-19T11:25:10.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(58 notify.7 [create] v33'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94251592084080] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94251592088016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa7df0 con 0x7f1b5c027020 2024-09-19T11:25:10.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 20 add_watcher() i=0 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 20 add_watcher() i=1 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 20 add_watcher() i=2 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 20 add_watcher() i=3 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 20 add_watcher() i=4 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 20 add_watcher() i=5 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.743+0000 7f1b7032a800 20 add_watcher() i=6 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.744+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94251592088016] v33'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.744+0000 7f1b7032a800 20 add_watcher() i=7 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.744+0000 7f1b7032a800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.744+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aab2e0 con 0x55b8a8a97680 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.745+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.745+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94251592082384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8aa8760 con 0x55b8a8a97680 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.746+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94251592082384] v33'269 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.746+0000 7f1a227e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.746+0000 7f1b7032a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.746+0000 7f1b7032a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.746+0000 7f1b7032a800 5 note: GC not initialized 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.746+0000 7f1a217e2640 20 reqs_thread_entry: start 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.746+0000 7f1b7032a800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.747+0000 7f1b7032a800 20 Filter name: none 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.747+0000 7f1992ffd640 20 reqs_thread_entry: start 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.747+0000 7f1b7032a800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.747+0000 7f1b7032a800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.747+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8ade070 con 0x7f1b5c027020 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.748+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.748+0000 7f1b7032a800 20 rados_obj.operate() r=0 bl.length=460 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.748+0000 7f1b7032a800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.748+0000 7f1b7032a800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.748+0000 7f1b7032a800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.748+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8ae44f0 con 0x7f1b5c027020 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v33'12 uv12 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b7032a800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b7032a800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b7032a800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94251592335088 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adfb50 con 0x55b8a8a97680 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 94251592080080 notify 141733920802 ret 0) ==== 691+0+0 (crc 0 0 0) 0x55b8a896f810 con 0x55b8a8a97680 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1a437fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920802 cookie 94251592080080 notifier 4914 bl.length()=649 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1a437fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1a437fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1a437fe640 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7f1a30003ea0 con 0x55b8a8a97680 2024-09-19T11:25:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 94251592335088 out=8b] v0'0 uv276 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.749+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.750+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 94251592335088 notify 141733920802 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f1b6005ba70 con 0x55b8a8a97680 2024-09-19T11:25:10.760 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.750+0000 7f1a227e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.750+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94251592082384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adf220 con 0x55b8a8a97680 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.751+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94251592082384] v33'270 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.751+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94251592066880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adf220 con 0x7f1b64021020 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.752+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94251592066880] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.753+0000 7f1b7032a800 20 remove_watcher() i=0 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.753+0000 7f1b7032a800 2 removed watcher, disabling cache 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.753+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94251592070752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adf220 con 0x55b8a8a97680 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.754+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94251592070752] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.754+0000 7f1b7032a800 20 remove_watcher() i=1 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.754+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94251592072288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adf220 con 0x55b8a8a9c090 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.755+0000 7f1b6c850640 1 -- 172.21.15.28:0/1459488933 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94251592072288] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b64055d70 con 0x55b8a8a9c090 2024-09-19T11:25:10.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.755+0000 7f1b7032a800 20 remove_watcher() i=2 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.755+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94251592073408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adbea0 con 0x7f1b64021020 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.756+0000 7f1b6c04f640 1 -- 172.21.15.28:0/1459488933 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94251592073408] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b5c01b070 con 0x7f1b64021020 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.756+0000 7f1b7032a800 20 remove_watcher() i=3 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.756+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94251592080080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adbea0 con 0x55b8a8a97680 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.757+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94251592080080] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x55b8a8a97680 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.757+0000 7f1b7032a800 20 remove_watcher() i=4 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.757+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94251592076592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adbea0 con 0x7f1b5c027020 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.758+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94251592076592] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.758+0000 7f1b7032a800 20 remove_watcher() i=5 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.758+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94251592084080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adbea0 con 0x7f1b5c027020 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.759+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94251592084080] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.759+0000 7f1b7032a800 20 remove_watcher() i=6 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.759+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94251592088016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x55b8a8adbea0 con 0x7f1b5c027020 2024-09-19T11:25:10.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94251592088016] v33'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b60057030 con 0x7f1b5c027020 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 20 remove_watcher() i=7 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b5c027020 msgr2=0x7f1b5c054050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f1b5c027020 0x7f1b5c054050 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8a97680 msgr2=0x55b8a8a99b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8a97680 0x55b8a8a99b40 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b64021020 msgr2=0x7f1b64044f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f1b64021020 0x7f1b64044f20 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a9c090 msgr2=0x55b8a8a9e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a9c090 0x55b8a8a9e480 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a3403d000 msgr2=0x7f1a3403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a3403d000 0x7f1a3403f4c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c01d850 tx=0x7f1b5c046000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 msgr2=0x55b8a8a8bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a8bda0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1b64016010 tx=0x7f1b64053ce0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b6b84e640 1 -- 172.21.15.28:0/1459488933 reap_dead start 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 shutdown_connections 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 --2- 172.21.15.28:0/1459488933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a84790 0x55b8a8a8bda0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 >> 172.21.15.28:0/1459488933 conn(0x55b8a8a80b90 msgr2=0x55b8a8a7eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 shutdown_connections 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.760+0000 7f1b7032a800 1 -- 172.21.15.28:0/1459488933 wait complete. 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4403d000 msgr2=0x7f1a4403f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4403d000 0x7f1a4403f4c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c052b50 tx=0x7f1b5c004d50 comp rx=0 tx=0).stop 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a77cf0 msgr2=0x55b8a8a78110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a77cf0 0x55b8a8a78110 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55b8a8a5dce0 tx=0x7f1b6400bf30 comp rx=0 tx=0).stop 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 shutdown_connections 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a4403d000 0x7f1a4403f4c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/2429523597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8a77cf0 0x55b8a8a78110 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 >> 172.21.15.28:0/2429523597 conn(0x55b8a8a6c0a0 msgr2=0x55b8a8a6c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 shutdown_connections 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/2429523597 wait complete. 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8966080 msgr2=0x55b8a8a64330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8966080 0x55b8a8a64330 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a66710 msgr2=0x55b8a8a68b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a66710 0x55b8a8a68b90 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5403cfb0 msgr2=0x7f1a5403f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5403cfb0 0x7f1a5403f470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b60057000 tx=0x7f1b60052000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 msgr2=0x55b8a8a5cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a5cce0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000f80 tx=0x7f1b5c001250 comp rx=0 tx=0).stop 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 shutdown_connections 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x55b8a8a66710 0x55b8a8a68b90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f1a5403cfb0 0x7f1a5403f470 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x55b8a8966080 0x55b8a8a64330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 --2- 172.21.15.28:0/3901310422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b8a8965b60 0x55b8a8a5cce0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 >> 172.21.15.28:0/3901310422 conn(0x55b8a89611d0 msgr2=0x55b8a8a56680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 shutdown_connections 2024-09-19T11:25:10.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.761+0000 7f1b7032a800 1 -- 172.21.15.28:0/3901310422 wait complete. 2024-09-19T11:25:10.772 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.772 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.772 ERROR:root:0 2024-09-19T11:25:10.772 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:10.772 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-09-19T11:25:10.773 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-09-19T11:25:10.820 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:25:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c4c0f800 1 Processor -- start 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c4c0f800 1 -- start start 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c4c0f800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558cad876cd0 0x558cad8770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c4c0f800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558cad8784a0 con 0x558cad8777a0 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c4c0f800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x558cad7027d0 con 0x558cad876cd0 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c26f9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558cad876cd0 0x558cad8770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c26f9640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x558cad876cd0 0x558cad8770d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45494/0 (socket says 172.21.15.28:45494) 2024-09-19T11:25:10.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00c26f9640 1 -- 172.21.15.28:0/793294662 learned_addr learned my addr 172.21.15.28:0/793294662 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.820+0000 7f00bbfff640 1 --1- 172.21.15.28:0/793294662 >> v1:172.21.15.22:6789/0 conn(0x558cad8777a0 0x558cad877c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:48378/0 (socket says 172.21.15.28:48378) 2024-09-19T11:25:10.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.821+0000 7f00c26f9640 1 -- 172.21.15.28:0/793294662 >> v1:172.21.15.22:6789/0 conn(0x558cad8777a0 legacy=0x558cad877c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.821+0000 7f00c26f9640 1 -- 172.21.15.28:0/793294662 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad87e720 con 0x558cad876cd0 2024-09-19T11:25:10.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c26f9640 1 --2- 172.21.15.28:0/793294662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad8770d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f00b400a6b0 tx=0x7f00b4047e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e34b4b4f3d2571 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c1ef8640 1 -- 172.21.15.28:0/793294662 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b400a200 con 0x558cad876cd0 2024-09-19T11:25:10.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c1ef8640 1 -- 172.21.15.28:0/793294662 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b400a3a0 con 0x558cad876cd0 2024-09-19T11:25:10.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c1ef8640 1 -- 172.21.15.28:0/793294662 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b4008700 con 0x558cad876cd0 2024-09-19T11:25:10.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c4c0f800 1 -- 172.21.15.28:0/793294662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 msgr2=0x558cad8770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/793294662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad8770d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f00b400a6b0 tx=0x7f00b4047e30 comp rx=0 tx=0).stop 2024-09-19T11:25:10.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c4c0f800 1 -- 172.21.15.28:0/793294662 shutdown_connections 2024-09-19T11:25:10.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/793294662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad8770d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c4c0f800 1 -- 172.21.15.28:0/793294662 >> 172.21.15.28:0/793294662 conn(0x558cad8711d0 msgr2=0x558cad873650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c4c0f800 1 -- 172.21.15.28:0/793294662 shutdown_connections 2024-09-19T11:25:10.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.823+0000 7f00c4c0f800 1 -- 172.21.15.28:0/793294662 wait complete. 2024-09-19T11:25:10.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.839+0000 7f00c4c0f800 1 Processor -- start 2024-09-19T11:25:10.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.839+0000 7f00c4c0f800 1 -- start start 2024-09-19T11:25:10.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.839+0000 7f00c4c0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad965140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.839+0000 7f00c4c0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558cad70fdd0 con 0x558cad876cd0 2024-09-19T11:25:10.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.839+0000 7f00c26f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad965140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.839+0000 7f00c26f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad965140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45506/0 (socket says 172.21.15.28:45506) 2024-09-19T11:25:10.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.839+0000 7f00c26f9640 1 -- 172.21.15.28:0/1928619363 learned_addr learned my addr 172.21.15.28:0/1928619363 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.840+0000 7f00c26f9640 1 -- 172.21.15.28:0/1928619363 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad965730 con 0x558cad876cd0 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.840+0000 7f00c26f9640 1 --2- 172.21.15.28:0/1928619363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad965140 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f00b4048360 tx=0x7f00b4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89c89a6b7b3adf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.840+0000 7effb0608640 1 -- 172.21.15.28:0/1928619363 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b400a200 con 0x558cad876cd0 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.840+0000 7effb0608640 1 -- 172.21.15.28:0/1928619363 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4065070 con 0x558cad876cd0 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.840+0000 7effb0608640 1 -- 172.21.15.28:0/1928619363 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b40086e0 con 0x558cad876cd0 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 -- 172.21.15.28:0/1928619363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 msgr2=0x558cad965140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/1928619363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad965140 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f00b4048360 tx=0x7f00b4051000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 -- 172.21.15.28:0/1928619363 shutdown_connections 2024-09-19T11:25:10.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/1928619363 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad965140 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 -- 172.21.15.28:0/1928619363 >> 172.21.15.28:0/1928619363 conn(0x558cad8711d0 msgr2=0x558cad960ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 -- 172.21.15.28:0/1928619363 shutdown_connections 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 -- 172.21.15.28:0/1928619363 wait complete. 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 Processor -- start 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 -- start start 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad96cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c4c0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558cad87d960 con 0x558cad876cd0 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c26f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad96cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c26f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad96cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45514/0 (socket says 172.21.15.28:45514) 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.841+0000 7f00c26f9640 1 -- 172.21.15.28:0/3468490021 learned_addr learned my addr 172.21.15.28:0/3468490021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.842+0000 7f00c26f9640 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad96d0f0 con 0x558cad876cd0 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.842+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad96cb90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000c00 tx=0x7f00b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.842+0000 7effaee05640 1 -- 172.21.15.28:0/3468490021 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b4008d90 con 0x558cad876cd0 2024-09-19T11:25:10.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.842+0000 7effaee05640 1 -- 172.21.15.28:0/3468490021 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b406f030 con 0x558cad876cd0 2024-09-19T11:25:10.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.842+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558cad96d380 con 0x558cad876cd0 2024-09-19T11:25:10.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.842+0000 7effaee05640 1 -- 172.21.15.28:0/3468490021 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00b404f630 con 0x558cad876cd0 2024-09-19T11:25:10.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.842+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cad96df90 con 0x558cad876cd0 2024-09-19T11:25:10.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.843+0000 7effaee05640 1 -- 172.21.15.28:0/3468490021 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f00b404f7d0 con 0x558cad876cd0 2024-09-19T11:25:10.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.843+0000 7effaee05640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7effa803d000 0x7effa803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.843+0000 7effaee05640 1 -- 172.21.15.28:0/3468490021 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f00b405d040 con 0x558cad876cd0 2024-09-19T11:25:10.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.843+0000 7f00bbfff640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7effa803d000 0x7effa803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.843+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad8777a0 0x558cad9741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.843+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad974720 con 0x558cad8777a0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.843+0000 7f00c2efa640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad8777a0 0x558cad9741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.844+0000 7f00bbfff640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7effa803d000 0x7effa803f4c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00ac001e30 tx=0x7f00ac001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.844+0000 7f00c2efa640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad8777a0 0x558cad9741e0 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.844+0000 7f00c2efa640 1 -- 172.21.15.28:0/3468490021 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad8777a0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.844+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad974ac0 con 0x558cad8777a0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.844+0000 7f00c2efa640 1 -- 172.21.15.28:0/3468490021 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad8777a0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.844+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9765c0 0x558cad978a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.844+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9791d0 con 0x558cad9765c0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.845+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9765c0 0x558cad978a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.845+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9765c0 0x558cad978a40 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.845+0000 7f00c26f9640 1 -- 172.21.15.28:0/3468490021 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f00b406b930 con 0x558cad9765c0 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.845+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad975310 con 0x558cad8777a0 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.845+0000 7f00c2efa640 1 -- 172.21.15.28:0/3468490021 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad8777a0 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.845+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad975760 con 0x558cad9765c0 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c26f9640 1 -- 172.21.15.28:0/3468490021 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f00b406b930 con 0x558cad9765c0 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c4c0f800 1 Processor -- start 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c4c0f800 1 -- start start 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c4c0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad987b40 0x558cad987f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c4c0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558cad76e4a0 con 0x558cad987b40 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c2efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad987b40 0x558cad987f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c2efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad987b40 0x558cad987f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45522/0 (socket says 172.21.15.28:45522) 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.846+0000 7f00c2efa640 1 -- 172.21.15.28:0/2726204846 learned_addr learned my addr 172.21.15.28:0/2726204846 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.847+0000 7f00c2efa640 1 -- 172.21.15.28:0/2726204846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad9885a0 con 0x558cad987b40 2024-09-19T11:25:10.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.847+0000 7f00c2efa640 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad987b40 0x558cad987f60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x558cad96db90 tx=0x7f00bc0167b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.847+0000 7effa7fff640 1 -- 172.21.15.28:0/2726204846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00bc013640 con 0x558cad987b40 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.847+0000 7effa7fff640 1 -- 172.21.15.28:0/2726204846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc0137e0 con 0x558cad987b40 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.847+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558cad988dd0 con 0x558cad987b40 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.847+0000 7effa7fff640 1 -- 172.21.15.28:0/2726204846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00bc013ae0 con 0x558cad987b40 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.847+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cad989370 con 0x558cad987b40 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.848+0000 7effa7fff640 1 -- 172.21.15.28:0/2726204846 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f00bc011040 con 0x558cad987b40 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.848+0000 7effa7fff640 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff9c03d050 0x7eff9c03f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.848+0000 7effa7fff640 1 -- 172.21.15.28:0/2726204846 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f00bc00e040 con 0x558cad987b40 2024-09-19T11:25:10.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.848+0000 7f00c26f9640 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff9c03d050 0x7eff9c03f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c4c0f800 1 Processor -- start 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c4c0f800 1 -- start start 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c26f9640 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff9c03d050 0x7eff9c03f510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00b406bc40 tx=0x7f00b40565a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c4c0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad9949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c4c0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558cad965aa0 con 0x558cad9945e0 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c2efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad9949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c2efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad9949c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45534/0 (socket says 172.21.15.28:45534) 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c2efa640 1 -- 172.21.15.28:0/3693311951 learned_addr learned my addr 172.21.15.28:0/3693311951 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c2efa640 1 -- 172.21.15.28:0/3693311951 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad994fb0 con 0x558cad9945e0 2024-09-19T11:25:10.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.849+0000 7f00c2efa640 1 --2- 172.21.15.28:0/3693311951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad9949c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0547f0 tx=0x7f00bc01fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d466431cb9eaa7be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7effa5ffb640 1 -- 172.21.15.28:0/3693311951 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00bc051020 con 0x558cad9945e0 2024-09-19T11:25:10.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7effa5ffb640 1 -- 172.21.15.28:0/3693311951 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc0143c0 con 0x558cad9945e0 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7effa5ffb640 1 -- 172.21.15.28:0/3693311951 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00bc01f490 con 0x558cad9945e0 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3693311951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 msgr2=0x558cad9949c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3693311951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad9949c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0547f0 tx=0x7f00bc01fe70 comp rx=0 tx=0).stop 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3693311951 shutdown_connections 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3693311951 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad9949c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3693311951 >> 172.21.15.28:0/3693311951 conn(0x558cad9909e0 msgr2=0x558cad98ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3693311951 shutdown_connections 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3693311951 wait complete. 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 Processor -- start 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.850+0000 7f00c4c0f800 1 -- start start 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c4c0f800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad99bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c4c0f800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558cad9955f0 con 0x558cad9945e0 2024-09-19T11:25:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c2efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad99bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c2efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad99bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:45542/0 (socket says 172.21.15.28:45542) 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 learned_addr learned my addr 172.21.15.28:0/3082295754 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad99c170 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c2efa640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad99bc10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f00bc021c70 tx=0x7f00bc054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7eff93fff640 1 -- 172.21.15.28:0/3082295754 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00bc021570 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7eff93fff640 1 -- 172.21.15.28:0/3082295754 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00bc0459c0 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558cad99c400 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7eff93fff640 1 -- 172.21.15.28:0/3082295754 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00bc045cc0 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.851+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cad99cf90 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.852+0000 7eff93fff640 1 -- 172.21.15.28:0/3082295754 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f00bc054930 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.852+0000 7eff93fff640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff8c03d000 0x7eff8c03f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.852+0000 7eff93fff640 1 -- 172.21.15.28:0/3082295754 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f00bc017450 con 0x558cad9945e0 2024-09-19T11:25:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.852+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff8c03d000 0x7eff8c03f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.853+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff8c03d000 0x7eff8c03f4c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f00b4073f90 tx=0x7f00b407d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.853+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.853+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad9a7480 0x558cad9a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.853+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9aa090 con 0x558cad9a7480 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.853+0000 7f00bbfff640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad9a7480 0x558cad9a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00bbfff640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad9a7480 0x558cad9a9940 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f00ac00f610 con 0x558cad9a7480 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 20 realm 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9a67d0 con 0x558cad9a7480 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f00ac00f610 con 0x558cad9a7480 2024-09-19T11:25:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9aab80 con 0x558cad9a7480 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f00ac00f610 con 0x558cad9a7480 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ab450 con 0x558cad9a7480 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.854+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f00ac00f610 con 0x558cad9a7480 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c4c0f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9abe70 0x558cad9ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ae9a0 con 0x558cad9abe70 2024-09-19T11:25:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c2efa640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9abe70 0x558cad9ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c2efa640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9abe70 0x558cad9ae260 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c4c0f800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c4c0f800 20 searching for the correct realm 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.855+0000 7f00c2efa640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f00bc054c20 0x7f00bc0080c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.856+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.856+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f00bc054c20 0x7f00bc0080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.856+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f00bc054c20 0x7f00bc0080c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.856+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.856+0000 7f00c26f9640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f00b406a3f0 0x7f00b40567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.856+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.856+0000 7f00bbfff640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f00b406a3f0 0x7f00b40567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.857+0000 7f00bbfff640 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f00b406a3f0 0x7f00b40567d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.857+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac00f610 con 0x7f00b406a3f0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.857+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.857+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac00f610 con 0x558cad9a7480 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.857+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.858+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.859+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.860+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.861+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.862+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.862+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.862+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.862+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.862+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.862+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.863+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.863+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.863+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.863+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.863+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00ac02b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9abe70 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x558cad9a7480 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.864+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b02d0 con 0x7f00bc054c20 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:25:10.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9af360 con 0x558cad9a7480 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9afbb0 con 0x558cad9a7480 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f00ac02b040 con 0x558cad9a7480 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:25:10.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.865+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b0750 con 0x558cad9abe70 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 zone default found 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 4 Realm: () 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 zonegroup default 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b0b10 con 0x7f00b406a3f0 2024-09-19T11:25:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f00ac02b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b2df0 con 0x558cad9abe70 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b3190 con 0x558cad9abe70 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.866+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.867+0000 7f00c4c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.867+0000 7f00c4c0f800 20 started sync module instance, tier type = 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.867+0000 7f00c4c0f800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:25:10.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.867+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b2230 con 0x7f00bc054c20 2024-09-19T11:25:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.868+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.868+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94062696406400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b35a0 con 0x7f00bc054c20 2024-09-19T11:25:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.868+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b3df0 con 0x558cad9a7480 2024-09-19T11:25:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.869+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94062696406400] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.869+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.869+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94062696413504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b4210 con 0x558cad9a7480 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.869+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b4a60 con 0x558cad9abe70 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.870+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94062696413504] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.870+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.870+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94062696415040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b4990 con 0x558cad9abe70 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.870+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b5500 con 0x7f00bc054c20 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.871+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.871+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94062696419312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b5da0 con 0x7f00bc054c20 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.871+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b68a0 con 0x558cad9a7480 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.871+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94062696415040] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.872+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.872+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94062696424336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b6cc0 con 0x558cad9a7480 2024-09-19T11:25:10.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.872+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b76e0 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.872+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94062696419312] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.873+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94062696424336] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.873+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.873+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94062696416288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b76a0 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.873+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b7f80 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.875+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94062696416288] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.875+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.875+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94062696427984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b76a0 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.875+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b85e0 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94062696427984] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94062696430208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b85a0 con 0x7f00b406a3f0 2024-09-19T11:25:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 20 add_watcher() i=0 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 20 add_watcher() i=1 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 20 add_watcher() i=2 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 20 add_watcher() i=3 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 20 add_watcher() i=4 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 20 add_watcher() i=5 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.876+0000 7f00c4c0f800 20 add_watcher() i=6 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.877+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94062696430208] v33'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.877+0000 7f00c4c0f800 20 add_watcher() i=7 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.877+0000 7f00c4c0f800 2 all 8 watchers are set, enabling cache 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.877+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9bb9d0 con 0x558cad9a7480 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.877+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.877+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94062696432320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9b8f20 con 0x558cad9a7480 2024-09-19T11:25:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.878+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94062696432320] v33'271 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.878+0000 7eff78fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.878+0000 7f00c4c0f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.878+0000 7f00c4c0f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.878+0000 7f00c4c0f800 5 note: GC not initialized 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.879+0000 7efef37fe640 20 reqs_thread_entry: start 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.879+0000 7f00c4c0f800 20 init_complete bucket index max shards: 11 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.879+0000 7f00c4c0f800 20 Filter name: none 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.879+0000 7efef17fa640 20 reqs_thread_entry: start 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.879+0000 7f00c4c0f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.879+0000 7f00c4c0f800 20 rados->read ofs=0 len=0 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.879+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ee940 con 0x7f00b406a3f0 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.880+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv12 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.880+0000 7f00c4c0f800 20 rados_obj.operate() r=0 bl.length=473 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.880+0000 7f00c4c0f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.880+0000 7f00c4c0f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.880+0000 7f00c4c0f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.880+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9f50d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v33'13 uv13 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7f00c4c0f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7f00c4c0f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7f00c4c0f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94062696680144 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x558cad9f0770 con 0x558cad9a7480 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== watch-notify(notify (1) cookie 94062696424336 notify 141733920803 ret 0) ==== 691+0+0 (crc 0 0 0) 0x7f00ac018040 con 0x558cad9a7480 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7effa5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920803 cookie 94062696424336 notifier 4921 bl.length()=649 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7effa5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-19T11:25:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7effa5ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-19T11:25:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7effa5ffb640 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x7eff88003ea0 con 0x558cad9a7480 2024-09-19T11:25:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.881+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(64 notify.4 [notify cookie 94062696680144 out=8b] v0'0 uv279 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.882+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.882+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== watch-notify(notify_complete (2) cookie 94062696680144 notify 141733920803 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f00ac009050 con 0x558cad9a7480 2024-09-19T11:25:10.893 INFO:teuthology.orchestra.run.smithi028.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-09-19T11:25:08.833060Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-09-19T11:25:08.833045Z","tags":[],"group_ids":[]} 2024-09-19T11:25:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.883+0000 7eff78fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:25:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.883+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94062696432320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9f0f30 con 0x558cad9a7480 2024-09-19T11:25:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.884+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94062696432320] v33'272 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.884+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94062696406400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9f0f30 con 0x7f00bc054c20 2024-09-19T11:25:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.885+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94062696406400] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.885+0000 7f00c4c0f800 20 remove_watcher() i=0 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.885+0000 7f00c4c0f800 2 removed watcher, disabling cache 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.885+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94062696413504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ec7d0 con 0x558cad9a7480 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.886+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94062696413504] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.886+0000 7f00c4c0f800 20 remove_watcher() i=1 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.886+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94062696415040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ec7d0 con 0x558cad9abe70 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.887+0000 7f00c2efa640 1 -- 172.21.15.28:0/3082295754 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94062696415040] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00bc00a4d0 con 0x558cad9abe70 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.887+0000 7f00c4c0f800 20 remove_watcher() i=2 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.887+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94062696419312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ec7d0 con 0x7f00bc054c20 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.888+0000 7f00c26f9640 1 -- 172.21.15.28:0/3082295754 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94062696419312] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00b406b930 con 0x7f00bc054c20 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.888+0000 7f00c4c0f800 20 remove_watcher() i=3 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.888+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94062696424336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ec7d0 con 0x558cad9a7480 2024-09-19T11:25:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.889+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94062696424336] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x558cad9a7480 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.889+0000 7f00c4c0f800 20 remove_watcher() i=4 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.889+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94062696416288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ec7d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.890+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94062696416288] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.890+0000 7f00c4c0f800 20 remove_watcher() i=5 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.890+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94062696427984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ec7d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.891+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94062696427984] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.891+0000 7f00c4c0f800 20 remove_watcher() i=6 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.891+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94062696430208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x558cad9ec7d0 con 0x7f00b406a3f0 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94062696430208] v33'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f00ac03b040 con 0x7f00b406a3f0 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 20 remove_watcher() i=7 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f00b406a3f0 msgr2=0x7f00b40567d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f00b406a3f0 0x7f00b40567d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad9a7480 msgr2=0x558cad9a9940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad9a7480 0x558cad9a9940 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f00bc054c20 msgr2=0x7f00bc0080c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f00bc054c20 0x7f00bc0080c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9abe70 msgr2=0x558cad9ae260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9abe70 0x558cad9ae260 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff8c03d000 msgr2=0x7eff8c03f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.892+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff8c03d000 0x7eff8c03f4c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f00b4073f90 tx=0x7f00b407d000 comp rx=0 tx=0).stop 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00bbfff640 1 -- 172.21.15.28:0/3082295754 reap_dead start 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 msgr2=0x558cad99bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad99bc10 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f00bc021c70 tx=0x7f00bc054780 comp rx=0 tx=0).stop 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 shutdown_connections 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3082295754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad9945e0 0x558cad99bc10 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 >> 172.21.15.28:0/3082295754 conn(0x558cad9909e0 msgr2=0x558cad98ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 shutdown_connections 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3082295754 wait complete. 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff9c03d050 msgr2=0x7eff9c03f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff9c03d050 0x7eff9c03f510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00b406bc40 tx=0x7f00b40565a0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad987b40 msgr2=0x558cad987f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad987b40 0x558cad987f60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x558cad96db90 tx=0x7f00bc0167b0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 shutdown_connections 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7eff9c03d050 0x7eff9c03f510 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/2726204846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad987b40 0x558cad987f60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 >> 172.21.15.28:0/2726204846 conn(0x558cad97bef0 msgr2=0x558cad97c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 shutdown_connections 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/2726204846 wait complete. 2024-09-19T11:25:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad8777a0 msgr2=0x558cad9741e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad8777a0 0x558cad9741e0 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9765c0 msgr2=0x558cad978a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.893+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9765c0 0x558cad978a40 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7effa803d000 msgr2=0x7effa803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7effa803d000 0x7effa803f4c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f00ac001e30 tx=0x7f00ac001390 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 msgr2=0x558cad96cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad96cb90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000c00 tx=0x7f00b4002fb0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 shutdown_connections 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x558cad9765c0 0x558cad978a40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7effa803d000 0x7effa803f4c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x558cad8777a0 0x558cad9741e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 --2- 172.21.15.28:0/3468490021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558cad876cd0 0x558cad96cb90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 >> 172.21.15.28:0/3468490021 conn(0x558cad8711d0 msgr2=0x558cad966530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:25:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 shutdown_connections 2024-09-19T11:25:10.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:25:10.894+0000 7f00c4c0f800 1 -- 172.21.15.28:0/3468490021 wait complete. 2024-09-19T11:25:10.904 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.905 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-09-19T11:25:08.833060Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-09-19T11:25:08.833045Z', 'tags': [], 'group_ids': []} 2024-09-19T11:25:10.905 ERROR:root:0 2024-09-19T11:25:10.905 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-09-19T11:25:10.920 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:10.920 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:10.934 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:10.935 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-09-19T11:25:10.949 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:10.949 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:10.962 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:10.962 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2024-09-19T11:25:10.978 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}] 2024-09-19T11:25:10.978 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:10.989 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:10.990 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2024-09-19T11:25:11.005 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.005 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.017 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:11.017 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2024-09-19T11:25:11.034 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2024-09-19T11:25:11.034 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.046 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:11.047 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2024-09-19T11:25:11.062 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2024-09-19T11:25:11.062 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.074 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:11.075 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2024-09-19T11:25:11.089 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.090 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.101 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:11.102 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2024-09-19T11:25:11.116 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.116 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.127 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:11.127 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2024-09-19T11:25:11.143 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.143 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.155 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:11.155 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.167 INFO:tasks.radosgw_admin_rest: json result: [] 2024-09-19T11:25:11.169 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-09-19T11:25:11.181 INFO:tasks.radosgw_admin_rest: json result: [] 2024-09-19T11:25:11.200 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-09-19T11:25:11.214 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2024-09-19T11:25:11.214 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-09-19T11:25:11.227 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:25:11.196031Z', 'creation_time': '2024-09-19T11:25:11.193494Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 1} 2024-09-19T11:25:11.227 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-09-19T11:25:11.240 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:25:11.196031Z', 'creation_time': '2024-09-19T11:25:11.193494Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 1}] 2024-09-19T11:25:11.245 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-09-19T11:25:11.260 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:25:11.196031Z', 'creation_time': '2024-09-19T11:25:11.193494Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 1} 2024-09-19T11:25:11.261 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2024-09-19T11:25:11.273 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'Message': '', 'RequestId': 'tx000003a059ce524e989a7-0066ec0a17-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:25:11.278 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2024-09-19T11:25:11.293 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.294 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2024-09-19T11:25:11.311 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'fud', 'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:25:11.320 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11'} 2024-09-19T11:25:11.339 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.385 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11'} 2024-09-19T11:25:11.405 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.406 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2024-09-19T11:25:11.424 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.439 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2024-09-19T11:25:11.455 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:25:11.455 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-09-19T11:25:11.468 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'd3056e04-d444-48b9-8e61-96c58b445cdd', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'marker': '293899f1-2738-4b64-a8e1-a18ff039d626.4229.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'reshard_status': 'None', 'judge_reshard_lock_time': '0.000000', 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-19T11:25:11.399662Z', 'creation_time': '2024-09-19T11:25:11.193494Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'read_tracker': 3} 2024-09-19T11:26:11.514 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2024-09-19T11:26:11.542 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'anonymous', 'categories': [], 's3select': {}}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'bar', 'categories': [], 's3select': {}}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'bud', 'categories': [], 's3select': {}}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'fud', 'categories': [], 's3select': {}}, {'bucket': 'myfoo7', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'fud', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:26:11.542 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2024-09-19T11:26:11.566 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'useless-bucket', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo7', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 242, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:26:11.567 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-09-19T11:26:11.579 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'useless-bucket', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:26:11.580 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2024-09-19T11:26:11.593 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:26:11.593 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2024-09-19T11:26:11.606 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:26:11.607 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2024-09-19T11:26:11.619 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:26:11.620 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2024-09-19T11:26:11.634 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-09-19T11:00:00.000000Z', 'epoch': 1726743600, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-09-19T11:26:11.635 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2024-09-19T11:26:11.650 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:26:11.650 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-09-19T11:26:11.663 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2024-09-19T11:26:11.663 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-09-19T11:26:11.681 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:26:11.693 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-09-19T11:26:11.711 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:26:11.975 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-09-19T11:26:11.987 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000dacb1cd52e7b0721-0066ec0a53-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.089 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-09-19T11:26:12.102 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-09-19T11:26:12.107 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-09-19T11:26:12.119 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'group': 1, 'permission': {'flags': 1}}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-09-19T11:26:12.233 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2024-09-19T11:26:12.277 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:26:12.278 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-09-19T11:26:12.292 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2024-09-19T11:26:12.292 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-09-19T11:26:12.306 INFO:tasks.radosgw_admin_rest: json result: [] 2024-09-19T11:26:12.317 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-09-19T11:26:12.329 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000a313217bedac94a8-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.346 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2024-09-19T11:26:12.384 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:26:12.384 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-19T11:26:12.397 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx00000a30a3dc8de9629e1-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.397 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2024-09-19T11:26:12.410 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': '200d5f3b-d410-4b92-82cf-1cbe85431f50'}]}} 2024-09-19T11:26:12.411 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2024-09-19T11:26:12.430 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'ratelimit_user', 'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-19T11:26:12.430 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-09-19T11:26:12.442 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-09-19T11:26:12.442 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2024-09-19T11:26:12.454 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx0000050128f40a4acc2df-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.454 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2024-09-19T11:26:12.466 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000004add615decd2928f-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.475 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-09-19T11:26:12.487 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-09-19T11:26:12.487 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2024-09-19T11:26:12.499 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000006b53b95c5e0fa092-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.500 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2024-09-19T11:26:12.512 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000006be98d43b4ce16f1-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.512 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2024-09-19T11:26:12.525 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-09-19T11:26:12.525 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2024-09-19T11:26:12.537 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx0000076093cf958e10df0-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.537 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2024-09-19T11:26:12.549 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000cdfb317294ed465e-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.549 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2024-09-19T11:26:12.561 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx0000007ddc3d1c14f18c6-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.561 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2024-09-19T11:26:12.572 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000cd82dad3ddd12092-0066ec0a54-4229-default', 'HostId': '4229-default-default'} 2024-09-19T11:26:12.572 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2024-09-19T11:26:12.590 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:26:12.590 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-09-19T11:26:12.602 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2024-09-19T11:26:12.602 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2024-09-19T11:26:12.616 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:26:12.616 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-09-19T11:26:12.628 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2024-09-19T11:26:12.628 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2024-09-19T11:26:12.642 INFO:tasks.radosgw_admin_rest: empty result 2024-09-19T11:26:12.647 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-09-19T11:26:12.660 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-09-19T11:26:12.660 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T11:26:12.661 INFO:tasks.rgw.client.0.smithi028.stdout:2024-09-19T11:26:12.659+0000 7fb0ffbb6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 34907) UID: 0 2024-09-19T11:26:12.661 INFO:tasks.rgw.client.0.smithi028.stdout:2024-09-19T11:26:12.659+0000 7fb103416880 -1 shutting down 2024-09-19T11:26:16.011 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-09-19T11:26:18.761 INFO:tasks.rgw.client.0:Stopped 2024-09-19T11:26:18.762 DEBUG:teuthology.orchestra.run.smithi028:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-09-19T11:26:18.782 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f /etc/ceph/vault-root-token 2024-09-19T11:26:18.854 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-09-19T11:26:18.924 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-09-19T11:26:18.924 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-09-19T11:26:18.924 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-09-19T11:26:19.014 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setuser ceph since I am not root 2024-09-19T11:26:19.015 INFO:teuthology.orchestra.run.smithi028.stderr:ignoring --setgroup ceph since I am not root 2024-09-19T11:26:19.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.013+0000 7f5a13240800 1 Processor -- start 2024-09-19T11:26:19.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.013+0000 7f5a13240800 1 -- start start 2024-09-19T11:26:19.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a13240800 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5621b05e87a0 0x5621b05e8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a13240800 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5621b05e92b0 con 0x5621b05e7ba0 2024-09-19T11:26:19.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a13240800 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x5621b04737d0 con 0x5621b05e87a0 2024-09-19T11:26:19.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a10529640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x5621b05e87a0 0x5621b05e8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a10d2a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x5621b05e7ba0 0x5621b05e7fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.28:59574/0 (socket says 172.21.15.28:59574) 2024-09-19T11:26:19.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2493595572 learned_addr learned my addr 172.21.15.28:0/2493595572 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-19T11:26:19.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a10529640 1 -- 172.21.15.28:0/2493595572 >> v1:172.21.15.22:6789/0 conn(0x5621b05e7ba0 legacy=0x5621b05e7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a0fd28640 1 -- 172.21.15.28:0/2493595572 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2711466882 0 0) 0x5621b05e92b0 con 0x5621b05e7ba0 2024-09-19T11:26:19.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.014+0000 7f5a10529640 1 -- 172.21.15.28:0/2493595572 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b05ef750 con 0x5621b05e87a0 2024-09-19T11:26:19.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.017+0000 7f5a10529640 1 --2- 172.21.15.28:0/2493595572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e87a0 0x5621b05e8c50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400a470 tx=0x7f5a04045a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247d0283846cad3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.017+0000 7f5a0fd28640 1 -- 172.21.15.28:0/2493595572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a04053570 con 0x5621b05e87a0 2024-09-19T11:26:19.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.017+0000 7f5a0fd28640 1 -- 172.21.15.28:0/2493595572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a04058070 con 0x5621b05e87a0 2024-09-19T11:26:19.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.017+0000 7f5a0fd28640 1 -- 172.21.15.28:0/2493595572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a04053880 con 0x5621b05e87a0 2024-09-19T11:26:19.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.018+0000 7f5a13240800 1 -- 172.21.15.28:0/2493595572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e87a0 msgr2=0x5621b05e8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.018+0000 7f5a13240800 1 --2- 172.21.15.28:0/2493595572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e87a0 0x5621b05e8c50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400a470 tx=0x7f5a04045a20 comp rx=0 tx=0).stop 2024-09-19T11:26:19.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.018+0000 7f5a13240800 1 -- 172.21.15.28:0/2493595572 shutdown_connections 2024-09-19T11:26:19.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.018+0000 7f5a13240800 1 --2- 172.21.15.28:0/2493595572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e87a0 0x5621b05e8c50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.018+0000 7f5a13240800 1 -- 172.21.15.28:0/2493595572 >> 172.21.15.28:0/2493595572 conn(0x5621b05e21c0 msgr2=0x5621b05e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.018+0000 7f5a13240800 1 -- 172.21.15.28:0/2493595572 shutdown_connections 2024-09-19T11:26:19.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.018+0000 7f5a13240800 1 -- 172.21.15.28:0/2493595572 wait complete. 2024-09-19T11:26:19.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.035+0000 7f5a13240800 1 Processor -- start 2024-09-19T11:26:19.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.035+0000 7f5a13240800 1 -- start start 2024-09-19T11:26:19.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.035+0000 7f5a13240800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b05e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.035+0000 7f5a13240800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5621b0480dd0 con 0x5621b05e7ba0 2024-09-19T11:26:19.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f5a10d2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b05e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f5a10d2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b05e6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:40612/0 (socket says 172.21.15.28:40612) 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f5a10d2a640 1 -- 172.21.15.28:0/726605841 learned_addr learned my addr 172.21.15.28:0/726605841 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f5a10d2a640 1 -- 172.21.15.28:0/726605841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b05e7310 con 0x5621b05e7ba0 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/726605841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b05e6d20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006fa0 tx=0x7f5a00001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393382b3e4c6a201 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f58fee0d640 1 -- 172.21.15.28:0/726605841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a00003450 con 0x5621b05e7ba0 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f58fee0d640 1 -- 172.21.15.28:0/726605841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a00003610 con 0x5621b05e7ba0 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.036+0000 7f58fee0d640 1 -- 172.21.15.28:0/726605841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a00003970 con 0x5621b05e7ba0 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 -- 172.21.15.28:0/726605841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 msgr2=0x5621b05e6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 --2- 172.21.15.28:0/726605841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b05e6d20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5a00006fa0 tx=0x7f5a00001b00 comp rx=0 tx=0).stop 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 -- 172.21.15.28:0/726605841 shutdown_connections 2024-09-19T11:26:19.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 --2- 172.21.15.28:0/726605841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b05e6d20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 -- 172.21.15.28:0/726605841 >> 172.21.15.28:0/726605841 conn(0x5621b05e21c0 msgr2=0x5621b06d2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 -- 172.21.15.28:0/726605841 shutdown_connections 2024-09-19T11:26:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 -- 172.21.15.28:0/726605841 wait complete. 2024-09-19T11:26:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.037+0000 7f5a13240800 1 Processor -- start 2024-09-19T11:26:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a13240800 1 -- start start 2024-09-19T11:26:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a13240800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b06d84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a13240800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5621b05eea40 con 0x5621b05e7ba0 2024-09-19T11:26:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a10d2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b06d84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a10d2a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b06d84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:40626/0 (socket says 172.21.15.28:40626) 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a10d2a640 1 -- 172.21.15.28:0/1760016351 learned_addr learned my addr 172.21.15.28:0/1760016351 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a10d2a640 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b06d8a30 con 0x5621b05e7ba0 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.038+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b06d84d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5a00001330 tx=0x7f5a00001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.039+0000 7f58fd60a640 1 -- 172.21.15.28:0/1760016351 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a00014a10 con 0x5621b05e7ba0 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.039+0000 7f58fd60a640 1 -- 172.21.15.28:0/1760016351 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a00014bb0 con 0x5621b05e7ba0 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.039+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5621b06d6de0 con 0x5621b05e7ba0 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.039+0000 7f58fd60a640 1 -- 172.21.15.28:0/1760016351 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a0001e3f0 con 0x5621b05e7ba0 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.039+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b06d73a0 con 0x5621b05e7ba0 2024-09-19T11:26:19.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.040+0000 7f58fd60a640 1 -- 172.21.15.28:0/1760016351 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5a00010070 con 0x5621b05e7ba0 2024-09-19T11:26:19.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.040+0000 7f58fd60a640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58f803d000 0x7f58f803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.040+0000 7f58fd60a640 1 -- 172.21.15.28:0/1760016351 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f5a00018070 con 0x5621b05e7ba0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.040+0000 7f5a10529640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58f803d000 0x7f58f803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.040+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b05e87a0 0x5621b06e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.040+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b06e9760 con 0x5621b05e87a0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.040+0000 7f5a1152b640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b05e87a0 0x5621b06e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.041+0000 7f5a10529640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58f803d000 0x7f58f803f4c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a04057000 tx=0x7f5a04052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.041+0000 7f5a1152b640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b05e87a0 0x5621b06e90f0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.041+0000 7f5a1152b640 1 -- 172.21.15.28:0/1760016351 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5a080083e0 con 0x5621b05e87a0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.041+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b06ea040 con 0x5621b05e87a0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.042+0000 7f5a1152b640 1 -- 172.21.15.28:0/1760016351 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5a080083e0 con 0x5621b05e87a0 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.042+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b06eba70 0x5621b06edec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.042+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:3 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b06ee650 con 0x5621b06eba70 2024-09-19T11:26:19.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.042+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b06eba70 0x5621b06edec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.042+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b06eba70 0x5621b06edec0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.042+0000 7f5a10d2a640 1 -- 172.21.15.28:0/1760016351 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(3 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [call out=48b,read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5a0001b070 con 0x5621b06eba70 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.042+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b06ea890 con 0x5621b05e87a0 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.043+0000 7f5a1152b640 1 -- 172.21.15.28:0/1760016351 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5a080083e0 con 0x5621b05e87a0 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.043+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b06eac30 con 0x5621b06eba70 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.043+0000 7f5a10d2a640 1 -- 172.21.15.28:0/1760016351 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(5 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [call out=48b,read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5a0001b070 con 0x5621b06eba70 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.043+0000 7f5a13240800 1 Processor -- start 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.043+0000 7f5a13240800 1 -- start start 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.044+0000 7f5a13240800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b06fcee0 0x5621b06fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.044+0000 7f5a13240800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5621b04df4a0 con 0x5621b06fcee0 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.044+0000 7f5a1152b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b06fcee0 0x5621b06fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.044+0000 7f5a1152b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b06fcee0 0x5621b06fd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:40634/0 (socket says 172.21.15.28:40634) 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.044+0000 7f5a1152b640 1 -- 172.21.15.28:0/3504267817 learned_addr learned my addr 172.21.15.28:0/3504267817 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.044+0000 7f5a1152b640 1 -- 172.21.15.28:0/3504267817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b06fd940 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.045+0000 7f5a1152b640 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b06fcee0 0x5621b06fd300 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x5621b06d77f0 tx=0x7f5a0800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.045+0000 7f58f67fc640 1 -- 172.21.15.28:0/3504267817 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a0801c020 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.045+0000 7f58f67fc640 1 -- 172.21.15.28:0/3504267817 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a08007c90 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.045+0000 7f58f67fc640 1 -- 172.21.15.28:0/3504267817 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a08018440 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.045+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5621b06fe170 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.045+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b06fe710 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.046+0000 7f58f67fc640 1 -- 172.21.15.28:0/3504267817 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5a0801f020 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.046+0000 7f58f67fc640 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58e803d000 0x7f58e803f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.046+0000 7f58f67fc640 1 -- 172.21.15.28:0/3504267817 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f5a08055b00 con 0x5621b06fcee0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.046+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58e803d000 0x7f58e803f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.046+0000 7f5a13240800 1 Processor -- start 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.046+0000 7f5a13240800 1 -- start start 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58e803d000 0x7f58e803f4c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5621b06ee9f0 tx=0x7f5a00014fb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a13240800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0709e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a13240800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5621b06db040 con 0x5621b0709a00 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a1152b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0709e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a1152b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0709e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:40636/0 (socket says 172.21.15.28:40636) 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a1152b640 1 -- 172.21.15.28:0/3982952674 learned_addr learned my addr 172.21.15.28:0/3982952674 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a1152b640 1 -- 172.21.15.28:0/3982952674 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b070a3f0 con 0x5621b0709a00 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.047+0000 7f5a1152b640 1 --2- 172.21.15.28:0/3982952674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0709e00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5a08053d80 tx=0x7f5a08054050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790f44414e5903db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f58e7fff640 1 -- 172.21.15.28:0/3982952674 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a08052070 con 0x5621b0709a00 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f58e7fff640 1 -- 172.21.15.28:0/3982952674 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a080156f0 con 0x5621b0709a00 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f58e7fff640 1 -- 172.21.15.28:0/3982952674 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a08016c50 con 0x5621b0709a00 2024-09-19T11:26:19.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 -- 172.21.15.28:0/3982952674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 msgr2=0x5621b0709e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 --2- 172.21.15.28:0/3982952674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0709e00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f5a08053d80 tx=0x7f5a08054050 comp rx=0 tx=0).stop 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 -- 172.21.15.28:0/3982952674 shutdown_connections 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 --2- 172.21.15.28:0/3982952674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0709e00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 -- 172.21.15.28:0/3982952674 >> 172.21.15.28:0/3982952674 conn(0x5621b0705e00 msgr2=0x5621b0704040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 -- 172.21.15.28:0/3982952674 shutdown_connections 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 -- 172.21.15.28:0/3982952674 wait complete. 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.048+0000 7f5a13240800 1 Processor -- start 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a13240800 1 -- start start 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a13240800 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0710f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a13240800 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5621b070a9e0 con 0x5621b0709a00 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a1152b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0710f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a1152b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0710f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.28:40642/0 (socket says 172.21.15.28:40642) 2024-09-19T11:26:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 learned_addr learned my addr 172.21.15.28:0/2702657944 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621b07114b0 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f5a1152b640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0710f50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5a08017f70 tx=0x7f5a08053d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.049+0000 7f58e67fc640 1 -- 172.21.15.28:0/2702657944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a08013660 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.050+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5621b0711740 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.050+0000 7f58e67fc640 1 -- 172.21.15.28:0/2702657944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0802e030 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.050+0000 7f58e67fc640 1 -- 172.21.15.28:0/2702657944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a080139e0 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.050+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621b07122d0 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.050+0000 7f58e67fc640 1 -- 172.21.15.28:0/2702657944 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f5a08016040 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.051+0000 7f58e67fc640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58dc03cfb0 0x7f58dc03f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.051+0000 7f58e67fc640 1 -- 172.21.15.28:0/2702657944 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f5a08033ae0 con 0x5621b0709a00 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.051+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58dc03cfb0 0x7f58dc03f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.051+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58dc03cfb0 0x7f58dc03f470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a00002020 tx=0x7f5a00035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b071c930 0x5621b071edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b071f540 con 0x5621b071c930 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a10529640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b071c930 0x5621b071edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a10529640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b071c930 0x5621b071edf0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a13240800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a13240800 20 realm 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.052+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b071f970 con 0x5621b071c930 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-19T11:26:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0720240 con 0x5621b071c930 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b07205e0 con 0x5621b071c930 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b0721340 0x5621b0723730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:5 5.e 5:76440cbf:::zone_info.293899f1-2738-4b64-a8e1-a18ff039d626:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0723d40 con 0x5621b0721340 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.053+0000 7f5a1152b640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b0721340 0x5621b0723730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a1152b640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b0721340 0x5621b0723730 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== osd_op_reply(5 zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 [read 0~993 out=993b] v0'0 uv5 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a13240800 20 rados_obj.operate() r=0 bl.length=993 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a13240800 20 searching for the correct realm 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a1152b640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5a0804e070 0x7f5a08028300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.054+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5a0804e070 0x7f5a08028300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.055+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5a0804e070 0x7f5a08028300 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.055+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.055+0000 7f5a10d2a640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5a00055920 0x7f5a0001d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.055+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a00055920 2024-09-19T11:26:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.055+0000 7f5a10529640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5a00055920 0x7f5a0001d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.056+0000 7f5a10529640 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5a00055920 0x7f5a0001d770 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.056+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.056+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.056+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 5 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.056+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.057+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 6 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.057+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a00055920 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.057+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.057+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a00055920 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.057+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 3 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.057+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.058+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.058+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.058+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 3 ==== osd_op_reply(14 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.058+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.058+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.058+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a00055920 2024-09-19T11:26:19.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.058+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 4 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.059+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.059+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.059+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.059+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=79b] v24'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.059+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.059+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 4 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.059+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 5 ==== osd_op_reply(20 [pgnls start_epoch 33 out=170b] v33'5 uv5 ondisk = 1) ==== 144+0+170 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 4 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 6 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 7 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.060+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.061+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 9 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.061+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.061+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 10 ==== osd_op_reply(25 [pgnls start_epoch 33 out=74b] v33'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.061+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a00055920 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.061+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 5 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.061+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.061+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 8 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.062+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.062+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 5 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.062+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.062+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 11 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.062+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a00055920 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.062+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=82b] v33'1 uv1 ondisk = 1) ==== 144+0+82 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.062+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.063+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 9 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.063+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a00055920 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.063+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 7 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.063+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.063+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 6 ==== osd_op_reply(33 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.063+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b0721340 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 10 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x5621b071c930 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 12 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v33'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725670 con 0x7f5a0804e070 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 8 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-09-19T11:26:19.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 RGWRados::pool_iterate: got zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 RGWRados::pool_iterate: got zone_info.293899f1-2738-4b64-a8e1-a18ff039d626 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 RGWRados::pool_iterate: got default.zone. 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 RGWRados::pool_iterate: got period_config.default 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 RGWRados::pool_iterate: got zone_names.default 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.064+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0724700 con 0x5621b071c930 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0724f50 con 0x5621b071c930 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 rados_obj.operate() r=0 bl.length=46 2024-09-19T11:26:19.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:40 5.e 5:75883de3:::zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725af0 con 0x5621b0721340 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 11 ==== osd_op_reply(40 zonegroup_info.d3056e04-d444-48b9-8e61-96c58b445cdd [read 0~436 out=436b] v0'0 uv2 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 rados_obj.operate() r=0 bl.length=436 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 zone default found 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 4 Realm: () 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 4 ZoneGroup: default (d3056e04-d444-48b9-8e61-96c58b445cdd) 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 4 Zone: default (293899f1-2738-4b64-a8e1-a18ff039d626) 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 zonegroup default 2024-09-19T11:26:19.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.065+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0725ed0 con 0x7f5a00055920 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 8 ==== osd_op_reply(41 period_config.default [read 0~187 out=187b] v0'0 uv1 ondisk = 0) ==== 165+0+187 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 20 rados_obj.operate() r=0 bl.length=187 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0727e00 con 0x5621b0721340 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 12 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 20 rados->read ofs=0 len=0 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0728230 con 0x5621b0721340 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 13 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 20 rados_obj.operate() r=-2 bl.length=0 2024-09-19T11:26:19.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 20 started sync module instance, tier type = 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 20 started zone id=293899f1-2738-4b64-a8e1-a18ff039d626 (name=default) with tier type = 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.066+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b07271a0 con 0x7f5a0804e070 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.068+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 9 ==== osd_op_reply(44 notify.0 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.068+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94702694198512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0728690 con 0x7f5a0804e070 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.068+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0728ee0 con 0x5621b071c930 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.069+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 10 ==== osd_op_reply(45 notify.0 [watch watch cookie 94702694198512] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.069+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 15 ==== osd_op_reply(46 notify.1 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.069+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94702694200768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0728f80 con 0x5621b071c930 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.069+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0729aa0 con 0x5621b0721340 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.071+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94702694200768] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.071+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 14 ==== osd_op_reply(48 notify.2 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.071+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94702694208880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0729a60 con 0x5621b0721340 2024-09-19T11:26:19.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.071+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072a500 con 0x7f5a0804e070 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.072+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 11 ==== osd_op_reply(50 notify.3 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.072+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 15 ==== osd_op_reply(49 notify.2 [watch watch cookie 94702694208880] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.072+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94702694206000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072a500 con 0x7f5a0804e070 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.072+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072afa0 con 0x5621b071c930 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.073+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 12 ==== osd_op_reply(51 notify.3 [watch watch cookie 94702694206000] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.073+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 17 ==== osd_op_reply(52 notify.4 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.073+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94702694214288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072af60 con 0x5621b071c930 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.073+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072ba10 con 0x7f5a00055920 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.075+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 9 ==== osd_op_reply(54 notify.5 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.075+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94702694216992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072bed0 con 0x7f5a00055920 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.075+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072c9f0 con 0x7f5a00055920 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.075+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94702694214288] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.076+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94702694216992] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.076+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 11 ==== osd_op_reply(56 notify.6 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.076+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94702694220992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072b940 con 0x7f5a00055920 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.076+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072b940 con 0x7f5a00055920 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94702694220992] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 13 ==== osd_op_reply(58 notify.7 [create] v33'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94702694211632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072cfd0 con 0x7f5a00055920 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 20 add_watcher() i=0 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 20 add_watcher() i=1 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 20 add_watcher() i=2 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 20 add_watcher() i=3 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 20 add_watcher() i=4 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 20 add_watcher() i=5 2024-09-19T11:26:19.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.077+0000 7f5a13240800 20 add_watcher() i=6 2024-09-19T11:26:19.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.078+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94702694211632] v33'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.078+0000 7f5a13240800 20 add_watcher() i=7 2024-09-19T11:26:19.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.078+0000 7f5a13240800 2 all 8 watchers are set, enabling cache 2024-09-19T11:26:19.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.078+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0730230 con 0x5621b071c930 2024-09-19T11:26:19.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.079+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.079+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94702694222992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072d560 con 0x5621b071c930 2024-09-19T11:26:19.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.080+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94702694222992] v33'274 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.080+0000 7f58c77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.080+0000 7f5a13240800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.080+0000 7f5a13240800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.080+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0738930 con 0x5621b071c930 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.081+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v33'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.081+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0738fe0 con 0x7f5a00055920 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.083+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v33'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.083+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0739420 con 0x5621b0721340 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.084+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 16 ==== osd_op_reply(64 gc.2 [create,call,call,call] v33'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.084+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b07393c0 con 0x5621b071c930 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.085+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v33'275 uv166 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.085+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b07399b0 con 0x7f5a0804e070 2024-09-19T11:26:19.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.086+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 13 ==== osd_op_reply(66 gc.4 [create,call,call,call] v33'83 uv55 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.086+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0739bf0 con 0x5621b0721340 2024-09-19T11:26:19.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.087+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 17 ==== osd_op_reply(67 gc.5 [create,call,call,call] v33'56 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.087+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0739bf0 con 0x5621b071c930 2024-09-19T11:26:19.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.088+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v33'34 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.088+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0739bf0 con 0x7f5a00055920 2024-09-19T11:26:19.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.090+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v33'56 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.090+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0739c80 con 0x7f5a0804e070 2024-09-19T11:26:19.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.091+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 14 ==== osd_op_reply(70 gc.8 [create,call,call,call] v33'32 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.091+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a130 con 0x5621b0721340 2024-09-19T11:26:19.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.092+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 18 ==== osd_op_reply(71 gc.9 [create,call,call,call] v33'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.092+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a130 con 0x7f5a0804e070 2024-09-19T11:26:19.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.093+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 15 ==== osd_op_reply(72 gc.10 [create,call,call,call] v33'84 uv51 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.093+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a130 con 0x7f5a00055920 2024-09-19T11:26:19.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.094+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v33'41 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.094+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a130 con 0x7f5a0804e070 2024-09-19T11:26:19.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.095+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 16 ==== osd_op_reply(74 gc.12 [create,call,call,call] v33'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.095+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a130 con 0x7f5a00055920 2024-09-19T11:26:19.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.097+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v33'63 uv41 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.097+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a130 con 0x5621b071c930 2024-09-19T11:26:19.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.098+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v33'276 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.098+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a130 con 0x5621b0721340 2024-09-19T11:26:19.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.099+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 19 ==== osd_op_reply(77 gc.15 [create,call,call,call] v33'77 uv45 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.099+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a1c0 con 0x7f5a0804e070 2024-09-19T11:26:19.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.100+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 17 ==== osd_op_reply(78 gc.16 [create,call,call,call] v33'85 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.100+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a1c0 con 0x5621b0721340 2024-09-19T11:26:19.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.101+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 20 ==== osd_op_reply(79 gc.17 [create,call,call,call] v33'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.101+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a1c0 con 0x5621b071c930 2024-09-19T11:26:19.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.102+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v33'39 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.102+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a1c0 con 0x5621b071c930 2024-09-19T11:26:19.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.103+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v33'277 uv164 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.103+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a1c0 con 0x5621b0721340 2024-09-19T11:26:19.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.104+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 21 ==== osd_op_reply(82 gc.20 [create,call,call,call] v33'78 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.104+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a1c0 con 0x7f5a0804e070 2024-09-19T11:26:19.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.105+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 18 ==== osd_op_reply(83 gc.21 [create,call,call,call] v33'41 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.106+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a1c0 con 0x5621b0721340 2024-09-19T11:26:19.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.106+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 22 ==== osd_op_reply(84 gc.22 [create,call,call,call] v33'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.106+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a230 con 0x5621b0721340 2024-09-19T11:26:19.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.107+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 23 ==== osd_op_reply(85 gc.23 [create,call,call,call] v33'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.107+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a230 con 0x5621b0721340 2024-09-19T11:26:19.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.108+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 24 ==== osd_op_reply(86 gc.24 [create,call,call,call] v33'27 uv17 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.108+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a270 con 0x5621b0721340 2024-09-19T11:26:19.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.109+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 25 ==== osd_op_reply(87 gc.25 [create,call,call,call] v33'57 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.109+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a270 con 0x7f5a00055920 2024-09-19T11:26:19.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.111+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v33'35 uv25 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.111+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a2b0 con 0x5621b0721340 2024-09-19T11:26:19.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.112+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 26 ==== osd_op_reply(89 gc.27 [create,call,call,call] v33'35 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.112+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a2b0 con 0x7f5a00055920 2024-09-19T11:26:19.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.113+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v33'57 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.113+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a2b0 con 0x5621b0721340 2024-09-19T11:26:19.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.114+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 27 ==== osd_op_reply(91 gc.29 [create,call,call,call] v33'79 uv49 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.114+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a2b0 con 0x7f5a00055920 2024-09-19T11:26:19.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.115+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v33'64 uv43 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.115+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b073a2f0 con 0x7f5a0804e070 2024-09-19T11:26:19.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.116+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 19 ==== osd_op_reply(93 gc.31 [create,call,call,call] v33'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.117+0000 7f58c67e4640 20 reqs_thread_entry: start 2024-09-19T11:26:19.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.117+0000 7f5a13240800 20 init_complete bucket index max shards: 11 2024-09-19T11:26:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.117+0000 7f5a13240800 20 Filter name: none 2024-09-19T11:26:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.117+0000 7f58bffff640 20 reqs_thread_entry: start 2024-09-19T11:26:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.117+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-09-19T11:26:19.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.117+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:94 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.118+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 28 ==== osd_op_reply(94 gc.24 [call] v33'28 uv28 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.118+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:95 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.118+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 29 ==== osd_op_reply(95 gc.24 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.118+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.118+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:96 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0765210 con 0x5621b0721340 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.118+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 30 ==== osd_op_reply(96 gc.24 [call out=24b] v0'0 uv28 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.118+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:97 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.119+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 31 ==== osd_op_reply(97 gc.24 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.119+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.119+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:98 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.119+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 32 ==== osd_op_reply(98 gc.24 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.119+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.119+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:99 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.120+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 33 ==== osd_op_reply(99 gc.24 [call] v33'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.120+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-09-19T11:26:19.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.120+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:100 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.121+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 34 ==== osd_op_reply(100 gc.25 [call] v33'58 uv58 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.121+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:101 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.121+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 35 ==== osd_op_reply(101 gc.25 [call out=15b] v0'0 uv58 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.121+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.121+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:102 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0765210 con 0x5621b0721340 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.121+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 36 ==== osd_op_reply(102 gc.25 [call out=24b] v0'0 uv58 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.122+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:103 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.122+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 37 ==== osd_op_reply(103 gc.25 [call out=15b] v0'0 uv58 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.122+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.122+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:104 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.122+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 38 ==== osd_op_reply(104 gc.25 [call out=15b] v0'0 uv58 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.122+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.122+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:105 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.123+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 39 ==== osd_op_reply(105 gc.25 [call] v33'59 uv59 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.123+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-09-19T11:26:19.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.123+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:106 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.124+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 22 ==== osd_op_reply(106 gc.26 [call] v33'36 uv36 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.124+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:107 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 23 ==== osd_op_reply(107 gc.26 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:108 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0765210 con 0x7f5a00055920 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 24 ==== osd_op_reply(108 gc.26 [call out=24b] v0'0 uv36 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:109 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 25 ==== osd_op_reply(109 gc.26 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.125+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:110 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.126+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 26 ==== osd_op_reply(110 gc.26 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.126+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.126+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.127+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 27 ==== osd_op_reply(111 gc.26 [call] v33'37 uv37 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.127+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-09-19T11:26:19.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.127+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:112 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 40 ==== osd_op_reply(112 gc.27 [call] v33'36 uv36 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:113 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 41 ==== osd_op_reply(113 gc.27 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0765210 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 42 ==== osd_op_reply(114 gc.27 [call out=24b] v0'0 uv36 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 43 ==== osd_op_reply(115 gc.27 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 44 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.128+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:117 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x5621b0721340 2024-09-19T11:26:19.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.129+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 45 ==== osd_op_reply(117 gc.27 [call] v33'37 uv37 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.129+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-09-19T11:26:19.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.129+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.131+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 28 ==== osd_op_reply(118 gc.28 [call] v33'58 uv58 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.131+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.131+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 29 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv58 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.131+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.131+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:120 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0765210 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.131+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 30 ==== osd_op_reply(120 gc.28 [call out=24b] v0'0 uv58 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.131+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:121 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 31 ==== osd_op_reply(121 gc.28 [call out=15b] v0'0 uv58 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:122 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0766220 con 0x7f5a00055920 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 32 ==== osd_op_reply(122 gc.28 [call out=1716b] v0'0 uv58 ondisk = 0) ==== 149+0+1716 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 20 garbage collection: RGWGC::process iterating over entry tag='293899f1-2738-4b64-a8e1-a18ff039d626.4229.13282591347822081814', time=2024-09-19T11:26:26.973865+0000, chain.objs.size()=7 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_1 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:123 3.3bs0 3:de46f095:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_1:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0764550 con 0x5621b0721340 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_2 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:124 3.18s0 3:1be90c1c:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_2:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b07693f0 con 0x5621b0721340 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_3 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:125 3.1s0 3:81b4f50d:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_3:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076bf80 con 0x7f5a00055920 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_4 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:126 3.13s0 3:c8839d6f:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_4:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076e980 con 0x5621b0721340 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_5 2024-09-19T11:26:19.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:127 3.30s0 3:0debc147:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_5:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b07693f0 con 0x5621b0721340 2024-09-19T11:26:19.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_6 2024-09-19T11:26:19.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:128 3.4s0 3:23542205:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_6:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076e980 con 0x7f5a00055920 2024-09-19T11:26:19.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_7 2024-09-19T11:26:19.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.132+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:129 3.ds0 3:b1c2b2c1:::293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_7:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0775c30 con 0x5621b0721340 2024-09-19T11:26:19.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.134+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 33 ==== osd_op_reply(125 293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_3 [call] v33'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.134+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 34 ==== osd_op_reply(128 293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_6 [call] v33'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.134+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 46 ==== osd_op_reply(123 293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_1 [call] v33'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.134+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 47 ==== osd_op_reply(124 293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_2 [call] v33'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.135+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 48 ==== osd_op_reply(129 293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_7 [call] v33'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.136+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 49 ==== osd_op_reply(127 293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_5 [call] v33'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.136+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 50 ==== osd_op_reply(126 293899f1-2738-4b64-a8e1-a18ff039d626.4229.11__shadow_.90zfI6pivAB9iFuIBSSOPkrtwnBGnMO_4 [call] v33'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.136+0000 7f5a13240800 5 garbage collection: RGWGC::process removing entries, marker: 2024-09-19T11:26:19.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.136+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:130 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763f40 con 0x7f5a00055920 2024-09-19T11:26:19.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.137+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 35 ==== osd_op_reply(130 gc.28 [call] v33'59 uv59 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.137+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:131 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x7f5a00055920 2024-09-19T11:26:19.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.138+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 36 ==== osd_op_reply(131 gc.28 [call] v33'60 uv60 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.138+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-09-19T11:26:19.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.138+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:132 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x5621b0721340 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.139+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 51 ==== osd_op_reply(132 gc.29 [call] v33'80 uv80 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.139+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:133 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x5621b0721340 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 52 ==== osd_op_reply(133 gc.29 [call out=15b] v0'0 uv80 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:134 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x5621b0721340 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 53 ==== osd_op_reply(134 gc.29 [call out=24b] v0'0 uv80 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:135 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x5621b0721340 2024-09-19T11:26:19.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 54 ==== osd_op_reply(135 gc.29 [call out=15b] v0'0 uv80 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.140+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:136 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x5621b0721340 2024-09-19T11:26:19.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.141+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 55 ==== osd_op_reply(136 gc.29 [call out=15b] v0'0 uv80 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.141+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.141+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:137 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x5621b0721340 2024-09-19T11:26:19.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.142+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 56 ==== osd_op_reply(137 gc.29 [call] v33'81 uv81 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.142+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-09-19T11:26:19.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.142+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:138 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a00055920 2024-09-19T11:26:19.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.143+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 37 ==== osd_op_reply(138 gc.30 [call] v33'65 uv65 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.143+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:139 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a00055920 2024-09-19T11:26:19.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.143+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 38 ==== osd_op_reply(139 gc.30 [call out=15b] v0'0 uv65 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.143+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.143+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:140 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a830 con 0x7f5a00055920 2024-09-19T11:26:19.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 39 ==== osd_op_reply(140 gc.30 [call out=24b] v0'0 uv65 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:141 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a00055920 2024-09-19T11:26:19.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 40 ==== osd_op_reply(141 gc.30 [call out=15b] v0'0 uv65 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:142 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a00055920 2024-09-19T11:26:19.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 41 ==== osd_op_reply(142 gc.30 [call out=15b] v0'0 uv65 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.144+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:143 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a00055920 2024-09-19T11:26:19.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.145+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 42 ==== osd_op_reply(143 gc.30 [call] v33'66 uv66 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.145+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-09-19T11:26:19.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.145+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:144 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a0804e070 2024-09-19T11:26:19.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.146+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 20 ==== osd_op_reply(144 gc.31 [call] v33'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.146+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:145 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a0804e070 2024-09-19T11:26:19.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.146+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 21 ==== osd_op_reply(145 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:146 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a830 con 0x7f5a0804e070 2024-09-19T11:26:19.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 22 ==== osd_op_reply(146 gc.31 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:147 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a0804e070 2024-09-19T11:26:19.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 23 ==== osd_op_reply(147 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:148 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a0804e070 2024-09-19T11:26:19.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 24 ==== osd_op_reply(148 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.147+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:149 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x7f5a0804e070 2024-09-19T11:26:19.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.148+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 25 ==== osd_op_reply(149 gc.31 [call] v33'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.148+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-09-19T11:26:19.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.148+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:150 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x5621b071c930 2024-09-19T11:26:19.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.149+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 27 ==== osd_op_reply(150 gc.0 [call] v33'38 uv38 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:151 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b0763ef0 con 0x5621b071c930 2024-09-19T11:26:19.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 28 ==== osd_op_reply(151 gc.0 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:152 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a830 con 0x5621b071c930 2024-09-19T11:26:19.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 29 ==== osd_op_reply(152 gc.0 [call out=24b] v0'0 uv38 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:153 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x5621b071c930 2024-09-19T11:26:19.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 30 ==== osd_op_reply(153 gc.0 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.150+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:154 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x5621b071c930 2024-09-19T11:26:19.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.151+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 31 ==== osd_op_reply(154 gc.0 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.151+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.151+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:155 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x5621b071c930 2024-09-19T11:26:19.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.152+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 32 ==== osd_op_reply(155 gc.0 [call] v33'39 uv39 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.152+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-09-19T11:26:19.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.152+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:156 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x7f5a00055920 2024-09-19T11:26:19.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.153+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 43 ==== osd_op_reply(156 gc.1 [call] v33'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.153+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:157 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 44 ==== osd_op_reply(157 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:158 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a830 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 45 ==== osd_op_reply(158 gc.1 [call out=24b] v0'0 uv33 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:159 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 46 ==== osd_op_reply(159 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:160 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 47 ==== osd_op_reply(160 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.154+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:161 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x7f5a00055920 2024-09-19T11:26:19.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.155+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 48 ==== osd_op_reply(161 gc.1 [call] v33'34 uv34 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.156+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-09-19T11:26:19.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.156+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:162 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b840 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.156+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 57 ==== osd_op_reply(162 gc.2 [call] v33'32 uv32 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.156+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:163 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 58 ==== osd_op_reply(163 gc.2 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:164 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 59 ==== osd_op_reply(164 gc.2 [call out=24b] v0'0 uv32 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:165 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 60 ==== osd_op_reply(165 gc.2 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:166 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 61 ==== osd_op_reply(166 gc.2 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.157+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:167 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.158+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 62 ==== osd_op_reply(167 gc.2 [call] v33'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.158+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-09-19T11:26:19.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.158+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:168 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.159+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 33 ==== osd_op_reply(168 gc.3 [call] v33'278 uv278 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.159+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:169 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 34 ==== osd_op_reply(169 gc.3 [call out=15b] v0'0 uv278 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:170 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 35 ==== osd_op_reply(170 gc.3 [call out=24b] v0'0 uv278 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:171 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 36 ==== osd_op_reply(171 gc.3 [call out=15b] v0'0 uv278 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:172 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 37 ==== osd_op_reply(172 gc.3 [call out=15b] v0'0 uv278 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.160+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:173 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.161+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 38 ==== osd_op_reply(173 gc.3 [call] v33'279 uv279 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.161+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-09-19T11:26:19.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.161+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:174 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.162+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 26 ==== osd_op_reply(174 gc.4 [call] v33'86 uv86 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.162+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:175 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 27 ==== osd_op_reply(175 gc.4 [call out=15b] v0'0 uv86 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:176 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a0804e070 2024-09-19T11:26:19.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 28 ==== osd_op_reply(176 gc.4 [call out=24b] v0'0 uv86 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:177 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 29 ==== osd_op_reply(177 gc.4 [call out=15b] v0'0 uv86 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:178 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 30 ==== osd_op_reply(178 gc.4 [call out=15b] v0'0 uv86 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.163+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:179 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.164+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 31 ==== osd_op_reply(179 gc.4 [call] v33'87 uv87 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.164+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.164+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:180 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.165+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 63 ==== osd_op_reply(180 gc.5 [call] v33'60 uv60 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.165+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:181 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.165+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 64 ==== osd_op_reply(181 gc.5 [call out=15b] v0'0 uv60 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.165+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.165+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:182 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 65 ==== osd_op_reply(182 gc.5 [call out=24b] v0'0 uv60 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:183 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 66 ==== osd_op_reply(183 gc.5 [call out=15b] v0'0 uv60 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:184 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 67 ==== osd_op_reply(184 gc.5 [call out=15b] v0'0 uv60 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.166+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:185 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.167+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 68 ==== osd_op_reply(185 gc.5 [call] v33'61 uv61 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.167+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-09-19T11:26:19.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.167+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:186 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 39 ==== osd_op_reply(186 gc.6 [call] v33'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:187 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 40 ==== osd_op_reply(187 gc.6 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:188 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b071c930 2024-09-19T11:26:19.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 41 ==== osd_op_reply(188 gc.6 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:189 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 42 ==== osd_op_reply(189 gc.6 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.169+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:190 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.170+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 43 ==== osd_op_reply(190 gc.6 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.170+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.170+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:191 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.171+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 44 ==== osd_op_reply(191 gc.6 [call] v33'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.171+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-09-19T11:26:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.171+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:192 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.172+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 49 ==== osd_op_reply(192 gc.7 [call] v33'61 uv61 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.172+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:193 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.172+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 50 ==== osd_op_reply(193 gc.7 [call out=15b] v0'0 uv61 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.172+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.172+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:194 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a00055920 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.172+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 51 ==== osd_op_reply(194 gc.7 [call out=24b] v0'0 uv61 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.172+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:195 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.173+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 52 ==== osd_op_reply(195 gc.7 [call out=15b] v0'0 uv61 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.173+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.173+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:196 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.173+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 53 ==== osd_op_reply(196 gc.7 [call out=15b] v0'0 uv61 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.173+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.173+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:197 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.174+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 54 ==== osd_op_reply(197 gc.7 [call] v33'62 uv62 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.174+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-09-19T11:26:19.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.174+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:198 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.175+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 32 ==== osd_op_reply(198 gc.8 [call] v33'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.175+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:199 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.175+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 33 ==== osd_op_reply(199 gc.8 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.175+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.175+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:200 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a0804e070 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.175+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 34 ==== osd_op_reply(200 gc.8 [call out=24b] v0'0 uv33 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.175+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:201 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.176+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 35 ==== osd_op_reply(201 gc.8 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.176+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.176+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:202 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.176+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 36 ==== osd_op_reply(202 gc.8 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.176+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.176+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:203 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.177+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 37 ==== osd_op_reply(203 gc.8 [call] v33'34 uv34 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.177+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-09-19T11:26:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.177+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:204 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 69 ==== osd_op_reply(204 gc.9 [call] v33'38 uv38 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:205 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 70 ==== osd_op_reply(205 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:206 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 71 ==== osd_op_reply(206 gc.9 [call out=24b] v0'0 uv38 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:207 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 72 ==== osd_op_reply(207 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.178+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:208 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.179+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 73 ==== osd_op_reply(208 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.179+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.179+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:209 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.180+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 74 ==== osd_op_reply(209 gc.9 [call] v33'39 uv39 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.180+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-09-19T11:26:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.180+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:210 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 38 ==== osd_op_reply(210 gc.10 [call] v33'88 uv88 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:211 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 39 ==== osd_op_reply(211 gc.10 [call out=15b] v0'0 uv88 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:212 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a0804e070 2024-09-19T11:26:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 40 ==== osd_op_reply(212 gc.10 [call out=24b] v0'0 uv88 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:213 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 41 ==== osd_op_reply(213 gc.10 [call out=15b] v0'0 uv88 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.181+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:214 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.182+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 42 ==== osd_op_reply(214 gc.10 [call out=15b] v0'0 uv88 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.182+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.182+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:215 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.183+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 43 ==== osd_op_reply(215 gc.10 [call] v33'89 uv89 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.183+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-09-19T11:26:19.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.183+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:216 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.184+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 55 ==== osd_op_reply(216 gc.11 [call] v33'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.184+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:217 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.184+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 56 ==== osd_op_reply(217 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.184+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.184+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:218 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a00055920 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.184+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 57 ==== osd_op_reply(218 gc.11 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.184+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:219 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.185+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 58 ==== osd_op_reply(219 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.185+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.185+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:220 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.185+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 59 ==== osd_op_reply(220 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.185+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.185+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:221 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.186+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 60 ==== osd_op_reply(221 gc.11 [call] v33'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.186+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-09-19T11:26:19.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.186+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:222 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.187+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 44 ==== osd_op_reply(222 gc.12 [call] v33'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.187+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:223 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.187+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 45 ==== osd_op_reply(223 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.187+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.187+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:224 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.187+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 46 ==== osd_op_reply(224 gc.12 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.187+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:225 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.188+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 47 ==== osd_op_reply(225 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.188+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.188+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:226 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.188+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 48 ==== osd_op_reply(226 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.188+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.188+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:227 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.189+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 49 ==== osd_op_reply(227 gc.12 [call] v33'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.189+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-09-19T11:26:19.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.189+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:228 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.190+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 61 ==== osd_op_reply(228 gc.13 [call] v33'67 uv67 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.190+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:229 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.190+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 62 ==== osd_op_reply(229 gc.13 [call out=15b] v0'0 uv67 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.190+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.190+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:230 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a00055920 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 63 ==== osd_op_reply(230 gc.13 [call out=24b] v0'0 uv67 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:231 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 64 ==== osd_op_reply(231 gc.13 [call out=15b] v0'0 uv67 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:232 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 65 ==== osd_op_reply(232 gc.13 [call out=15b] v0'0 uv67 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.191+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:233 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a00055920 2024-09-19T11:26:19.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.192+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 66 ==== osd_op_reply(233 gc.13 [call] v33'68 uv68 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.192+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-09-19T11:26:19.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.192+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:234 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.193+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 45 ==== osd_op_reply(234 gc.14 [call] v33'280 uv280 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.193+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:235 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 46 ==== osd_op_reply(235 gc.14 [call out=15b] v0'0 uv280 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:236 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 47 ==== osd_op_reply(236 gc.14 [call out=24b] v0'0 uv280 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:237 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 48 ==== osd_op_reply(237 gc.14 [call out=15b] v0'0 uv280 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:238 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 49 ==== osd_op_reply(238 gc.14 [call out=15b] v0'0 uv280 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.194+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:239 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.195+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 50 ==== osd_op_reply(239 gc.14 [call] v33'281 uv281 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.195+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-09-19T11:26:19.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.195+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:240 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.196+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 75 ==== osd_op_reply(240 gc.15 [call] v33'82 uv82 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.196+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:241 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 76 ==== osd_op_reply(241 gc.15 [call out=15b] v0'0 uv82 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:242 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 77 ==== osd_op_reply(242 gc.15 [call out=24b] v0'0 uv82 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:243 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 78 ==== osd_op_reply(243 gc.15 [call out=15b] v0'0 uv82 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:244 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 79 ==== osd_op_reply(244 gc.15 [call out=15b] v0'0 uv82 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.197+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:245 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.199+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 80 ==== osd_op_reply(245 gc.15 [call] v33'83 uv83 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.199+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-09-19T11:26:19.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.199+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:246 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 50 ==== osd_op_reply(246 gc.16 [call] v33'90 uv90 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:247 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 51 ==== osd_op_reply(247 gc.16 [call out=15b] v0'0 uv90 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:248 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 52 ==== osd_op_reply(248 gc.16 [call out=24b] v0'0 uv90 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:249 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 53 ==== osd_op_reply(249 gc.16 [call out=15b] v0'0 uv90 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.200+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:250 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.201+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 54 ==== osd_op_reply(250 gc.16 [call out=15b] v0'0 uv90 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.201+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.201+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:251 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.202+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 55 ==== osd_op_reply(251 gc.16 [call] v33'91 uv91 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.202+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-09-19T11:26:19.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.202+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:252 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 81 ==== osd_op_reply(252 gc.17 [call] v33'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:253 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 82 ==== osd_op_reply(253 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:254 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 83 ==== osd_op_reply(254 gc.17 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:255 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 84 ==== osd_op_reply(255 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.203+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:256 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.204+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 85 ==== osd_op_reply(256 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.204+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.204+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:257 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.204+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 86 ==== osd_op_reply(257 gc.17 [call] v33'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.205+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-09-19T11:26:19.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.205+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:258 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.206+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 51 ==== osd_op_reply(258 gc.18 [call] v33'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.206+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:259 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.206+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 52 ==== osd_op_reply(259 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.206+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.206+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:260 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b071c930 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.206+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 53 ==== osd_op_reply(260 gc.18 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.206+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:261 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.207+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 54 ==== osd_op_reply(261 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.207+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.207+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:262 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.207+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 55 ==== osd_op_reply(262 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.207+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.207+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:263 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.208+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 56 ==== osd_op_reply(263 gc.18 [call] v33'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.208+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-09-19T11:26:19.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.208+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:264 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.209+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 57 ==== osd_op_reply(264 gc.19 [call] v33'282 uv282 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.209+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:265 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.209+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 58 ==== osd_op_reply(265 gc.19 [call out=15b] v0'0 uv282 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.209+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.209+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:266 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b071c930 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.209+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 59 ==== osd_op_reply(266 gc.19 [call out=24b] v0'0 uv282 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.209+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:267 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.210+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 60 ==== osd_op_reply(267 gc.19 [call out=15b] v0'0 uv282 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.210+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.210+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:268 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.210+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 61 ==== osd_op_reply(268 gc.19 [call out=15b] v0'0 uv282 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.210+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.210+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:269 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b071c930 2024-09-19T11:26:19.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.211+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 62 ==== osd_op_reply(269 gc.19 [call] v33'283 uv283 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.211+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-09-19T11:26:19.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.211+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:270 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.212+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 87 ==== osd_op_reply(270 gc.20 [call] v33'84 uv84 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.212+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:271 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.212+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 88 ==== osd_op_reply(271 gc.20 [call out=15b] v0'0 uv84 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.212+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.212+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:272 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.212+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 89 ==== osd_op_reply(272 gc.20 [call out=24b] v0'0 uv84 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.212+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:273 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.213+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 90 ==== osd_op_reply(273 gc.20 [call out=15b] v0'0 uv84 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.213+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.213+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:274 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.213+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 91 ==== osd_op_reply(274 gc.20 [call out=15b] v0'0 uv84 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.213+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.213+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:275 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.214+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 92 ==== osd_op_reply(275 gc.20 [call] v33'85 uv85 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.214+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.214+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:276 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.215+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 56 ==== osd_op_reply(276 gc.21 [call] v33'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.215+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:277 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.215+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 57 ==== osd_op_reply(277 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.215+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.215+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:278 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 58 ==== osd_op_reply(278 gc.21 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:279 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 59 ==== osd_op_reply(279 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:280 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 60 ==== osd_op_reply(280 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.216+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:281 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x7f5a0804e070 2024-09-19T11:26:19.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.217+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 61 ==== osd_op_reply(281 gc.21 [call] v33'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.217+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.217+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:282 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.218+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 93 ==== osd_op_reply(282 gc.22 [call] v33'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.218+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:283 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.218+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 94 ==== osd_op_reply(283 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.218+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.218+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:284 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.218+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 95 ==== osd_op_reply(284 gc.22 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.218+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:285 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.219+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 96 ==== osd_op_reply(285 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.219+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.219+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:286 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.219+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 97 ==== osd_op_reply(286 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.219+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.219+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:287 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.220+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 98 ==== osd_op_reply(287 gc.22 [call] v33'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.220+0000 7f5a13240800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.220+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:288 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 99 ==== osd_op_reply(288 gc.23 [call] v33'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:289 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 100 ==== osd_op_reply(289 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:290 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 101 ==== osd_op_reply(290 gc.23 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:291 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.221+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 102 ==== osd_op_reply(291 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.222+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.222+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:292 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5621b076a740 con 0x5621b0721340 2024-09-19T11:26:19.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.222+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 103 ==== osd_op_reply(292 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.222+0000 7f5a13240800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-19T11:26:19.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.222+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:293 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b076b750 con 0x5621b0721340 2024-09-19T11:26:19.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.223+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 104 ==== osd_op_reply(293 gc.23 [call] v33'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.224+0000 7f58c77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-19T11:26:19.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.224+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94702694222992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0737b40 con 0x5621b071c930 2024-09-19T11:26:19.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.225+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 63 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 94702694222992] v33'284 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.225+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94702694198512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072cc90 con 0x7f5a0804e070 2024-09-19T11:26:19.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.225+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 62 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 94702694198512] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.225+0000 7f5a13240800 20 remove_watcher() i=0 2024-09-19T11:26:19.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.225+0000 7f5a13240800 2 removed watcher, disabling cache 2024-09-19T11:26:19.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.226+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94702694200768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072cc90 con 0x5621b071c930 2024-09-19T11:26:19.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.227+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 64 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 94702694200768] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.227+0000 7f5a13240800 20 remove_watcher() i=1 2024-09-19T11:26:19.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.227+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94702694208880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b07275c0 con 0x5621b0721340 2024-09-19T11:26:19.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.228+0000 7f5a1152b640 1 -- 172.21.15.28:0/2702657944 <== osd.3 v2:172.21.15.28:6808/3027134945 105 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 94702694208880] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0803a070 con 0x5621b0721340 2024-09-19T11:26:19.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.228+0000 7f5a13240800 20 remove_watcher() i=2 2024-09-19T11:26:19.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.228+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94702694206000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072cc90 con 0x7f5a0804e070 2024-09-19T11:26:19.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.229+0000 7f5a10d2a640 1 -- 172.21.15.28:0/2702657944 <== osd.2 v2:172.21.15.28:6800/2071226075 63 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 94702694206000] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a0001b070 con 0x7f5a0804e070 2024-09-19T11:26:19.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.229+0000 7f5a13240800 20 remove_watcher() i=3 2024-09-19T11:26:19.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.229+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94702694214288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0729570 con 0x5621b071c930 2024-09-19T11:26:19.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.230+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.1 v2:172.21.15.28:6816/656119574 65 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 94702694214288] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x5621b071c930 2024-09-19T11:26:19.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.230+0000 7f5a13240800 20 remove_watcher() i=4 2024-09-19T11:26:19.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.230+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94702694216992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072aa90 con 0x7f5a00055920 2024-09-19T11:26:19.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.231+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 67 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 94702694216992] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.231+0000 7f5a13240800 20 remove_watcher() i=5 2024-09-19T11:26:19.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.231+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94702694220992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b0729570 con 0x7f5a00055920 2024-09-19T11:26:19.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.232+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 68 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 94702694220992] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.232+0000 7f5a13240800 20 remove_watcher() i=6 2024-09-19T11:26:19.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.232+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94702694211632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) -- 0x5621b072aa90 con 0x7f5a00055920 2024-09-19T11:26:19.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.233+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 <== osd.0 v2:172.21.15.22:6800/3144434255 69 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 94702694211632] v33'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5a04051070 con 0x7f5a00055920 2024-09-19T11:26:19.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.233+0000 7f5a13240800 20 remove_watcher() i=7 2024-09-19T11:26:19.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5a00055920 msgr2=0x7f5a0001d770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f5a00055920 0x7f5a0001d770 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b071c930 msgr2=0x5621b071edf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b071c930 0x5621b071edf0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5a0804e070 msgr2=0x7f5a08028300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7f5a0804e070 0x7f5a08028300 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b0721340 msgr2=0x5621b0723730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b0721340 0x5621b0723730 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58dc03cfb0 msgr2=0x7f58dc03f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58dc03cfb0 0x7f58dc03f470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a00002020 tx=0x7f5a00035000 comp rx=0 tx=0).stop 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 msgr2=0x5621b0710f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a10529640 1 -- 172.21.15.28:0/2702657944 reap_dead start 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0710f50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5a08017f70 tx=0x7f5a08053d10 comp rx=0 tx=0).stop 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 shutdown_connections 2024-09-19T11:26:19.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/2702657944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b0709a00 0x5621b0710f50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 >> 172.21.15.28:0/2702657944 conn(0x5621b0705e00 msgr2=0x5621b070b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 shutdown_connections 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/2702657944 wait complete. 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58e803d000 msgr2=0x7f58e803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58e803d000 0x7f58e803f4c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x5621b06ee9f0 tx=0x7f5a00014fb0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b06fcee0 msgr2=0x5621b06fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b06fcee0 0x5621b06fd300 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x5621b06d77f0 tx=0x7f5a0800bf30 comp rx=0 tx=0).stop 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 shutdown_connections 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58e803d000 0x7f58e803f4c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 --2- 172.21.15.28:0/3504267817 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b06fcee0 0x5621b06fd300 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 >> 172.21.15.28:0/3504267817 conn(0x5621b06f1290 msgr2=0x5621b06f1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 shutdown_connections 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.234+0000 7f5a13240800 1 -- 172.21.15.28:0/3504267817 wait complete. 2024-09-19T11:26:19.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b05e87a0 msgr2=0x5621b06e90f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b05e87a0 0x5621b06e90f0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b06eba70 msgr2=0x5621b06edec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b06eba70 0x5621b06edec0 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58f803d000 msgr2=0x7f58f803f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58f803d000 0x7f58f803f4c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5a04057000 tx=0x7f5a04052000 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 msgr2=0x5621b06d84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b06d84d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5a00001330 tx=0x7f5a00001250 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 shutdown_connections 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x5621b06eba70 0x5621b06edec0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f58f803d000 0x7f58f803f4c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x5621b05e87a0 0x5621b06e90f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 --2- 172.21.15.28:0/1760016351 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621b05e7ba0 0x5621b06d84d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 >> 172.21.15.28:0/1760016351 conn(0x5621b05e21c0 msgr2=0x5621b06d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 shutdown_connections 2024-09-19T11:26:19.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-19T11:26:19.235+0000 7f5a13240800 1 -- 172.21.15.28:0/1760016351 wait complete. 2024-09-19T11:26:19.246 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-19T11:26:19.256 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T11:26:19.256 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:26:19.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.359+0000 7f2064b94640 1 Processor -- start 2024-09-19T11:26:19.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.359+0000 7f2064b94640 1 -- start start 2024-09-19T11:26:19.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f2064b94640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f206006dfb0 0x7f206006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f2064b94640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2060071660 con 0x7f206006e8d0 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f2064b94640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2060072550 con 0x7f206006dfb0 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f205e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f206006dfb0 0x7f206006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f205e59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f206006dfb0 0x7f206006e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35068/0 (socket says 172.21.15.22:35068) 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f205dd9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f206006e8d0 0x7f2060097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41930/0 (socket says 172.21.15.22:41930) 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f205e59c640 1 -- 172.21.15.22:0/2827948056 learned_addr learned my addr 172.21.15.22:0/2827948056 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f205e59c640 1 -- 172.21.15.22:0/2827948056 >> v1:172.21.15.22:6789/0 conn(0x7f206006e8d0 legacy=0x7f2060097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.360+0000 7f205e59c640 1 -- 172.21.15.22:0/2827948056 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2060099e20 con 0x7f206006dfb0 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.361+0000 7f205e59c640 1 --2- 172.21.15.22:0/2827948056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f206006e390 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f20500097f0 tx=0x7f205002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d017180577fb2807 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.361+0000 7f205d59a640 1 -- 172.21.15.22:0/2827948056 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2050037070 con 0x7f206006dfb0 2024-09-19T11:26:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.361+0000 7f205d59a640 1 -- 172.21.15.22:0/2827948056 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2050032440 con 0x7f206006dfb0 2024-09-19T11:26:19.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.361+0000 7f205d59a640 1 -- 172.21.15.22:0/2827948056 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2050032740 con 0x7f206006dfb0 2024-09-19T11:26:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.362+0000 7f2064b94640 1 -- 172.21.15.22:0/2827948056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 msgr2=0x7f206006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.362+0000 7f2064b94640 1 --2- 172.21.15.22:0/2827948056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f206006e390 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f20500097f0 tx=0x7f205002a270 comp rx=0 tx=0).stop 2024-09-19T11:26:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.362+0000 7f2064b94640 1 -- 172.21.15.22:0/2827948056 shutdown_connections 2024-09-19T11:26:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.362+0000 7f2064b94640 1 --2- 172.21.15.22:0/2827948056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f206006e390 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.362+0000 7f2064b94640 1 -- 172.21.15.22:0/2827948056 >> 172.21.15.22:0/2827948056 conn(0x7f206009abc0 msgr2=0x7f2060098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.362+0000 7f2064b94640 1 -- 172.21.15.22:0/2827948056 shutdown_connections 2024-09-19T11:26:19.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.362+0000 7f2064b94640 1 -- 172.21.15.22:0/2827948056 wait complete. 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.363+0000 7f2064b94640 1 Processor -- start 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.363+0000 7f2064b94640 1 -- start start 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.363+0000 7f2064b94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f2060129290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.363+0000 7f2064b94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f206006ece0 con 0x7f206006dfb0 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.363+0000 7f205e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f2060129290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.363+0000 7f205e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f2060129290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35082/0 (socket says 172.21.15.22:35082) 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.363+0000 7f205e59c640 1 -- 172.21.15.22:0/3062791347 learned_addr learned my addr 172.21.15.22:0/3062791347 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.364+0000 7f205e59c640 1 -- 172.21.15.22:0/3062791347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2060096e80 con 0x7f206006dfb0 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.364+0000 7f205e59c640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f2060129290 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2050035040 tx=0x7f20500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.364+0000 7f2056ffd640 1 -- 172.21.15.22:0/3062791347 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2050041070 con 0x7f206006dfb0 2024-09-19T11:26:19.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.364+0000 7f2056ffd640 1 -- 172.21.15.22:0/3062791347 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205003c7a0 con 0x7f206006dfb0 2024-09-19T11:26:19.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.364+0000 7f2056ffd640 1 -- 172.21.15.22:0/3062791347 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2050037040 con 0x7f206006dfb0 2024-09-19T11:26:19.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.364+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206008bfe0 con 0x7f206006dfb0 2024-09-19T11:26:19.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.364+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206008d880 con 0x7f206006dfb0 2024-09-19T11:26:19.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.365+0000 7f2056ffd640 1 -- 172.21.15.22:0/3062791347 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f2050044070 con 0x7f206006dfb0 2024-09-19T11:26:19.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.365+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206006e390 con 0x7f206006dfb0 2024-09-19T11:26:19.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.365+0000 7f2056ffd640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f204003e710 0x7f2040040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.366+0000 7f2056ffd640 1 -- 172.21.15.22:0/3062791347 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f2050074210 con 0x7f206006dfb0 2024-09-19T11:26:19.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.366+0000 7f205dd9b640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f204003e710 0x7f2040040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.366+0000 7f205dd9b640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f204003e710 0x7f2040040bd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2048001630 tx=0x7f2048007a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.371+0000 7f2056ffd640 1 -- 172.21.15.22:0/3062791347 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f2050045dc0 con 0x7f206006dfb0 2024-09-19T11:26:19.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.557+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2060074110 con 0x7f204003e710 2024-09-19T11:26:19.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.568+0000 7f2056ffd640 1 -- 172.21.15.22:0/3062791347 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665813 (secure 0 0 0) 0x7f20480070d0 con 0x7f204003e710 2024-09-19T11:26:19.578 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:26:19.584 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f204003e710 msgr2=0x7f2040040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f204003e710 0x7f2040040bd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2048001630 tx=0x7f2048007a20 comp rx=0 tx=0).stop 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 msgr2=0x7f2060129290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f2060129290 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2050035040 tx=0x7f20500046c0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 shutdown_connections 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f204003e710 0x7f2040040bd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 --2- 172.21.15.22:0/3062791347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f206006dfb0 0x7f2060129290 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 >> 172.21.15.22:0/3062791347 conn(0x7f206009abc0 msgr2=0x7f206009c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 shutdown_connections 2024-09-19T11:26:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.573+0000 7f2064b94640 1 -- 172.21.15.22:0/3062791347 wait complete. 2024-09-19T11:26:19.588 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-19T11:26:18.841537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412817,"num_objects":229,"num_object_clones":0,"num_object_copies":465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":229,"num_whiteouts":0,"num_read":4973,"num_read_kb":4511,"num_write":2048,"num_write_kb":60586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4434,"ondisk_log_size":4434,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":152956,"kb_used_data":40700,"kb_used_omap":6,"kb_used_meta":112185,"kb_avail":377334404,"statfs":{"total":386547056640,"available":386390429696,"internally_reserved":0,"allocated":41676800,"data_stored":40498678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":114878230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811688,"num_objects":9,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":477,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940271+0000","last_change":"2024-09-19T11:23:03.259044+0000","last_active":"2024-09-19T11:23:43.940271+0000","last_peered":"2024-09-19T11:23:43.940271+0000","last_clean":"2024-09-19T11:23:43.940271+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:23:43.940271+0000","last_undegraded":"2024-09-19T11:23:43.940271+0000","last_fullsized":"2024-09-19T11:23:43.940271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:41.435159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333097+0000","last_change":"2024-09-19T11:23:05.248495+0000","last_active":"2024-09-19T11:26:12.333097+0000","last_peered":"2024-09-19T11:26:12.333097+0000","last_clean":"2024-09-19T11:26:12.333097+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:12.333097+0000","last_undegraded":"2024-09-19T11:26:12.333097+0000","last_fullsized":"2024-09-19T11:26:12.333097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:27.929797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.237576+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:20:44.729040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333061+0000","last_change":"2024-09-19T11:23:05.242656+0000","last_active":"2024-09-19T11:26:12.333061+0000","last_peered":"2024-09-19T11:26:12.333061+0000","last_clean":"2024-09-19T11:26:12.333061+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:26:12.333061+0000","last_undegraded":"2024-09-19T11:26:12.333061+0000","last_fullsized":"2024-09-19T11:26:12.333061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.791517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.022905+0000","last_change":"2024-09-19T11:23:03.252002+0000","last_active":"2024-09-19T11:25:08.022905+0000","last_peered":"2024-09-19T11:25:08.022905+0000","last_clean":"2024-09-19T11:25:08.022905+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:25:08.022905+0000","last_undegraded":"2024-09-19T11:25:08.022905+0000","last_fullsized":"2024-09-19T11:25:08.022905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:04:47.977517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'11","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470023+0000","last_change":"2024-09-19T11:23:42.285084+0000","last_active":"2024-09-19T11:26:12.470023+0000","last_peered":"2024-09-19T11:26:12.470023+0000","last_clean":"2024-09-19T11:26:12.470023+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:12.470023+0000","last_undegraded":"2024-09-19T11:26:12.470023+0000","last_fullsized":"2024-09-19T11:26:12.470023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:31.524562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.939807+0000","last_change":"2024-09-19T11:23:03.257377+0000","last_active":"2024-09-19T11:23:43.939807+0000","last_peered":"2024-09-19T11:23:43.939807+0000","last_clean":"2024-09-19T11:23:43.939807+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:23:43.939807+0000","last_undegraded":"2024-09-19T11:23:43.939807+0000","last_fullsized":"2024-09-19T11:23:43.939807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:06.788652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944750+0000","last_change":"2024-09-19T11:23:05.233162+0000","last_active":"2024-09-19T11:23:42.944750+0000","last_peered":"2024-09-19T11:23:42.944750+0000","last_clean":"2024-09-19T11:23:42.944750+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:23:42.944750+0000","last_undegraded":"2024-09-19T11:23:42.944750+0000","last_fullsized":"2024-09-19T11:23:42.944750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:49.026520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:36.813838+0000","last_change":"2024-09-19T11:23:03.254504+0000","last_active":"2024-09-19T11:25:36.813838+0000","last_peered":"2024-09-19T11:25:36.813838+0000","last_clean":"2024-09-19T11:25:36.813838+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:25:36.813838+0000","last_undegraded":"2024-09-19T11:25:36.813838+0000","last_fullsized":"2024-09-19T11:25:36.813838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:05.223629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'11","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.128818+0000","last_change":"2024-09-19T11:23:05.247147+0000","last_active":"2024-09-19T11:26:12.128818+0000","last_peered":"2024-09-19T11:26:12.128818+0000","last_clean":"2024-09-19T11:26:12.128818+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:12.128818+0000","last_undegraded":"2024-09-19T11:26:12.128818+0000","last_fullsized":"2024-09-19T11:26:12.128818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:25.162001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276791+0000","last_change":"2024-09-19T11:23:03.256689+0000","last_active":"2024-09-19T11:23:42.276791+0000","last_peered":"2024-09-19T11:23:42.276791+0000","last_clean":"2024-09-19T11:23:42.276791+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:23:42.276791+0000","last_undegraded":"2024-09-19T11:23:42.276791+0000","last_fullsized":"2024-09-19T11:23:42.276791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:09.931084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374918+0000","last_change":"2024-09-19T11:23:05.254262+0000","last_active":"2024-09-19T11:26:12.374918+0000","last_peered":"2024-09-19T11:26:12.374918+0000","last_clean":"2024-09-19T11:26:12.374918+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:26:12.374918+0000","last_undegraded":"2024-09-19T11:26:12.374918+0000","last_fullsized":"2024-09-19T11:26:12.374918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:58.352519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.252828+0000","last_change":"2024-09-19T11:23:03.239664+0000","last_active":"2024-09-19T11:26:12.252828+0000","last_peered":"2024-09-19T11:26:12.252828+0000","last_clean":"2024-09-19T11:26:12.252828+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:12.252828+0000","last_undegraded":"2024-09-19T11:26:12.252828+0000","last_fullsized":"2024-09-19T11:26:12.252828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:43.733405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333282+0000","last_change":"2024-09-19T11:23:05.250554+0000","last_active":"2024-09-19T11:26:12.333282+0000","last_peered":"2024-09-19T11:26:12.333282+0000","last_clean":"2024-09-19T11:26:12.333282+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:12.333282+0000","last_undegraded":"2024-09-19T11:26:12.333282+0000","last_fullsized":"2024-09-19T11:26:12.333282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278484+0000","last_change":"2024-09-19T11:23:03.237565+0000","last_active":"2024-09-19T11:23:42.278484+0000","last_peered":"2024-09-19T11:23:42.278484+0000","last_clean":"2024-09-19T11:23:42.278484+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:23:42.278484+0000","last_undegraded":"2024-09-19T11:23:42.278484+0000","last_fullsized":"2024-09-19T11:23:42.278484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:24:40.441642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.012417+0000","last_change":"2024-09-19T11:23:05.246773+0000","last_active":"2024-09-19T11:23:45.012417+0000","last_peered":"2024-09-19T11:23:45.012417+0000","last_clean":"2024-09-19T11:23:45.012417+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:23:45.012417+0000","last_undegraded":"2024-09-19T11:23:45.012417+0000","last_fullsized":"2024-09-19T11:23:45.012417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:05:02.041530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:05.241849+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:04.673024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283475+0000","last_change":"2024-09-19T11:23:03.250062+0000","last_active":"2024-09-19T11:23:42.283475+0000","last_peered":"2024-09-19T11:23:42.283475+0000","last_clean":"2024-09-19T11:23:42.283475+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:23:42.283475+0000","last_undegraded":"2024-09-19T11:23:42.283475+0000","last_fullsized":"2024-09-19T11:23:42.283475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:32.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.483130+0000","last_change":"2024-09-19T11:23:05.246402+0000","last_active":"2024-09-19T11:23:45.483130+0000","last_peered":"2024-09-19T11:23:45.483130+0000","last_clean":"2024-09-19T11:23:45.483130+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:23:45.483130+0000","last_undegraded":"2024-09-19T11:23:45.483130+0000","last_fullsized":"2024-09-19T11:23:45.483130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:44.280015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281849+0000","last_change":"2024-09-19T11:23:03.259838+0000","last_active":"2024-09-19T11:23:42.281849+0000","last_peered":"2024-09-19T11:23:42.281849+0000","last_clean":"2024-09-19T11:23:42.281849+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:23:42.281849+0000","last_undegraded":"2024-09-19T11:23:42.281849+0000","last_fullsized":"2024-09-19T11:23:42.281849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342502+0000","last_change":"2024-09-19T11:23:05.248326+0000","last_active":"2024-09-19T11:24:27.342502+0000","last_peered":"2024-09-19T11:24:27.342502+0000","last_clean":"2024-09-19T11:24:27.342502+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:24:27.342502+0000","last_undegraded":"2024-09-19T11:24:27.342502+0000","last_fullsized":"2024-09-19T11:24:27.342502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:28.400630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.969817+0000","last_change":"2024-09-19T11:23:03.250122+0000","last_active":"2024-09-19T11:26:11.969817+0000","last_peered":"2024-09-19T11:26:11.969817+0000","last_clean":"2024-09-19T11:26:11.969817+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:11.969817+0000","last_undegraded":"2024-09-19T11:26:11.969817+0000","last_fullsized":"2024-09-19T11:26:11.969817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:13.455767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123608+0000","last_change":"2024-09-19T11:23:05.240918+0000","last_active":"2024-09-19T11:26:12.123608+0000","last_peered":"2024-09-19T11:26:12.123608+0000","last_clean":"2024-09-19T11:26:12.123608+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:12.123608+0000","last_undegraded":"2024-09-19T11:26:12.123608+0000","last_fullsized":"2024-09-19T11:26:12.123608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:19.590735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282416+0000","last_change":"2024-09-19T11:23:03.245456+0000","last_active":"2024-09-19T11:23:42.282416+0000","last_peered":"2024-09-19T11:23:42.282416+0000","last_clean":"2024-09-19T11:23:42.282416+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:23:42.282416+0000","last_undegraded":"2024-09-19T11:23:42.282416+0000","last_fullsized":"2024-09-19T11:23:42.282416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:14.954640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266363+0000","last_change":"2024-09-19T11:23:05.245486+0000","last_active":"2024-09-19T11:26:12.266363+0000","last_peered":"2024-09-19T11:26:12.266363+0000","last_clean":"2024-09-19T11:26:12.266363+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:12.266363+0000","last_undegraded":"2024-09-19T11:26:12.266363+0000","last_fullsized":"2024-09-19T11:26:12.266363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.454541+0000","last_change":"2024-09-19T11:23:03.243483+0000","last_active":"2024-09-19T11:25:11.454541+0000","last_peered":"2024-09-19T11:25:11.454541+0000","last_clean":"2024-09-19T11:25:11.454541+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:25:11.454541+0000","last_undegraded":"2024-09-19T11:25:11.454541+0000","last_fullsized":"2024-09-19T11:25:11.454541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:10.061252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266347+0000","last_change":"2024-09-19T11:23:05.248431+0000","last_active":"2024-09-19T11:26:12.266347+0000","last_peered":"2024-09-19T11:26:12.266347+0000","last_clean":"2024-09-19T11:26:12.266347+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:12.266347+0000","last_undegraded":"2024-09-19T11:26:12.266347+0000","last_fullsized":"2024-09-19T11:26:12.266347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:17.337349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945193+0000","last_change":"2024-09-19T11:23:03.240547+0000","last_active":"2024-09-19T11:23:42.945193+0000","last_peered":"2024-09-19T11:23:42.945193+0000","last_clean":"2024-09-19T11:23:42.945193+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:23:42.945193+0000","last_undegraded":"2024-09-19T11:23:42.945193+0000","last_fullsized":"2024-09-19T11:23:42.945193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:55.218589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'37","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.305388+0000","last_change":"2024-09-19T11:23:13.275524+0000","last_active":"2024-09-19T11:26:12.305388+0000","last_peered":"2024-09-19T11:26:12.305388+0000","last_clean":"2024-09-19T11:26:12.305388+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:26:12.305388+0000","last_undegraded":"2024-09-19T11:26:12.305388+0000","last_fullsized":"2024-09-19T11:26:12.305388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.670701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.018412+0000","last_change":"2024-09-19T11:23:42.283032+0000","last_active":"2024-09-19T11:26:12.018412+0000","last_peered":"2024-09-19T11:26:12.018412+0000","last_clean":"2024-09-19T11:26:12.018412+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:12.018412+0000","last_undegraded":"2024-09-19T11:26:12.018412+0000","last_fullsized":"2024-09-19T11:26:12.018412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.760502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811976+0000","last_change":"2024-09-19T11:23:03.258428+0000","last_active":"2024-09-19T11:25:26.811976+0000","last_peered":"2024-09-19T11:25:26.811976+0000","last_clean":"2024-09-19T11:25:26.811976+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:25:26.811976+0000","last_undegraded":"2024-09-19T11:25:26.811976+0000","last_fullsized":"2024-09-19T11:25:26.811976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:51.654373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945113+0000","last_change":"2024-09-19T11:23:07.245040+0000","last_active":"2024-09-19T11:23:42.945113+0000","last_peered":"2024-09-19T11:23:42.945113+0000","last_clean":"2024-09-19T11:23:42.945113+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:23:42.945113+0000","last_undegraded":"2024-09-19T11:23:42.945113+0000","last_fullsized":"2024-09-19T11:23:42.945113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.056714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.749025+0000","last_change":"2024-09-19T11:23:09.256765+0000","last_active":"2024-09-19T11:25:09.749025+0000","last_peered":"2024-09-19T11:25:09.749025+0000","last_clean":"2024-09-19T11:25:09.749025+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:25:09.749025+0000","last_undegraded":"2024-09-19T11:25:09.749025+0000","last_fullsized":"2024-09-19T11:25:09.749025+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:39:32.451376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283356+0000","last_change":"2024-09-19T11:23:11.267021+0000","last_active":"2024-09-19T11:23:42.283356+0000","last_peered":"2024-09-19T11:23:42.283356+0000","last_clean":"2024-09-19T11:23:42.283356+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:23:42.283356+0000","last_undegraded":"2024-09-19T11:23:42.283356+0000","last_fullsized":"2024-09-19T11:23:42.283356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.806510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.707113+0000","last_change":"2024-09-19T11:23:13.274325+0000","last_active":"2024-09-19T11:26:11.707113+0000","last_peered":"2024-09-19T11:26:11.707113+0000","last_clean":"2024-09-19T11:26:11.707113+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:26:11.707113+0000","last_undegraded":"2024-09-19T11:26:11.707113+0000","last_fullsized":"2024-09-19T11:26:11.707113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:53:22.517816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.332597+0000","last_change":"2024-09-19T11:23:05.237978+0000","last_active":"2024-09-19T11:26:12.332597+0000","last_peered":"2024-09-19T11:26:12.332597+0000","last_clean":"2024-09-19T11:26:12.332597+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:12.332597+0000","last_undegraded":"2024-09-19T11:26:12.332597+0000","last_fullsized":"2024-09-19T11:26:12.332597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:02.964073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.319875+0000","last_change":"2024-09-19T11:23:03.256489+0000","last_active":"2024-09-19T11:25:11.319875+0000","last_peered":"2024-09-19T11:25:11.319875+0000","last_clean":"2024-09-19T11:25:11.319875+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:25:11.319875+0000","last_undegraded":"2024-09-19T11:25:11.319875+0000","last_fullsized":"2024-09-19T11:25:11.319875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:34.312130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279716+0000","last_change":"2024-09-19T11:23:07.245213+0000","last_active":"2024-09-19T11:23:42.279716+0000","last_peered":"2024-09-19T11:23:42.279716+0000","last_clean":"2024-09-19T11:23:42.279716+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:23:42.279716+0000","last_undegraded":"2024-09-19T11:23:42.279716+0000","last_fullsized":"2024-09-19T11:23:42.279716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:56.312564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'7","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.551668+0000","last_change":"2024-09-19T11:23:09.248257+0000","last_active":"2024-09-19T11:24:27.551668+0000","last_peered":"2024-09-19T11:24:27.551668+0000","last_clean":"2024-09-19T11:24:27.551668+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:24:27.551668+0000","last_undegraded":"2024-09-19T11:24:27.551668+0000","last_fullsized":"2024-09-19T11:24:27.551668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:18.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281824+0000","last_change":"2024-09-19T11:23:11.263544+0000","last_active":"2024-09-19T11:23:42.281824+0000","last_peered":"2024-09-19T11:23:42.281824+0000","last_clean":"2024-09-19T11:23:42.281824+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:23:42.281824+0000","last_undegraded":"2024-09-19T11:23:42.281824+0000","last_fullsized":"2024-09-19T11:23:42.281824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:44.652949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278313+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:23:42.278313+0000","last_peered":"2024-09-19T11:23:42.278313+0000","last_clean":"2024-09-19T11:23:42.278313+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:23:42.278313+0000","last_undegraded":"2024-09-19T11:23:42.278313+0000","last_fullsized":"2024-09-19T11:23:42.278313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:33:33.618068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510054+0000","last_change":"2024-09-19T11:23:05.236597+0000","last_active":"2024-09-19T11:25:11.510054+0000","last_peered":"2024-09-19T11:25:11.510054+0000","last_clean":"2024-09-19T11:25:11.510054+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:25:11.510054+0000","last_undegraded":"2024-09-19T11:25:11.510054+0000","last_fullsized":"2024-09-19T11:25:11.510054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:11.702021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.486679+0000","last_change":"2024-09-19T11:23:03.257697+0000","last_active":"2024-09-19T11:25:11.486679+0000","last_peered":"2024-09-19T11:25:11.486679+0000","last_clean":"2024-09-19T11:25:11.486679+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:25:11.486679+0000","last_undegraded":"2024-09-19T11:25:11.486679+0000","last_fullsized":"2024-09-19T11:25:11.486679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:55.338804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796210+0000","last_change":"2024-09-19T11:23:42.284049+0000","last_active":"2024-09-19T11:23:44.796210+0000","last_peered":"2024-09-19T11:23:44.796210+0000","last_clean":"2024-09-19T11:23:44.796210+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:23:44.796210+0000","last_undegraded":"2024-09-19T11:23:44.796210+0000","last_fullsized":"2024-09-19T11:23:44.796210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279160+0000","last_change":"2024-09-19T11:23:09.253591+0000","last_active":"2024-09-19T11:23:42.279160+0000","last_peered":"2024-09-19T11:23:42.279160+0000","last_clean":"2024-09-19T11:23:42.279160+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:23:42.279160+0000","last_undegraded":"2024-09-19T11:23:42.279160+0000","last_fullsized":"2024-09-19T11:23:42.279160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:52.970909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'282","reported_seq":386,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.615383+0000","last_change":"2024-09-19T11:23:11.271056+0000","last_active":"2024-09-19T11:26:12.615383+0000","last_peered":"2024-09-19T11:26:12.615383+0000","last_clean":"2024-09-19T11:26:12.615383+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:26:12.615383+0000","last_undegraded":"2024-09-19T11:26:12.615383+0000","last_fullsized":"2024-09-19T11:26:12.615383+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:07.822761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.312802+0000","last_change":"2024-09-19T11:23:13.270157+0000","last_active":"2024-09-19T11:26:12.312802+0000","last_peered":"2024-09-19T11:26:12.312802+0000","last_clean":"2024-09-19T11:26:12.312802+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:26:12.312802+0000","last_undegraded":"2024-09-19T11:26:12.312802+0000","last_fullsized":"2024-09-19T11:26:12.312802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:32.371537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.781965+0000","last_change":"2024-09-19T11:23:05.248712+0000","last_active":"2024-09-19T11:23:44.781965+0000","last_peered":"2024-09-19T11:23:44.781965+0000","last_clean":"2024-09-19T11:23:44.781965+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:23:44.781965+0000","last_undegraded":"2024-09-19T11:23:44.781965+0000","last_fullsized":"2024-09-19T11:23:44.781965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:01:31.288073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279237+0000","last_change":"2024-09-19T11:23:03.250291+0000","last_active":"2024-09-19T11:23:42.279237+0000","last_peered":"2024-09-19T11:23:42.279237+0000","last_clean":"2024-09-19T11:23:42.279237+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:23:42.279237+0000","last_undegraded":"2024-09-19T11:23:42.279237+0000","last_fullsized":"2024-09-19T11:23:42.279237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:27.615110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947691+0000","last_change":"2024-09-19T11:23:07.248024+0000","last_active":"2024-09-19T11:23:42.947691+0000","last_peered":"2024-09-19T11:23:42.947691+0000","last_clean":"2024-09-19T11:23:42.947691+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:23:42.947691+0000","last_undegraded":"2024-09-19T11:23:42.947691+0000","last_fullsized":"2024-09-19T11:23:42.947691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:02.812050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.743749+0000","last_change":"2024-09-19T11:23:09.260866+0000","last_active":"2024-09-19T11:25:09.743749+0000","last_peered":"2024-09-19T11:25:09.743749+0000","last_clean":"2024-09-19T11:25:09.743749+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:25:09.743749+0000","last_undegraded":"2024-09-19T11:25:09.743749+0000","last_fullsized":"2024-09-19T11:25:09.743749+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:22.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'282","reported_seq":402,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.371212+0000","last_change":"2024-09-19T11:23:11.260474+0000","last_active":"2024-09-19T11:26:12.371212+0000","last_peered":"2024-09-19T11:26:12.371212+0000","last_clean":"2024-09-19T11:26:12.371212+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:26:12.371212+0000","last_undegraded":"2024-09-19T11:26:12.371212+0000","last_fullsized":"2024-09-19T11:26:12.371212+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:32.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032873+0000","last_change":"2024-09-19T11:23:05.253843+0000","last_active":"2024-09-19T11:25:08.032873+0000","last_peered":"2024-09-19T11:25:08.032873+0000","last_clean":"2024-09-19T11:25:08.032873+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:25:08.032873+0000","last_undegraded":"2024-09-19T11:25:08.032873+0000","last_fullsized":"2024-09-19T11:25:08.032873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:22.232407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279601+0000","last_change":"2024-09-19T11:23:03.256377+0000","last_active":"2024-09-19T11:23:42.279601+0000","last_peered":"2024-09-19T11:23:42.279601+0000","last_clean":"2024-09-19T11:23:42.279601+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:23:42.279601+0000","last_undegraded":"2024-09-19T11:23:42.279601+0000","last_fullsized":"2024-09-19T11:23:42.279601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:29.305136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281170+0000","last_change":"2024-09-19T11:23:07.244371+0000","last_active":"2024-09-19T11:23:42.281170+0000","last_peered":"2024-09-19T11:23:42.281170+0000","last_clean":"2024-09-19T11:23:42.281170+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:23:42.281170+0000","last_undegraded":"2024-09-19T11:23:42.281170+0000","last_fullsized":"2024-09-19T11:23:42.281170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:53.267571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'40","reported_seq":88,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.753372+0000","last_change":"2024-09-19T11:23:09.249075+0000","last_active":"2024-09-19T11:25:09.753372+0000","last_peered":"2024-09-19T11:25:09.753372+0000","last_clean":"2024-09-19T11:25:09.753372+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:25:09.753372+0000","last_undegraded":"2024-09-19T11:25:09.753372+0000","last_fullsized":"2024-09-19T11:25:09.753372+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:03.176195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281316+0000","last_change":"2024-09-19T11:23:11.258275+0000","last_active":"2024-09-19T11:23:42.281316+0000","last_peered":"2024-09-19T11:23:42.281316+0000","last_clean":"2024-09-19T11:23:42.281316+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:23:42.281316+0000","last_undegraded":"2024-09-19T11:23:42.281316+0000","last_fullsized":"2024-09-19T11:23:42.281316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:14.017912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.349482+0000","last_change":"2024-09-19T11:23:13.274689+0000","last_active":"2024-09-19T11:24:27.349482+0000","last_peered":"2024-09-19T11:24:27.349482+0000","last_clean":"2024-09-19T11:24:27.349482+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:24:27.349482+0000","last_undegraded":"2024-09-19T11:24:27.349482+0000","last_fullsized":"2024-09-19T11:24:27.349482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:05.210803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281462+0000","last_change":"2024-09-19T11:23:05.245370+0000","last_active":"2024-09-19T11:23:42.281462+0000","last_peered":"2024-09-19T11:23:42.281462+0000","last_clean":"2024-09-19T11:23:42.281462+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:23:42.281462+0000","last_undegraded":"2024-09-19T11:23:42.281462+0000","last_fullsized":"2024-09-19T11:23:42.281462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:55:45.570918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280712+0000","last_change":"2024-09-19T11:23:03.257588+0000","last_active":"2024-09-19T11:23:42.280712+0000","last_peered":"2024-09-19T11:23:42.280712+0000","last_clean":"2024-09-19T11:23:42.280712+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:23:42.280712+0000","last_undegraded":"2024-09-19T11:23:42.280712+0000","last_fullsized":"2024-09-19T11:23:42.280712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.928791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278703+0000","last_change":"2024-09-19T11:23:07.247386+0000","last_active":"2024-09-19T11:23:42.278703+0000","last_peered":"2024-09-19T11:23:42.278703+0000","last_clean":"2024-09-19T11:23:42.278703+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:23:42.278703+0000","last_undegraded":"2024-09-19T11:23:42.278703+0000","last_fullsized":"2024-09-19T11:23:42.278703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:53.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'30","reported_seq":73,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.733747+0000","last_change":"2024-09-19T11:23:09.262668+0000","last_active":"2024-09-19T11:25:09.733747+0000","last_peered":"2024-09-19T11:25:09.733747+0000","last_clean":"2024-09-19T11:25:09.733747+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:25:09.733747+0000","last_undegraded":"2024-09-19T11:25:09.733747+0000","last_fullsized":"2024-09-19T11:25:09.733747+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:42.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278485+0000","last_change":"2024-09-19T11:23:11.265649+0000","last_active":"2024-09-19T11:23:42.278485+0000","last_peered":"2024-09-19T11:23:42.278485+0000","last_clean":"2024-09-19T11:23:42.278485+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:23:42.278485+0000","last_undegraded":"2024-09-19T11:23:42.278485+0000","last_fullsized":"2024-09-19T11:23:42.278485+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:22.946739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'5","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.124970+0000","last_change":"2024-09-19T11:23:13.268845+0000","last_active":"2024-09-19T11:26:12.124970+0000","last_peered":"2024-09-19T11:26:12.124970+0000","last_clean":"2024-09-19T11:26:12.124970+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:26:12.124970+0000","last_undegraded":"2024-09-19T11:26:12.124970+0000","last_fullsized":"2024-09-19T11:26:12.124970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:30.688522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342674+0000","last_change":"2024-09-19T11:23:05.251302+0000","last_active":"2024-09-19T11:24:27.342674+0000","last_peered":"2024-09-19T11:24:27.342674+0000","last_clean":"2024-09-19T11:24:27.342674+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:24:27.342674+0000","last_undegraded":"2024-09-19T11:24:27.342674+0000","last_fullsized":"2024-09-19T11:24:27.342674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:55.904446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'3","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.249995+0000","last_change":"2024-09-19T11:23:03.242612+0000","last_active":"2024-09-19T11:26:12.249995+0000","last_peered":"2024-09-19T11:26:12.249995+0000","last_clean":"2024-09-19T11:26:12.249995+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:12.249995+0000","last_undegraded":"2024-09-19T11:26:12.249995+0000","last_fullsized":"2024-09-19T11:26:12.249995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:48.666399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":11,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281855+0000","last_change":"2024-09-19T11:23:07.247594+0000","last_active":"2024-09-19T11:23:42.281855+0000","last_peered":"2024-09-19T11:23:42.281855+0000","last_clean":"2024-09-19T11:23:42.281855+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:23:42.281855+0000","last_undegraded":"2024-09-19T11:23:42.281855+0000","last_fullsized":"2024-09-19T11:23:42.281855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:41.108832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.755394+0000","last_change":"2024-09-19T11:23:09.259476+0000","last_active":"2024-09-19T11:25:09.755394+0000","last_peered":"2024-09-19T11:25:09.755394+0000","last_clean":"2024-09-19T11:25:09.755394+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:25:09.755394+0000","last_undegraded":"2024-09-19T11:25:09.755394+0000","last_fullsized":"2024-09-19T11:25:09.755394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:21.364894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'282","reported_seq":349,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.641675+0000","last_change":"2024-09-19T11:23:11.267050+0000","last_active":"2024-09-19T11:26:12.641675+0000","last_peered":"2024-09-19T11:26:12.641675+0000","last_clean":"2024-09-19T11:26:12.641675+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:26:12.641675+0000","last_undegraded":"2024-09-19T11:26:12.641675+0000","last_fullsized":"2024-09-19T11:26:12.641675+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.330204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.334604+0000","last_change":"2024-09-19T11:23:13.269663+0000","last_active":"2024-09-19T11:26:12.334604+0000","last_peered":"2024-09-19T11:26:12.334604+0000","last_clean":"2024-09-19T11:26:12.334604+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:26:12.334604+0000","last_undegraded":"2024-09-19T11:26:12.334604+0000","last_fullsized":"2024-09-19T11:26:12.334604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:57:11.742213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368223+0000","last_change":"2024-09-19T11:23:05.235632+0000","last_active":"2024-09-19T11:26:12.368223+0000","last_peered":"2024-09-19T11:26:12.368223+0000","last_clean":"2024-09-19T11:26:12.368223+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:12.368223+0000","last_undegraded":"2024-09-19T11:26:12.368223+0000","last_fullsized":"2024-09-19T11:26:12.368223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945253+0000","last_change":"2024-09-19T11:23:03.257570+0000","last_active":"2024-09-19T11:23:42.945253+0000","last_peered":"2024-09-19T11:23:42.945253+0000","last_clean":"2024-09-19T11:23:42.945253+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:23:42.945253+0000","last_undegraded":"2024-09-19T11:23:42.945253+0000","last_fullsized":"2024-09-19T11:23:42.945253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:41:29.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279784+0000","last_change":"2024-09-19T11:23:07.247239+0000","last_active":"2024-09-19T11:23:42.279784+0000","last_peered":"2024-09-19T11:23:42.279784+0000","last_clean":"2024-09-19T11:23:42.279784+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:23:42.279784+0000","last_undegraded":"2024-09-19T11:23:42.279784+0000","last_fullsized":"2024-09-19T11:23:42.279784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:21.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'55","reported_seq":112,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.512267+0000","last_change":"2024-09-19T11:23:09.259576+0000","last_active":"2024-09-19T11:25:11.512267+0000","last_peered":"2024-09-19T11:25:11.512267+0000","last_clean":"2024-09-19T11:25:11.512267+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:25:11.512267+0000","last_undegraded":"2024-09-19T11:25:11.512267+0000","last_fullsized":"2024-09-19T11:25:11.512267+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.556001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278323+0000","last_change":"2024-09-19T11:23:11.260403+0000","last_active":"2024-09-19T11:23:42.278323+0000","last_peered":"2024-09-19T11:23:42.278323+0000","last_clean":"2024-09-19T11:23:42.278323+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:23:42.278323+0000","last_undegraded":"2024-09-19T11:23:42.278323+0000","last_fullsized":"2024-09-19T11:23:42.278323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:11.335945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'18","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585256+0000","last_change":"2024-09-19T11:23:13.273720+0000","last_active":"2024-09-19T11:26:12.585256+0000","last_peered":"2024-09-19T11:26:12.585256+0000","last_clean":"2024-09-19T11:26:12.585256+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:26:12.585256+0000","last_undegraded":"2024-09-19T11:26:12.585256+0000","last_fullsized":"2024-09-19T11:26:12.585256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:46.547393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:21.810844+0000","last_change":"2024-09-19T11:23:05.247032+0000","last_active":"2024-09-19T11:25:21.810844+0000","last_peered":"2024-09-19T11:25:21.810844+0000","last_clean":"2024-09-19T11:25:21.810844+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:25:21.810844+0000","last_undegraded":"2024-09-19T11:25:21.810844+0000","last_fullsized":"2024-09-19T11:25:21.810844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:24.496941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.255786+0000","last_change":"2024-09-19T11:23:03.234223+0000","last_active":"2024-09-19T11:26:12.255786+0000","last_peered":"2024-09-19T11:26:12.255786+0000","last_clean":"2024-09-19T11:26:12.255786+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:12.255786+0000","last_undegraded":"2024-09-19T11:26:12.255786+0000","last_fullsized":"2024-09-19T11:26:12.255786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278469+0000","last_change":"2024-09-19T11:23:07.244885+0000","last_active":"2024-09-19T11:23:42.278469+0000","last_peered":"2024-09-19T11:23:42.278469+0000","last_clean":"2024-09-19T11:23:42.278469+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:23:42.278469+0000","last_undegraded":"2024-09-19T11:23:42.278469+0000","last_fullsized":"2024-09-19T11:23:42.278469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:45:02.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'33","reported_seq":77,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811791+0000","last_change":"2024-09-19T11:23:09.248190+0000","last_active":"2024-09-19T11:25:26.811791+0000","last_peered":"2024-09-19T11:25:26.811791+0000","last_clean":"2024-09-19T11:25:26.811791+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:25:26.811791+0000","last_undegraded":"2024-09-19T11:25:26.811791+0000","last_fullsized":"2024-09-19T11:25:26.811791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:38.180379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946607+0000","last_change":"2024-09-19T11:23:11.270709+0000","last_active":"2024-09-19T11:23:42.946607+0000","last_peered":"2024-09-19T11:23:42.946607+0000","last_clean":"2024-09-19T11:23:42.946607+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:23:42.946607+0000","last_undegraded":"2024-09-19T11:23:42.946607+0000","last_fullsized":"2024-09-19T11:23:42.946607+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:39.157590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809883+0000","last_change":"2024-09-19T11:23:13.272542+0000","last_active":"2024-09-19T11:25:16.809883+0000","last_peered":"2024-09-19T11:25:16.809883+0000","last_clean":"2024-09-19T11:25:16.809883+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:25:16.809883+0000","last_undegraded":"2024-09-19T11:25:16.809883+0000","last_fullsized":"2024-09-19T11:25:16.809883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:52.793325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433382+0000","last_change":"2024-09-19T11:23:05.254593+0000","last_active":"2024-09-19T11:23:44.433382+0000","last_peered":"2024-09-19T11:23:44.433382+0000","last_clean":"2024-09-19T11:23:44.433382+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:23:44.433382+0000","last_undegraded":"2024-09-19T11:23:44.433382+0000","last_fullsized":"2024-09-19T11:23:44.433382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:27.816142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945690+0000","last_change":"2024-09-19T11:23:03.241691+0000","last_active":"2024-09-19T11:23:42.945690+0000","last_peered":"2024-09-19T11:23:42.945690+0000","last_clean":"2024-09-19T11:23:42.945690+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:23:42.945690+0000","last_undegraded":"2024-09-19T11:23:42.945690+0000","last_fullsized":"2024-09-19T11:23:42.945690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:21.286478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279479+0000","last_change":"2024-09-19T11:23:07.247411+0000","last_active":"2024-09-19T11:23:42.279479+0000","last_peered":"2024-09-19T11:23:42.279479+0000","last_clean":"2024-09-19T11:23:42.279479+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:23:42.279479+0000","last_undegraded":"2024-09-19T11:23:42.279479+0000","last_fullsized":"2024-09-19T11:23:42.279479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:20:41.691053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.763568+0000","last_change":"2024-09-19T11:23:09.246515+0000","last_active":"2024-09-19T11:25:09.763568+0000","last_peered":"2024-09-19T11:25:09.763568+0000","last_clean":"2024-09-19T11:25:09.763568+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:25:09.763568+0000","last_undegraded":"2024-09-19T11:25:09.763568+0000","last_fullsized":"2024-09-19T11:25:09.763568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:54.969396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'282","reported_seq":404,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.589475+0000","last_change":"2024-09-19T11:23:11.263693+0000","last_active":"2024-09-19T11:26:12.589475+0000","last_peered":"2024-09-19T11:26:12.589475+0000","last_clean":"2024-09-19T11:26:12.589475+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:26:12.589475+0000","last_undegraded":"2024-09-19T11:26:12.589475+0000","last_fullsized":"2024-09-19T11:26:12.589475+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:29.980859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'14","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.475120+0000","last_change":"2024-09-19T11:23:13.265074+0000","last_active":"2024-09-19T11:26:12.475120+0000","last_peered":"2024-09-19T11:26:12.475120+0000","last_clean":"2024-09-19T11:26:12.475120+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:26:12.475120+0000","last_undegraded":"2024-09-19T11:26:12.475120+0000","last_fullsized":"2024-09-19T11:26:12.475120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:47.118156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368115+0000","last_change":"2024-09-19T11:23:05.250216+0000","last_active":"2024-09-19T11:26:12.368115+0000","last_peered":"2024-09-19T11:26:12.368115+0000","last_clean":"2024-09-19T11:26:12.368115+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:12.368115+0000","last_undegraded":"2024-09-19T11:26:12.368115+0000","last_fullsized":"2024-09-19T11:26:12.368115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.941399+0000","last_change":"2024-09-19T11:23:03.255564+0000","last_active":"2024-09-19T11:23:43.941399+0000","last_peered":"2024-09-19T11:23:43.941399+0000","last_clean":"2024-09-19T11:23:43.941399+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:23:43.941399+0000","last_undegraded":"2024-09-19T11:23:43.941399+0000","last_fullsized":"2024-09-19T11:23:43.941399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:10:03.443220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'3","reported_seq":414,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.866528+0000","last_change":"2024-09-19T11:23:07.242151+0000","last_active":"2024-09-19T11:25:10.866528+0000","last_peered":"2024-09-19T11:25:10.866528+0000","last_clean":"2024-09-19T11:25:10.866528+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:25:10.866528+0000","last_undegraded":"2024-09-19T11:25:10.866528+0000","last_fullsized":"2024-09-19T11:25:10.866528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:15.211742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946255+0000","last_change":"2024-09-19T11:23:09.255928+0000","last_active":"2024-09-19T11:23:42.946255+0000","last_peered":"2024-09-19T11:23:42.946255+0000","last_clean":"2024-09-19T11:23:42.946255+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:23:42.946255+0000","last_undegraded":"2024-09-19T11:23:42.946255+0000","last_fullsized":"2024-09-19T11:23:42.946255+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:04.689951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'282","reported_seq":466,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.473914+0000","last_change":"2024-09-19T11:23:11.255933+0000","last_active":"2024-09-19T11:26:12.473914+0000","last_peered":"2024-09-19T11:26:12.473914+0000","last_clean":"2024-09-19T11:26:12.473914+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:26:12.473914+0000","last_undegraded":"2024-09-19T11:26:12.473914+0000","last_fullsized":"2024-09-19T11:26:12.473914+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:01.160503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.141728+0000","last_change":"2024-09-19T11:23:13.266957+0000","last_active":"2024-09-19T11:25:11.141728+0000","last_peered":"2024-09-19T11:25:11.141728+0000","last_clean":"2024-09-19T11:25:11.141728+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:25:11.141728+0000","last_undegraded":"2024-09-19T11:25:11.141728+0000","last_fullsized":"2024-09-19T11:25:11.141728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:59.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":101,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338578+0000","last_change":"2024-09-19T11:23:05.255344+0000","last_active":"2024-09-19T11:26:12.338578+0000","last_peered":"2024-09-19T11:26:12.338578+0000","last_clean":"2024-09-19T11:26:12.338578+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:12.338578+0000","last_undegraded":"2024-09-19T11:26:12.338578+0000","last_fullsized":"2024-09-19T11:26:12.338578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:22.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944743+0000","last_change":"2024-09-19T11:23:03.254989+0000","last_active":"2024-09-19T11:23:42.944743+0000","last_peered":"2024-09-19T11:23:42.944743+0000","last_clean":"2024-09-19T11:23:42.944743+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:23:42.944743+0000","last_undegraded":"2024-09-19T11:23:42.944743+0000","last_fullsized":"2024-09-19T11:23:42.944743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:39.840111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":416,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.866830+0000","last_change":"2024-09-19T11:23:07.250416+0000","last_active":"2024-09-19T11:25:10.866830+0000","last_peered":"2024-09-19T11:25:10.866830+0000","last_clean":"2024-09-19T11:25:10.866830+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:25:10.866830+0000","last_undegraded":"2024-09-19T11:25:10.866830+0000","last_fullsized":"2024-09-19T11:25:10.866830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":580,"num_read_kb":580,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'38","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.750105+0000","last_change":"2024-09-19T11:23:09.247733+0000","last_active":"2024-09-19T11:25:09.750105+0000","last_peered":"2024-09-19T11:25:09.750105+0000","last_clean":"2024-09-19T11:25:09.750105+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:25:09.750105+0000","last_undegraded":"2024-09-19T11:25:09.750105+0000","last_fullsized":"2024-09-19T11:25:09.750105+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:38.921761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279060+0000","last_change":"2024-09-19T11:23:11.266008+0000","last_active":"2024-09-19T11:23:42.279060+0000","last_peered":"2024-09-19T11:23:42.279060+0000","last_clean":"2024-09-19T11:23:42.279060+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:23:42.279060+0000","last_undegraded":"2024-09-19T11:23:42.279060+0000","last_fullsized":"2024-09-19T11:23:42.279060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:38:43.871781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.426809+0000","last_change":"2024-09-19T11:23:13.263896+0000","last_active":"2024-09-19T11:26:12.426809+0000","last_peered":"2024-09-19T11:26:12.426809+0000","last_clean":"2024-09-19T11:26:12.426809+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:26:12.426809+0000","last_undegraded":"2024-09-19T11:26:12.426809+0000","last_fullsized":"2024-09-19T11:26:12.426809+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:35.364893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'7","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.273438+0000","last_change":"2024-09-19T11:23:05.254167+0000","last_active":"2024-09-19T11:26:12.273438+0000","last_peered":"2024-09-19T11:26:12.273438+0000","last_clean":"2024-09-19T11:26:12.273438+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:12.273438+0000","last_undegraded":"2024-09-19T11:26:12.273438+0000","last_fullsized":"2024-09-19T11:26:12.273438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:44.425539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":146,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279138+0000","last_change":"2024-09-19T11:22:46.832286+0000","last_active":"2024-09-19T11:23:42.279138+0000","last_peered":"2024-09-19T11:23:42.279138+0000","last_clean":"2024-09-19T11:23:42.279138+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:23:42.279138+0000","last_undegraded":"2024-09-19T11:23:42.279138+0000","last_fullsized":"2024-09-19T11:23:42.279138+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_clean_scrub_stamp":"2024-09-19T11:22:45.784999+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:24.575130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944159+0000","last_change":"2024-09-19T11:22:55.817718+0000","last_active":"2024-09-19T11:23:42.944159+0000","last_peered":"2024-09-19T11:23:42.944159+0000","last_clean":"2024-09-19T11:23:42.944159+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:23:42.944159+0000","last_undegraded":"2024-09-19T11:23:42.944159+0000","last_fullsized":"2024-09-19T11:23:42.944159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:33.890847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944557+0000","last_change":"2024-09-19T11:23:03.258130+0000","last_active":"2024-09-19T11:23:42.944557+0000","last_peered":"2024-09-19T11:23:42.944557+0000","last_clean":"2024-09-19T11:23:42.944557+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:23:42.944557+0000","last_undegraded":"2024-09-19T11:23:42.944557+0000","last_fullsized":"2024-09-19T11:23:42.944557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:11.381818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281740+0000","last_change":"2024-09-19T11:23:07.238893+0000","last_active":"2024-09-19T11:23:42.281740+0000","last_peered":"2024-09-19T11:23:42.281740+0000","last_clean":"2024-09-19T11:23:42.281740+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:23:42.281740+0000","last_undegraded":"2024-09-19T11:23:42.281740+0000","last_fullsized":"2024-09-19T11:23:42.281740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:35.034936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'76","reported_seq":146,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.761411+0000","last_change":"2024-09-19T11:23:09.260843+0000","last_active":"2024-09-19T11:25:09.761411+0000","last_peered":"2024-09-19T11:25:09.761411+0000","last_clean":"2024-09-19T11:25:09.761411+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:25:09.761411+0000","last_undegraded":"2024-09-19T11:25:09.761411+0000","last_fullsized":"2024-09-19T11:25:09.761411+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:17.846685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280169+0000","last_change":"2024-09-19T11:23:11.266314+0000","last_active":"2024-09-19T11:23:42.280169+0000","last_peered":"2024-09-19T11:23:42.280169+0000","last_clean":"2024-09-19T11:23:42.280169+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:23:42.280169+0000","last_undegraded":"2024-09-19T11:23:42.280169+0000","last_fullsized":"2024-09-19T11:23:42.280169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:28.832170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":163,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.377387+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:26:12.377387+0000","last_peered":"2024-09-19T11:26:12.377387+0000","last_clean":"2024-09-19T11:26:12.377387+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:26:12.377387+0000","last_undegraded":"2024-09-19T11:26:12.377387+0000","last_fullsized":"2024-09-19T11:26:12.377387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:51.720881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'5","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.810212+0000","last_change":"2024-09-19T11:23:05.233784+0000","last_active":"2024-09-19T11:25:16.810212+0000","last_peered":"2024-09-19T11:25:16.810212+0000","last_clean":"2024-09-19T11:25:16.810212+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:25:16.810212+0000","last_undegraded":"2024-09-19T11:25:16.810212+0000","last_fullsized":"2024-09-19T11:25:16.810212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943897+0000","last_change":"2024-09-19T11:22:55.817444+0000","last_active":"2024-09-19T11:23:42.943897+0000","last_peered":"2024-09-19T11:23:42.943897+0000","last_clean":"2024-09-19T11:23:42.943897+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:23:42.943897+0000","last_undegraded":"2024-09-19T11:23:42.943897+0000","last_fullsized":"2024-09-19T11:23:42.943897+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:39.661543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.805962+0000","last_change":"2024-09-19T11:23:03.236740+0000","last_active":"2024-09-19T11:25:10.805962+0000","last_peered":"2024-09-19T11:25:10.805962+0000","last_clean":"2024-09-19T11:25:10.805962+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:25:10.805962+0000","last_undegraded":"2024-09-19T11:25:10.805962+0000","last_fullsized":"2024-09-19T11:25:10.805962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:40.283716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948430+0000","last_change":"2024-09-19T11:23:07.246124+0000","last_active":"2024-09-19T11:23:42.948430+0000","last_peered":"2024-09-19T11:23:42.948430+0000","last_clean":"2024-09-19T11:23:42.948430+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:23:42.948430+0000","last_undegraded":"2024-09-19T11:23:42.948430+0000","last_fullsized":"2024-09-19T11:23:42.948430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'273","reported_seq":477,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327313+0000","last_change":"2024-09-19T11:23:09.257240+0000","last_active":"2024-09-19T11:26:12.327313+0000","last_peered":"2024-09-19T11:26:12.327313+0000","last_clean":"2024-09-19T11:26:12.327313+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:26:12.327313+0000","last_undegraded":"2024-09-19T11:26:12.327313+0000","last_fullsized":"2024-09-19T11:26:12.327313+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:14.514960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944824+0000","last_change":"2024-09-19T11:23:11.270807+0000","last_active":"2024-09-19T11:23:42.944824+0000","last_peered":"2024-09-19T11:23:42.944824+0000","last_clean":"2024-09-19T11:23:42.944824+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:23:42.944824+0000","last_undegraded":"2024-09-19T11:23:42.944824+0000","last_fullsized":"2024-09-19T11:23:42.944824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:23.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.346118+0000","last_change":"2024-09-19T11:23:13.269927+0000","last_active":"2024-09-19T11:24:27.346118+0000","last_peered":"2024-09-19T11:24:27.346118+0000","last_clean":"2024-09-19T11:24:27.346118+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:24:27.346118+0000","last_undegraded":"2024-09-19T11:24:27.346118+0000","last_fullsized":"2024-09-19T11:24:27.346118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:59.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123437+0000","last_change":"2024-09-19T11:23:05.248320+0000","last_active":"2024-09-19T11:26:12.123437+0000","last_peered":"2024-09-19T11:26:12.123437+0000","last_clean":"2024-09-19T11:26:12.123437+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:12.123437+0000","last_undegraded":"2024-09-19T11:26:12.123437+0000","last_fullsized":"2024-09-19T11:26:12.123437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:59.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276942+0000","last_change":"2024-09-19T11:22:55.886390+0000","last_active":"2024-09-19T11:23:42.276942+0000","last_peered":"2024-09-19T11:23:42.276942+0000","last_clean":"2024-09-19T11:23:42.276942+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:23:42.276942+0000","last_undegraded":"2024-09-19T11:23:42.276942+0000","last_fullsized":"2024-09-19T11:23:42.276942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:42.873153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.275364+0000","last_change":"2024-09-19T11:23:03.244094+0000","last_active":"2024-09-19T11:25:11.275364+0000","last_peered":"2024-09-19T11:25:11.275364+0000","last_clean":"2024-09-19T11:25:11.275364+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:25:11.275364+0000","last_undegraded":"2024-09-19T11:25:11.275364+0000","last_fullsized":"2024-09-19T11:25:11.275364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280552+0000","last_change":"2024-09-19T11:23:07.245296+0000","last_active":"2024-09-19T11:23:42.280552+0000","last_peered":"2024-09-19T11:23:42.280552+0000","last_clean":"2024-09-19T11:23:42.280552+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:23:42.280552+0000","last_undegraded":"2024-09-19T11:23:42.280552+0000","last_fullsized":"2024-09-19T11:23:42.280552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:23:05.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'16","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327294+0000","last_change":"2024-09-19T11:23:09.249984+0000","last_active":"2024-09-19T11:26:12.327294+0000","last_peered":"2024-09-19T11:26:12.327294+0000","last_clean":"2024-09-19T11:26:12.327294+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:12.327294+0000","last_undegraded":"2024-09-19T11:26:12.327294+0000","last_fullsized":"2024-09-19T11:26:12.327294+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:47.233637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280391+0000","last_change":"2024-09-19T11:23:11.265633+0000","last_active":"2024-09-19T11:23:42.280391+0000","last_peered":"2024-09-19T11:23:42.280391+0000","last_clean":"2024-09-19T11:23:42.280391+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:23:42.280391+0000","last_undegraded":"2024-09-19T11:23:42.280391+0000","last_fullsized":"2024-09-19T11:23:42.280391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:16.181121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943722+0000","last_change":"2024-09-19T11:23:13.273373+0000","last_active":"2024-09-19T11:23:42.943722+0000","last_peered":"2024-09-19T11:23:42.943722+0000","last_clean":"2024-09-19T11:23:42.943722+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:23:42.943722+0000","last_undegraded":"2024-09-19T11:23:42.943722+0000","last_fullsized":"2024-09-19T11:23:42.943722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:11:13.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368226+0000","last_change":"2024-09-19T11:23:05.238227+0000","last_active":"2024-09-19T11:26:12.368226+0000","last_peered":"2024-09-19T11:26:12.368226+0000","last_clean":"2024-09-19T11:26:12.368226+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:12.368226+0000","last_undegraded":"2024-09-19T11:26:12.368226+0000","last_fullsized":"2024-09-19T11:26:12.368226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:05.469042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944111+0000","last_change":"2024-09-19T11:22:55.817947+0000","last_active":"2024-09-19T11:23:42.944111+0000","last_peered":"2024-09-19T11:23:42.944111+0000","last_clean":"2024-09-19T11:23:42.944111+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:23:42.944111+0000","last_undegraded":"2024-09-19T11:23:42.944111+0000","last_fullsized":"2024-09-19T11:23:42.944111+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:52.495235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.363564+0000","last_change":"2024-09-19T11:23:03.247494+0000","last_active":"2024-09-19T11:26:12.363564+0000","last_peered":"2024-09-19T11:26:12.363564+0000","last_clean":"2024-09-19T11:26:12.363564+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:12.363564+0000","last_undegraded":"2024-09-19T11:26:12.363564+0000","last_fullsized":"2024-09-19T11:26:12.363564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:58.764280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279722+0000","last_change":"2024-09-19T11:23:07.243946+0000","last_active":"2024-09-19T11:23:42.279722+0000","last_peered":"2024-09-19T11:23:42.279722+0000","last_clean":"2024-09-19T11:23:42.279722+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:23:42.279722+0000","last_undegraded":"2024-09-19T11:23:42.279722+0000","last_fullsized":"2024-09-19T11:23:42.279722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:02.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'13","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.530475+0000","last_change":"2024-09-19T11:23:09.254499+0000","last_active":"2024-09-19T11:26:11.530475+0000","last_peered":"2024-09-19T11:26:11.530475+0000","last_clean":"2024-09-19T11:26:11.530475+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:26:11.530475+0000","last_undegraded":"2024-09-19T11:26:11.530475+0000","last_fullsized":"2024-09-19T11:26:11.530475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:18.957107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947826+0000","last_change":"2024-09-19T11:23:11.271073+0000","last_active":"2024-09-19T11:23:42.947826+0000","last_peered":"2024-09-19T11:23:42.947826+0000","last_clean":"2024-09-19T11:23:42.947826+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:23:42.947826+0000","last_undegraded":"2024-09-19T11:23:42.947826+0000","last_fullsized":"2024-09-19T11:23:42.947826+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:52.079937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.088603+0000","last_change":"2024-09-19T11:23:13.273910+0000","last_active":"2024-09-19T11:25:11.088603+0000","last_peered":"2024-09-19T11:25:11.088603+0000","last_clean":"2024-09-19T11:25:11.088603+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:25:11.088603+0000","last_undegraded":"2024-09-19T11:25:11.088603+0000","last_fullsized":"2024-09-19T11:25:11.088603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:02.581618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470157+0000","last_change":"2024-09-19T11:23:05.247999+0000","last_active":"2024-09-19T11:26:12.470157+0000","last_peered":"2024-09-19T11:26:12.470157+0000","last_clean":"2024-09-19T11:26:12.470157+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:12.470157+0000","last_undegraded":"2024-09-19T11:26:12.470157+0000","last_fullsized":"2024-09-19T11:26:12.470157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.575444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277923+0000","last_change":"2024-09-19T11:23:03.251364+0000","last_active":"2024-09-19T11:23:42.277923+0000","last_peered":"2024-09-19T11:23:42.277923+0000","last_clean":"2024-09-19T11:23:42.277923+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:23:42.277923+0000","last_undegraded":"2024-09-19T11:23:42.277923+0000","last_fullsized":"2024-09-19T11:23:42.277923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279906+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.279906+0000","last_peered":"2024-09-19T11:23:42.279906+0000","last_clean":"2024-09-19T11:23:42.279906+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:23:42.279906+0000","last_undegraded":"2024-09-19T11:23:42.279906+0000","last_fullsized":"2024-09-19T11:23:42.279906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:41.618944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:15.806655+0000","last_change":"2024-09-19T11:23:07.247913+0000","last_active":"2024-09-19T11:25:15.806655+0000","last_peered":"2024-09-19T11:25:15.806655+0000","last_clean":"2024-09-19T11:25:15.806655+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:25:15.806655+0000","last_undegraded":"2024-09-19T11:25:15.806655+0000","last_fullsized":"2024-09-19T11:25:15.806655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:03.208183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'62","reported_seq":122,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.805937+0000","last_change":"2024-09-19T11:23:09.257853+0000","last_active":"2024-09-19T11:25:10.805937+0000","last_peered":"2024-09-19T11:25:10.805937+0000","last_clean":"2024-09-19T11:25:10.805937+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:25:10.805937+0000","last_undegraded":"2024-09-19T11:25:10.805937+0000","last_fullsized":"2024-09-19T11:25:10.805937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:02.366161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277438+0000","last_change":"2024-09-19T11:23:11.268600+0000","last_active":"2024-09-19T11:23:42.277438+0000","last_peered":"2024-09-19T11:23:42.277438+0000","last_clean":"2024-09-19T11:23:42.277438+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:23:42.277438+0000","last_undegraded":"2024-09-19T11:23:42.277438+0000","last_fullsized":"2024-09-19T11:23:42.277438+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:50.540232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.422953+0000","last_change":"2024-09-19T11:23:13.267518+0000","last_active":"2024-09-19T11:25:11.422953+0000","last_peered":"2024-09-19T11:25:11.422953+0000","last_clean":"2024-09-19T11:25:11.422953+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:25:11.422953+0000","last_undegraded":"2024-09-19T11:25:11.422953+0000","last_fullsized":"2024-09-19T11:25:11.422953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:35.030653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338573+0000","last_change":"2024-09-19T11:23:05.247369+0000","last_active":"2024-09-19T11:26:12.338573+0000","last_peered":"2024-09-19T11:26:12.338573+0000","last_clean":"2024-09-19T11:26:12.338573+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:12.338573+0000","last_undegraded":"2024-09-19T11:26:12.338573+0000","last_fullsized":"2024-09-19T11:26:12.338573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:05.530837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.806030+0000","last_change":"2024-09-19T11:23:03.236746+0000","last_active":"2024-09-19T11:25:10.806030+0000","last_peered":"2024-09-19T11:25:10.806030+0000","last_clean":"2024-09-19T11:25:10.806030+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:25:10.806030+0000","last_undegraded":"2024-09-19T11:25:10.806030+0000","last_fullsized":"2024-09-19T11:25:10.806030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:23.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945428+0000","last_change":"2024-09-19T11:22:55.818302+0000","last_active":"2024-09-19T11:23:42.945428+0000","last_peered":"2024-09-19T11:23:42.945428+0000","last_clean":"2024-09-19T11:23:42.945428+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:23:42.945428+0000","last_undegraded":"2024-09-19T11:23:42.945428+0000","last_fullsized":"2024-09-19T11:23:42.945428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:51.911098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943947+0000","last_change":"2024-09-19T11:23:07.243184+0000","last_active":"2024-09-19T11:23:42.943947+0000","last_peered":"2024-09-19T11:23:42.943947+0000","last_clean":"2024-09-19T11:23:42.943947+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:23:42.943947+0000","last_undegraded":"2024-09-19T11:23:42.943947+0000","last_fullsized":"2024-09-19T11:23:42.943947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:32:59.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'15","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809866+0000","last_change":"2024-09-19T11:23:09.261039+0000","last_active":"2024-09-19T11:25:16.809866+0000","last_peered":"2024-09-19T11:25:16.809866+0000","last_clean":"2024-09-19T11:25:16.809866+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:25:16.809866+0000","last_undegraded":"2024-09-19T11:25:16.809866+0000","last_fullsized":"2024-09-19T11:25:16.809866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:42.183684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'282","reported_seq":356,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327172+0000","last_change":"2024-09-19T11:23:11.263125+0000","last_active":"2024-09-19T11:26:12.327172+0000","last_peered":"2024-09-19T11:26:12.327172+0000","last_clean":"2024-09-19T11:26:12.327172+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:26:12.327172+0000","last_undegraded":"2024-09-19T11:26:12.327172+0000","last_fullsized":"2024-09-19T11:26:12.327172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:10.328770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946621+0000","last_change":"2024-09-19T11:23:13.269759+0000","last_active":"2024-09-19T11:23:42.946621+0000","last_peered":"2024-09-19T11:23:42.946621+0000","last_clean":"2024-09-19T11:23:42.946621+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:23:42.946621+0000","last_undegraded":"2024-09-19T11:23:42.946621+0000","last_fullsized":"2024-09-19T11:23:42.946621+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:46.820239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'11","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.025445+0000","last_change":"2024-09-19T11:23:05.254850+0000","last_active":"2024-09-19T11:26:12.025445+0000","last_peered":"2024-09-19T11:26:12.025445+0000","last_clean":"2024-09-19T11:26:12.025445+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:12.025445+0000","last_undegraded":"2024-09-19T11:26:12.025445+0000","last_fullsized":"2024-09-19T11:26:12.025445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:44.121192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948269+0000","last_change":"2024-09-19T11:23:03.241499+0000","last_active":"2024-09-19T11:23:42.948269+0000","last_peered":"2024-09-19T11:23:42.948269+0000","last_clean":"2024-09-19T11:23:42.948269+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:23:42.948269+0000","last_undegraded":"2024-09-19T11:23:42.948269+0000","last_fullsized":"2024-09-19T11:23:42.948269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:40.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280554+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.280554+0000","last_peered":"2024-09-19T11:23:42.280554+0000","last_clean":"2024-09-19T11:23:42.280554+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:23:42.280554+0000","last_undegraded":"2024-09-19T11:23:42.280554+0000","last_fullsized":"2024-09-19T11:23:42.280554+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:08:18.682837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946584+0000","last_change":"2024-09-19T11:23:07.245138+0000","last_active":"2024-09-19T11:23:42.946584+0000","last_peered":"2024-09-19T11:23:42.946584+0000","last_clean":"2024-09-19T11:23:42.946584+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:23:42.946584+0000","last_undegraded":"2024-09-19T11:23:42.946584+0000","last_fullsized":"2024-09-19T11:23:42.946584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:31.680535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'11","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945354+0000","last_change":"2024-09-19T11:23:09.256698+0000","last_active":"2024-09-19T11:23:42.945354+0000","last_peered":"2024-09-19T11:23:42.945354+0000","last_clean":"2024-09-19T11:23:42.945354+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:23:42.945354+0000","last_undegraded":"2024-09-19T11:23:42.945354+0000","last_fullsized":"2024-09-19T11:23:42.945354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:21.852554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'282","reported_seq":430,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585906+0000","last_change":"2024-09-19T11:23:11.258279+0000","last_active":"2024-09-19T11:26:12.585906+0000","last_peered":"2024-09-19T11:26:12.585906+0000","last_clean":"2024-09-19T11:26:12.585906+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:26:12.585906+0000","last_undegraded":"2024-09-19T11:26:12.585906+0000","last_fullsized":"2024-09-19T11:26:12.585906+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:24.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281484+0000","last_change":"2024-09-19T11:23:13.274470+0000","last_active":"2024-09-19T11:23:42.281484+0000","last_peered":"2024-09-19T11:23:42.281484+0000","last_clean":"2024-09-19T11:23:42.281484+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:23:42.281484+0000","last_undegraded":"2024-09-19T11:23:42.281484+0000","last_fullsized":"2024-09-19T11:23:42.281484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:33.500152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550363+0000","last_change":"2024-09-19T11:23:05.250199+0000","last_active":"2024-09-19T11:24:27.550363+0000","last_peered":"2024-09-19T11:24:27.550363+0000","last_clean":"2024-09-19T11:24:27.550363+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:24:27.550363+0000","last_undegraded":"2024-09-19T11:24:27.550363+0000","last_fullsized":"2024-09-19T11:24:27.550363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:20.188966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282786+0000","last_change":"2024-09-19T11:23:03.243876+0000","last_active":"2024-09-19T11:23:42.282786+0000","last_peered":"2024-09-19T11:23:42.282786+0000","last_clean":"2024-09-19T11:23:42.282786+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:23:42.282786+0000","last_undegraded":"2024-09-19T11:23:42.282786+0000","last_fullsized":"2024-09-19T11:23:42.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:15.709376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946831+0000","last_change":"2024-09-19T11:22:55.818050+0000","last_active":"2024-09-19T11:23:42.946831+0000","last_peered":"2024-09-19T11:23:42.946831+0000","last_clean":"2024-09-19T11:23:42.946831+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:23:42.946831+0000","last_undegraded":"2024-09-19T11:23:42.946831+0000","last_fullsized":"2024-09-19T11:23:42.946831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.933095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945462+0000","last_change":"2024-09-19T11:23:07.248324+0000","last_active":"2024-09-19T11:23:42.945462+0000","last_peered":"2024-09-19T11:23:42.945462+0000","last_clean":"2024-09-19T11:23:42.945462+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:23:42.945462+0000","last_undegraded":"2024-09-19T11:23:42.945462+0000","last_fullsized":"2024-09-19T11:23:42.945462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.767188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'82","reported_seq":155,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.748098+0000","last_change":"2024-09-19T11:23:09.258292+0000","last_active":"2024-09-19T11:25:09.748098+0000","last_peered":"2024-09-19T11:25:09.748098+0000","last_clean":"2024-09-19T11:25:09.748098+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:25:09.748098+0000","last_undegraded":"2024-09-19T11:25:09.748098+0000","last_fullsized":"2024-09-19T11:25:09.748098+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:51.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280109+0000","last_change":"2024-09-19T11:23:11.263614+0000","last_active":"2024-09-19T11:23:42.280109+0000","last_peered":"2024-09-19T11:23:42.280109+0000","last_clean":"2024-09-19T11:23:42.280109+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:23:42.280109+0000","last_undegraded":"2024-09-19T11:23:42.280109+0000","last_fullsized":"2024-09-19T11:23:42.280109+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.842570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278855+0000","last_change":"2024-09-19T11:23:13.265949+0000","last_active":"2024-09-19T11:23:42.278855+0000","last_peered":"2024-09-19T11:23:42.278855+0000","last_clean":"2024-09-19T11:23:42.278855+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:23:42.278855+0000","last_undegraded":"2024-09-19T11:23:42.278855+0000","last_fullsized":"2024-09-19T11:23:42.278855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:37.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278953+0000","last_change":"2024-09-19T11:23:13.271066+0000","last_active":"2024-09-19T11:23:42.278953+0000","last_peered":"2024-09-19T11:23:42.278953+0000","last_clean":"2024-09-19T11:23:42.278953+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:23:42.278953+0000","last_undegraded":"2024-09-19T11:23:42.278953+0000","last_fullsized":"2024-09-19T11:23:42.278953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:34.307629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277914+0000","last_change":"2024-09-19T11:23:05.245620+0000","last_active":"2024-09-19T11:23:42.277914+0000","last_peered":"2024-09-19T11:23:42.277914+0000","last_clean":"2024-09-19T11:23:42.277914+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:23:42.277914+0000","last_undegraded":"2024-09-19T11:23:42.277914+0000","last_fullsized":"2024-09-19T11:23:42.277914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.990710+0000","last_change":"2024-09-19T11:23:03.254634+0000","last_active":"2024-09-19T11:23:44.990710+0000","last_peered":"2024-09-19T11:23:44.990710+0000","last_clean":"2024-09-19T11:23:44.990710+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:23:44.990710+0000","last_undegraded":"2024-09-19T11:23:44.990710+0000","last_fullsized":"2024-09-19T11:23:44.990710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.416384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282381+0000","last_change":"2024-09-19T11:23:07.240696+0000","last_active":"2024-09-19T11:23:42.282381+0000","last_peered":"2024-09-19T11:23:42.282381+0000","last_clean":"2024-09-19T11:23:42.282381+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:23:42.282381+0000","last_undegraded":"2024-09-19T11:23:42.282381+0000","last_fullsized":"2024-09-19T11:23:42.282381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:02.659138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'32","reported_seq":76,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.754473+0000","last_change":"2024-09-19T11:23:09.259148+0000","last_active":"2024-09-19T11:25:09.754473+0000","last_peered":"2024-09-19T11:25:09.754473+0000","last_clean":"2024-09-19T11:25:09.754473+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:25:09.754473+0000","last_undegraded":"2024-09-19T11:25:09.754473+0000","last_fullsized":"2024-09-19T11:25:09.754473+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:29.000573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945264+0000","last_change":"2024-09-19T11:23:11.271128+0000","last_active":"2024-09-19T11:23:42.945264+0000","last_peered":"2024-09-19T11:23:42.945264+0000","last_clean":"2024-09-19T11:23:42.945264+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:23:42.945264+0000","last_undegraded":"2024-09-19T11:23:42.945264+0000","last_fullsized":"2024-09-19T11:23:42.945264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:35.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.427118+0000","last_change":"2024-09-19T11:23:13.273576+0000","last_active":"2024-09-19T11:25:09.427118+0000","last_peered":"2024-09-19T11:25:09.427118+0000","last_clean":"2024-09-19T11:25:09.427118+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:25:09.427118+0000","last_undegraded":"2024-09-19T11:25:09.427118+0000","last_fullsized":"2024-09-19T11:25:09.427118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:43.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.782138+0000","last_change":"2024-09-19T11:23:42.285785+0000","last_active":"2024-09-19T11:23:44.782138+0000","last_peered":"2024-09-19T11:23:44.782138+0000","last_clean":"2024-09-19T11:23:44.782138+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:23:44.782138+0000","last_undegraded":"2024-09-19T11:23:44.782138+0000","last_fullsized":"2024-09-19T11:23:44.782138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:15.907367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277946+0000","last_change":"2024-09-19T11:23:03.255902+0000","last_active":"2024-09-19T11:23:42.277946+0000","last_peered":"2024-09-19T11:23:42.277946+0000","last_clean":"2024-09-19T11:23:42.277946+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:23:42.277946+0000","last_undegraded":"2024-09-19T11:23:42.277946+0000","last_fullsized":"2024-09-19T11:23:42.277946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:41.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945177+0000","last_change":"2024-09-19T11:23:07.248908+0000","last_active":"2024-09-19T11:23:42.945177+0000","last_peered":"2024-09-19T11:23:42.945177+0000","last_clean":"2024-09-19T11:23:42.945177+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:23:42.945177+0000","last_undegraded":"2024-09-19T11:23:42.945177+0000","last_fullsized":"2024-09-19T11:23:42.945177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:57.835222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'5","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.535761+0000","last_change":"2024-09-19T11:23:09.244959+0000","last_active":"2024-09-19T11:26:11.535761+0000","last_peered":"2024-09-19T11:26:11.535761+0000","last_clean":"2024-09-19T11:26:11.535761+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:26:11.535761+0000","last_undegraded":"2024-09-19T11:26:11.535761+0000","last_fullsized":"2024-09-19T11:26:11.535761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:13.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277093+0000","last_change":"2024-09-19T11:23:11.268038+0000","last_active":"2024-09-19T11:23:42.277093+0000","last_peered":"2024-09-19T11:23:42.277093+0000","last_clean":"2024-09-19T11:23:42.277093+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:23:42.277093+0000","last_undegraded":"2024-09-19T11:23:42.277093+0000","last_fullsized":"2024-09-19T11:23:42.277093+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:40.232592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279462+0000","last_change":"2024-09-19T11:23:13.267238+0000","last_active":"2024-09-19T11:23:42.279462+0000","last_peered":"2024-09-19T11:23:42.279462+0000","last_clean":"2024-09-19T11:23:42.279462+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:23:42.279462+0000","last_undegraded":"2024-09-19T11:23:42.279462+0000","last_fullsized":"2024-09-19T11:23:42.279462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:36.252981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123419+0000","last_change":"2024-09-19T11:23:05.240162+0000","last_active":"2024-09-19T11:26:12.123419+0000","last_peered":"2024-09-19T11:26:12.123419+0000","last_clean":"2024-09-19T11:26:12.123419+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:12.123419+0000","last_undegraded":"2024-09-19T11:26:12.123419+0000","last_fullsized":"2024-09-19T11:26:12.123419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:45.738004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.261555+0000","last_change":"2024-09-19T11:23:03.257719+0000","last_active":"2024-09-19T11:26:12.261555+0000","last_peered":"2024-09-19T11:26:12.261555+0000","last_clean":"2024-09-19T11:26:12.261555+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:12.261555+0000","last_undegraded":"2024-09-19T11:26:12.261555+0000","last_fullsized":"2024-09-19T11:26:12.261555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:46.873725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277308+0000","last_change":"2024-09-19T11:23:07.243467+0000","last_active":"2024-09-19T11:23:42.277308+0000","last_peered":"2024-09-19T11:23:42.277308+0000","last_clean":"2024-09-19T11:23:42.277308+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:23:42.277308+0000","last_undegraded":"2024-09-19T11:23:42.277308+0000","last_fullsized":"2024-09-19T11:23:42.277308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:48.792369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'40","reported_seq":100,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.662362+0000","last_change":"2024-09-19T11:23:09.248715+0000","last_active":"2024-09-19T11:26:11.662362+0000","last_peered":"2024-09-19T11:26:11.662362+0000","last_clean":"2024-09-19T11:26:11.662362+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:26:11.662362+0000","last_undegraded":"2024-09-19T11:26:11.662362+0000","last_fullsized":"2024-09-19T11:26:11.662362+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:51.723342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280357+0000","last_change":"2024-09-19T11:23:11.262690+0000","last_active":"2024-09-19T11:23:42.280357+0000","last_peered":"2024-09-19T11:23:42.280357+0000","last_clean":"2024-09-19T11:23:42.280357+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:23:42.280357+0000","last_undegraded":"2024-09-19T11:23:42.280357+0000","last_fullsized":"2024-09-19T11:23:42.280357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:55.778119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.511055+0000","last_change":"2024-09-19T11:23:05.253736+0000","last_active":"2024-09-19T11:25:11.511055+0000","last_peered":"2024-09-19T11:25:11.511055+0000","last_clean":"2024-09-19T11:25:11.511055+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:25:11.511055+0000","last_undegraded":"2024-09-19T11:25:11.511055+0000","last_fullsized":"2024-09-19T11:25:11.511055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:09.050323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283174+0000","last_change":"2024-09-19T11:23:03.248284+0000","last_active":"2024-09-19T11:23:42.283174+0000","last_peered":"2024-09-19T11:23:42.283174+0000","last_clean":"2024-09-19T11:23:42.283174+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:23:42.283174+0000","last_undegraded":"2024-09-19T11:23:42.283174+0000","last_fullsized":"2024-09-19T11:23:42.283174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:40.686948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944123+0000","last_change":"2024-09-19T11:23:07.245862+0000","last_active":"2024-09-19T11:23:42.944123+0000","last_peered":"2024-09-19T11:23:42.944123+0000","last_clean":"2024-09-19T11:23:42.944123+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:23:42.944123+0000","last_undegraded":"2024-09-19T11:23:42.944123+0000","last_fullsized":"2024-09-19T11:23:42.944123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:09.692396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.739853+0000","last_change":"2024-09-19T11:23:09.247041+0000","last_active":"2024-09-19T11:25:09.739853+0000","last_peered":"2024-09-19T11:25:09.739853+0000","last_clean":"2024-09-19T11:25:09.739853+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:25:09.739853+0000","last_undegraded":"2024-09-19T11:25:09.739853+0000","last_fullsized":"2024-09-19T11:25:09.739853+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:21.727511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277107+0000","last_change":"2024-09-19T11:23:11.253755+0000","last_active":"2024-09-19T11:23:42.277107+0000","last_peered":"2024-09-19T11:23:42.277107+0000","last_clean":"2024-09-19T11:23:42.277107+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:23:42.277107+0000","last_undegraded":"2024-09-19T11:23:42.277107+0000","last_fullsized":"2024-09-19T11:23:42.277107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:51.209331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510416+0000","last_change":"2024-09-19T11:23:05.236615+0000","last_active":"2024-09-19T11:25:11.510416+0000","last_peered":"2024-09-19T11:25:11.510416+0000","last_clean":"2024-09-19T11:25:11.510416+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:25:11.510416+0000","last_undegraded":"2024-09-19T11:25:11.510416+0000","last_fullsized":"2024-09-19T11:25:11.510416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:36.749643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'3","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:31.812710+0000","last_change":"2024-09-19T11:23:03.255452+0000","last_active":"2024-09-19T11:25:31.812710+0000","last_peered":"2024-09-19T11:25:31.812710+0000","last_clean":"2024-09-19T11:25:31.812710+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:25:31.812710+0000","last_undegraded":"2024-09-19T11:25:31.812710+0000","last_fullsized":"2024-09-19T11:25:31.812710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:34.861274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796174+0000","last_change":"2024-09-19T11:23:42.945035+0000","last_active":"2024-09-19T11:23:44.796174+0000","last_peered":"2024-09-19T11:23:44.796174+0000","last_clean":"2024-09-19T11:23:44.796174+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:23:44.796174+0000","last_undegraded":"2024-09-19T11:23:44.796174+0000","last_fullsized":"2024-09-19T11:23:44.796174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'55","reported_seq":112,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:40.810485+0000","last_change":"2024-09-19T11:23:09.246587+0000","last_active":"2024-09-19T11:25:40.810485+0000","last_peered":"2024-09-19T11:25:40.810485+0000","last_clean":"2024-09-19T11:25:40.810485+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:25:40.810485+0000","last_undegraded":"2024-09-19T11:25:40.810485+0000","last_fullsized":"2024-09-19T11:25:40.810485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:53.040162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":42,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278167+0000","last_change":"2024-09-19T11:23:11.260449+0000","last_active":"2024-09-19T11:23:42.278167+0000","last_peered":"2024-09-19T11:23:42.278167+0000","last_clean":"2024-09-19T11:23:42.278167+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:23:42.278167+0000","last_undegraded":"2024-09-19T11:23:42.278167+0000","last_fullsized":"2024-09-19T11:23:42.278167+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:30.983027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.419187+0000","last_change":"2024-09-19T11:23:13.268105+0000","last_active":"2024-09-19T11:25:11.419187+0000","last_peered":"2024-09-19T11:25:11.419187+0000","last_clean":"2024-09-19T11:25:11.419187+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:25:11.419187+0000","last_undegraded":"2024-09-19T11:25:11.419187+0000","last_fullsized":"2024-09-19T11:25:11.419187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:21.113571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368061+0000","last_change":"2024-09-19T11:23:05.246352+0000","last_active":"2024-09-19T11:26:12.368061+0000","last_peered":"2024-09-19T11:26:12.368061+0000","last_clean":"2024-09-19T11:26:12.368061+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:12.368061+0000","last_undegraded":"2024-09-19T11:26:12.368061+0000","last_fullsized":"2024-09-19T11:26:12.368061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.248495+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.959521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279359+0000","last_change":"2024-09-19T11:23:07.243165+0000","last_active":"2024-09-19T11:23:42.279359+0000","last_peered":"2024-09-19T11:23:42.279359+0000","last_clean":"2024-09-19T11:23:42.279359+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:23:42.279359+0000","last_undegraded":"2024-09-19T11:23:42.279359+0000","last_fullsized":"2024-09-19T11:23:42.279359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:21.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.740822+0000","last_change":"2024-09-19T11:23:09.256642+0000","last_active":"2024-09-19T11:25:09.740822+0000","last_peered":"2024-09-19T11:25:09.740822+0000","last_clean":"2024-09-19T11:25:09.740822+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:25:09.740822+0000","last_undegraded":"2024-09-19T11:25:09.740822+0000","last_fullsized":"2024-09-19T11:25:09.740822+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:42.797325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282300+0000","last_change":"2024-09-19T11:23:11.266817+0000","last_active":"2024-09-19T11:23:42.282300+0000","last_peered":"2024-09-19T11:23:42.282300+0000","last_clean":"2024-09-19T11:23:42.282300+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:23:42.282300+0000","last_undegraded":"2024-09-19T11:23:42.282300+0000","last_fullsized":"2024-09-19T11:23:42.282300+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:41:28.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947653+0000","last_change":"2024-09-19T11:23:05.254589+0000","last_active":"2024-09-19T11:23:42.947653+0000","last_peered":"2024-09-19T11:23:42.947653+0000","last_clean":"2024-09-19T11:23:42.947653+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:23:42.947653+0000","last_undegraded":"2024-09-19T11:23:42.947653+0000","last_fullsized":"2024-09-19T11:23:42.947653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:11.078441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280548+0000","last_change":"2024-09-19T11:23:03.244404+0000","last_active":"2024-09-19T11:23:42.280548+0000","last_peered":"2024-09-19T11:23:42.280548+0000","last_clean":"2024-09-19T11:23:42.280548+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:23:42.280548+0000","last_undegraded":"2024-09-19T11:23:42.280548+0000","last_fullsized":"2024-09-19T11:23:42.280548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:07.948011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278448+0000","last_change":"2024-09-19T11:23:07.242009+0000","last_active":"2024-09-19T11:23:42.278448+0000","last_peered":"2024-09-19T11:23:42.278448+0000","last_clean":"2024-09-19T11:23:42.278448+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:23:42.278448+0000","last_undegraded":"2024-09-19T11:23:42.278448+0000","last_fullsized":"2024-09-19T11:23:42.278448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:37.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'34","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.758462+0000","last_change":"2024-09-19T11:23:09.258190+0000","last_active":"2024-09-19T11:25:09.758462+0000","last_peered":"2024-09-19T11:25:09.758462+0000","last_clean":"2024-09-19T11:25:09.758462+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:25:09.758462+0000","last_undegraded":"2024-09-19T11:25:09.758462+0000","last_fullsized":"2024-09-19T11:25:09.758462+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:31.989630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":46,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280329+0000","last_change":"2024-09-19T11:23:11.261698+0000","last_active":"2024-09-19T11:23:42.280329+0000","last_peered":"2024-09-19T11:23:42.280329+0000","last_clean":"2024-09-19T11:23:42.280329+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:23:42.280329+0000","last_undegraded":"2024-09-19T11:23:42.280329+0000","last_fullsized":"2024-09-19T11:23:42.280329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:48.537146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032904+0000","last_change":"2024-09-19T11:23:05.238234+0000","last_active":"2024-09-19T11:25:08.032904+0000","last_peered":"2024-09-19T11:25:08.032904+0000","last_clean":"2024-09-19T11:25:08.032904+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:25:08.032904+0000","last_undegraded":"2024-09-19T11:25:08.032904+0000","last_fullsized":"2024-09-19T11:25:08.032904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:03:26.967357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944417+0000","last_change":"2024-09-19T11:23:03.253512+0000","last_active":"2024-09-19T11:23:42.944417+0000","last_peered":"2024-09-19T11:23:42.944417+0000","last_clean":"2024-09-19T11:23:42.944417+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:23:42.944417+0000","last_undegraded":"2024-09-19T11:23:42.944417+0000","last_fullsized":"2024-09-19T11:23:42.944417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:17.009054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.286788+0000","last_change":"2024-09-19T11:23:42.286200+0000","last_active":"2024-09-19T11:23:42.286788+0000","last_peered":"2024-09-19T11:23:42.286788+0000","last_clean":"2024-09-19T11:23:42.286788+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:23:42.286788+0000","last_undegraded":"2024-09-19T11:23:42.286788+0000","last_fullsized":"2024-09-19T11:23:42.286788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.397616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946537+0000","last_change":"2024-09-19T11:23:09.257194+0000","last_active":"2024-09-19T11:23:42.946537+0000","last_peered":"2024-09-19T11:23:42.946537+0000","last_clean":"2024-09-19T11:23:42.946537+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:23:42.946537+0000","last_undegraded":"2024-09-19T11:23:42.946537+0000","last_fullsized":"2024-09-19T11:23:42.946537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:10.690183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'282","reported_seq":418,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.372744+0000","last_change":"2024-09-19T11:23:11.268055+0000","last_active":"2024-09-19T11:26:12.372744+0000","last_peered":"2024-09-19T11:26:12.372744+0000","last_clean":"2024-09-19T11:26:12.372744+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:26:12.372744+0000","last_undegraded":"2024-09-19T11:26:12.372744+0000","last_fullsized":"2024-09-19T11:26:12.372744+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:33.997140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943313+0000","last_change":"2024-09-19T11:23:13.272264+0000","last_active":"2024-09-19T11:23:42.943313+0000","last_peered":"2024-09-19T11:23:42.943313+0000","last_clean":"2024-09-19T11:23:42.943313+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:23:42.943313+0000","last_undegraded":"2024-09-19T11:23:42.943313+0000","last_fullsized":"2024-09-19T11:23:42.943313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:34.821219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123209+0000","last_change":"2024-09-19T11:23:05.235764+0000","last_active":"2024-09-19T11:26:12.123209+0000","last_peered":"2024-09-19T11:26:12.123209+0000","last_clean":"2024-09-19T11:26:12.123209+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:12.123209+0000","last_undegraded":"2024-09-19T11:26:12.123209+0000","last_fullsized":"2024-09-19T11:26:12.123209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:41.326878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282378+0000","last_change":"2024-09-19T11:23:03.245357+0000","last_active":"2024-09-19T11:23:42.282378+0000","last_peered":"2024-09-19T11:23:42.282378+0000","last_clean":"2024-09-19T11:23:42.282378+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:23:42.282378+0000","last_undegraded":"2024-09-19T11:23:42.282378+0000","last_fullsized":"2024-09-19T11:23:42.282378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:23.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281352+0000","last_change":"2024-09-19T11:23:07.247890+0000","last_active":"2024-09-19T11:23:42.281352+0000","last_peered":"2024-09-19T11:23:42.281352+0000","last_clean":"2024-09-19T11:23:42.281352+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:23:42.281352+0000","last_undegraded":"2024-09-19T11:23:42.281352+0000","last_fullsized":"2024-09-19T11:23:42.281352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.412033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.731864+0000","last_change":"2024-09-19T11:23:09.246755+0000","last_active":"2024-09-19T11:25:09.731864+0000","last_peered":"2024-09-19T11:25:09.731864+0000","last_clean":"2024-09-19T11:25:09.731864+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:25:09.731864+0000","last_undegraded":"2024-09-19T11:25:09.731864+0000","last_fullsized":"2024-09-19T11:25:09.731864+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.229678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278129+0000","last_change":"2024-09-19T11:23:11.266713+0000","last_active":"2024-09-19T11:23:42.278129+0000","last_peered":"2024-09-19T11:23:42.278129+0000","last_clean":"2024-09-19T11:23:42.278129+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:23:42.278129+0000","last_undegraded":"2024-09-19T11:23:42.278129+0000","last_fullsized":"2024-09-19T11:23:42.278129+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:38.162577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283516+0000","last_change":"2024-09-19T11:23:13.268771+0000","last_active":"2024-09-19T11:23:42.283516+0000","last_peered":"2024-09-19T11:23:42.283516+0000","last_clean":"2024-09-19T11:23:42.283516+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:23:42.283516+0000","last_undegraded":"2024-09-19T11:23:42.283516+0000","last_fullsized":"2024-09-19T11:23:42.283516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:19.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266078+0000","last_change":"2024-09-19T11:23:05.236610+0000","last_active":"2024-09-19T11:26:12.266078+0000","last_peered":"2024-09-19T11:26:12.266078+0000","last_clean":"2024-09-19T11:26:12.266078+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:12.266078+0000","last_undegraded":"2024-09-19T11:26:12.266078+0000","last_fullsized":"2024-09-19T11:26:12.266078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:44.218259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946253+0000","last_change":"2024-09-19T11:23:03.258006+0000","last_active":"2024-09-19T11:23:42.946253+0000","last_peered":"2024-09-19T11:23:42.946253+0000","last_clean":"2024-09-19T11:23:42.946253+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:23:42.946253+0000","last_undegraded":"2024-09-19T11:23:42.946253+0000","last_fullsized":"2024-09-19T11:23:42.946253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:55.934419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278151+0000","last_change":"2024-09-19T11:23:07.247875+0000","last_active":"2024-09-19T11:23:42.278151+0000","last_peered":"2024-09-19T11:23:42.278151+0000","last_clean":"2024-09-19T11:23:42.278151+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:23:42.278151+0000","last_undegraded":"2024-09-19T11:23:42.278151+0000","last_fullsized":"2024-09-19T11:23:42.278151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:20.420374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.732852+0000","last_change":"2024-09-19T11:23:09.257850+0000","last_active":"2024-09-19T11:25:09.732852+0000","last_peered":"2024-09-19T11:25:09.732852+0000","last_clean":"2024-09-19T11:25:09.732852+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:25:09.732852+0000","last_undegraded":"2024-09-19T11:25:09.732852+0000","last_fullsized":"2024-09-19T11:25:09.732852+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:48.134834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943735+0000","last_change":"2024-09-19T11:23:11.271462+0000","last_active":"2024-09-19T11:23:42.943735+0000","last_peered":"2024-09-19T11:23:42.943735+0000","last_clean":"2024-09-19T11:23:42.943735+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:23:42.943735+0000","last_undegraded":"2024-09-19T11:23:42.943735+0000","last_fullsized":"2024-09-19T11:23:42.943735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.194246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368608+0000","last_change":"2024-09-19T11:23:42.282153+0000","last_active":"2024-09-19T11:26:12.368608+0000","last_peered":"2024-09-19T11:26:12.368608+0000","last_clean":"2024-09-19T11:26:12.368608+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:12.368608+0000","last_undegraded":"2024-09-19T11:26:12.368608+0000","last_fullsized":"2024-09-19T11:26:12.368608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.828536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.999397+0000","last_change":"2024-09-19T11:23:03.237858+0000","last_active":"2024-09-19T11:23:44.999397+0000","last_peered":"2024-09-19T11:23:44.999397+0000","last_clean":"2024-09-19T11:23:44.999397+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:23:44.999397+0000","last_undegraded":"2024-09-19T11:23:44.999397+0000","last_fullsized":"2024-09-19T11:23:44.999397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:38.094142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280358+0000","last_change":"2024-09-19T11:23:07.245080+0000","last_active":"2024-09-19T11:23:42.280358+0000","last_peered":"2024-09-19T11:23:42.280358+0000","last_clean":"2024-09-19T11:23:42.280358+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:23:42.280358+0000","last_undegraded":"2024-09-19T11:23:42.280358+0000","last_fullsized":"2024-09-19T11:23:42.280358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:41.815514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'26","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.756320+0000","last_change":"2024-09-19T11:23:09.258685+0000","last_active":"2024-09-19T11:25:09.756320+0000","last_peered":"2024-09-19T11:25:09.756320+0000","last_clean":"2024-09-19T11:25:09.756320+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:25:09.756320+0000","last_undegraded":"2024-09-19T11:25:09.756320+0000","last_fullsized":"2024-09-19T11:25:09.756320+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:33.508526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944317+0000","last_change":"2024-09-19T11:23:11.267904+0000","last_active":"2024-09-19T11:23:42.944317+0000","last_peered":"2024-09-19T11:23:42.944317+0000","last_clean":"2024-09-19T11:23:42.944317+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:23:42.944317+0000","last_undegraded":"2024-09-19T11:23:42.944317+0000","last_fullsized":"2024-09-19T11:23:42.944317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:16.874318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032486+0000","last_change":"2024-09-19T11:23:05.235573+0000","last_active":"2024-09-19T11:25:08.032486+0000","last_peered":"2024-09-19T11:25:08.032486+0000","last_clean":"2024-09-19T11:25:08.032486+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:25:08.032486+0000","last_undegraded":"2024-09-19T11:25:08.032486+0000","last_fullsized":"2024-09-19T11:25:08.032486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:07.062938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":46,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.996626+0000","last_change":"2024-09-19T11:23:03.249699+0000","last_active":"2024-09-19T11:23:44.996626+0000","last_peered":"2024-09-19T11:23:44.996626+0000","last_clean":"2024-09-19T11:23:44.996626+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:23:44.996626+0000","last_undegraded":"2024-09-19T11:23:44.996626+0000","last_fullsized":"2024-09-19T11:23:44.996626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:36.434981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'2","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.365932+0000","last_change":"2024-09-19T11:23:07.247476+0000","last_active":"2024-09-19T11:25:08.365932+0000","last_peered":"2024-09-19T11:25:08.365932+0000","last_clean":"2024-09-19T11:25:08.365932+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:25:08.365932+0000","last_undegraded":"2024-09-19T11:25:08.365932+0000","last_fullsized":"2024-09-19T11:25:08.365932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:03.848364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'34","reported_seq":79,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.561308+0000","last_change":"2024-09-19T11:23:09.255745+0000","last_active":"2024-09-19T11:26:11.561308+0000","last_peered":"2024-09-19T11:26:11.561308+0000","last_clean":"2024-09-19T11:26:11.561308+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:26:11.561308+0000","last_undegraded":"2024-09-19T11:26:11.561308+0000","last_fullsized":"2024-09-19T11:26:11.561308+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:35.538204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279071+0000","last_change":"2024-09-19T11:23:11.266117+0000","last_active":"2024-09-19T11:23:42.279071+0000","last_peered":"2024-09-19T11:23:42.279071+0000","last_clean":"2024-09-19T11:23:42.279071+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:23:42.279071+0000","last_undegraded":"2024-09-19T11:23:42.279071+0000","last_fullsized":"2024-09-19T11:23:42.279071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:22.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550414+0000","last_change":"2024-09-19T11:23:05.252243+0000","last_active":"2024-09-19T11:24:27.550414+0000","last_peered":"2024-09-19T11:24:27.550414+0000","last_clean":"2024-09-19T11:24:27.550414+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:24:27.550414+0000","last_undegraded":"2024-09-19T11:24:27.550414+0000","last_fullsized":"2024-09-19T11:24:27.550414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:18.179391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276311+0000","last_change":"2024-09-19T11:23:03.257178+0000","last_active":"2024-09-19T11:23:42.276311+0000","last_peered":"2024-09-19T11:23:42.276311+0000","last_clean":"2024-09-19T11:23:42.276311+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:23:42.276311+0000","last_undegraded":"2024-09-19T11:23:42.276311+0000","last_fullsized":"2024-09-19T11:23:42.276311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:13.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.024977+0000","last_change":"2024-09-19T11:23:05.238242+0000","last_active":"2024-09-19T11:26:12.024977+0000","last_peered":"2024-09-19T11:26:12.024977+0000","last_clean":"2024-09-19T11:26:12.024977+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:12.024977+0000","last_undegraded":"2024-09-19T11:26:12.024977+0000","last_fullsized":"2024-09-19T11:26:12.024977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278230+0000","last_change":"2024-09-19T11:23:03.238163+0000","last_active":"2024-09-19T11:23:42.278230+0000","last_peered":"2024-09-19T11:23:42.278230+0000","last_clean":"2024-09-19T11:23:42.278230+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:23:42.278230+0000","last_undegraded":"2024-09-19T11:23:42.278230+0000","last_fullsized":"2024-09-19T11:23:42.278230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:49:17.258919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342000+0000","last_change":"2024-09-19T11:23:05.237372+0000","last_active":"2024-09-19T11:24:27.342000+0000","last_peered":"2024-09-19T11:24:27.342000+0000","last_clean":"2024-09-19T11:24:27.342000+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:24:27.342000+0000","last_undegraded":"2024-09-19T11:24:27.342000+0000","last_fullsized":"2024-09-19T11:24:27.342000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:27.975988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940912+0000","last_change":"2024-09-19T11:23:03.256664+0000","last_active":"2024-09-19T11:23:43.940912+0000","last_peered":"2024-09-19T11:23:43.940912+0000","last_clean":"2024-09-19T11:23:43.940912+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:23:43.940912+0000","last_undegraded":"2024-09-19T11:23:43.940912+0000","last_fullsized":"2024-09-19T11:23:43.940912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:24.307449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.011182+0000","last_change":"2024-09-19T11:23:05.237761+0000","last_active":"2024-09-19T11:23:45.011182+0000","last_peered":"2024-09-19T11:23:45.011182+0000","last_clean":"2024-09-19T11:23:45.011182+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:23:45.011182+0000","last_undegraded":"2024-09-19T11:23:45.011182+0000","last_fullsized":"2024-09-19T11:23:45.011182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:29:24.776293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281499+0000","last_change":"2024-09-19T11:23:03.243433+0000","last_active":"2024-09-19T11:23:42.281499+0000","last_peered":"2024-09-19T11:23:42.281499+0000","last_clean":"2024-09-19T11:23:42.281499+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:23:42.281499+0000","last_undegraded":"2024-09-19T11:23:42.281499+0000","last_fullsized":"2024-09-19T11:23:42.281499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:45.982872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338525+0000","last_change":"2024-09-19T11:23:05.240645+0000","last_active":"2024-09-19T11:26:12.338525+0000","last_peered":"2024-09-19T11:26:12.338525+0000","last_clean":"2024-09-19T11:26:12.338525+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:12.338525+0000","last_undegraded":"2024-09-19T11:26:12.338525+0000","last_fullsized":"2024-09-19T11:26:12.338525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.997259+0000","last_change":"2024-09-19T11:23:03.234311+0000","last_active":"2024-09-19T11:26:11.997259+0000","last_peered":"2024-09-19T11:26:11.997259+0000","last_clean":"2024-09-19T11:26:11.997259+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:11.997259+0000","last_undegraded":"2024-09-19T11:26:11.997259+0000","last_fullsized":"2024-09-19T11:26:11.997259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:10.195241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342109+0000","last_change":"2024-09-19T11:23:05.251796+0000","last_active":"2024-09-19T11:24:27.342109+0000","last_peered":"2024-09-19T11:24:27.342109+0000","last_clean":"2024-09-19T11:24:27.342109+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:24:27.342109+0000","last_undegraded":"2024-09-19T11:24:27.342109+0000","last_fullsized":"2024-09-19T11:24:27.342109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:53.953637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281786+0000","last_change":"2024-09-19T11:23:03.248461+0000","last_active":"2024-09-19T11:23:42.281786+0000","last_peered":"2024-09-19T11:23:42.281786+0000","last_clean":"2024-09-19T11:23:42.281786+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:23:42.281786+0000","last_undegraded":"2024-09-19T11:23:42.281786+0000","last_fullsized":"2024-09-19T11:23:42.281786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:40.877834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279051+0000","last_change":"2024-09-19T11:23:05.248747+0000","last_active":"2024-09-19T11:23:42.279051+0000","last_peered":"2024-09-19T11:23:42.279051+0000","last_clean":"2024-09-19T11:23:42.279051+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:23:42.279051+0000","last_undegraded":"2024-09-19T11:23:42.279051+0000","last_fullsized":"2024-09-19T11:23:42.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.258514+0000","last_change":"2024-09-19T11:23:03.257927+0000","last_active":"2024-09-19T11:26:12.258514+0000","last_peered":"2024-09-19T11:26:12.258514+0000","last_clean":"2024-09-19T11:26:12.258514+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:12.258514+0000","last_undegraded":"2024-09-19T11:26:12.258514+0000","last_fullsized":"2024-09-19T11:26:12.258514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374459+0000","last_change":"2024-09-19T11:23:05.248818+0000","last_active":"2024-09-19T11:26:12.374459+0000","last_peered":"2024-09-19T11:26:12.374459+0000","last_clean":"2024-09-19T11:26:12.374459+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:12.374459+0000","last_undegraded":"2024-09-19T11:26:12.374459+0000","last_fullsized":"2024-09-19T11:26:12.374459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:41.343464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280250+0000","last_change":"2024-09-19T11:23:03.238591+0000","last_active":"2024-09-19T11:23:42.280250+0000","last_peered":"2024-09-19T11:23:42.280250+0000","last_clean":"2024-09-19T11:23:42.280250+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:23:42.280250+0000","last_undegraded":"2024-09-19T11:23:42.280250+0000","last_fullsized":"2024-09-19T11:23:42.280250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:42.869904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342282+0000","last_change":"2024-09-19T11:23:05.248696+0000","last_active":"2024-09-19T11:24:27.342282+0000","last_peered":"2024-09-19T11:24:27.342282+0000","last_clean":"2024-09-19T11:24:27.342282+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:24:27.342282+0000","last_undegraded":"2024-09-19T11:24:27.342282+0000","last_fullsized":"2024-09-19T11:24:27.342282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:37:55.133352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944801+0000","last_change":"2024-09-19T11:23:03.255783+0000","last_active":"2024-09-19T11:23:42.944801+0000","last_peered":"2024-09-19T11:23:42.944801+0000","last_clean":"2024-09-19T11:23:42.944801+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:23:42.944801+0000","last_undegraded":"2024-09-19T11:23:42.944801+0000","last_fullsized":"2024-09-19T11:23:42.944801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:24.578577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368100+0000","last_change":"2024-09-19T11:23:05.243797+0000","last_active":"2024-09-19T11:26:12.368100+0000","last_peered":"2024-09-19T11:26:12.368100+0000","last_clean":"2024-09-19T11:26:12.368100+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:26:12.368100+0000","last_undegraded":"2024-09-19T11:26:12.368100+0000","last_fullsized":"2024-09-19T11:26:12.368100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:33.524342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278358+0000","last_change":"2024-09-19T11:23:03.243321+0000","last_active":"2024-09-19T11:23:42.278358+0000","last_peered":"2024-09-19T11:23:42.278358+0000","last_clean":"2024-09-19T11:23:42.278358+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:23:42.278358+0000","last_undegraded":"2024-09-19T11:23:42.278358+0000","last_fullsized":"2024-09-19T11:23:42.278358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342015+0000","last_change":"2024-09-19T11:23:05.241366+0000","last_active":"2024-09-19T11:24:27.342015+0000","last_peered":"2024-09-19T11:24:27.342015+0000","last_clean":"2024-09-19T11:24:27.342015+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:24:27.342015+0000","last_undegraded":"2024-09-19T11:24:27.342015+0000","last_fullsized":"2024-09-19T11:24:27.342015+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.002195+0000","last_change":"2024-09-19T11:23:03.249920+0000","last_active":"2024-09-19T11:23:45.002195+0000","last_peered":"2024-09-19T11:23:45.002195+0000","last_clean":"2024-09-19T11:23:45.002195+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:23:45.002195+0000","last_undegraded":"2024-09-19T11:23:45.002195+0000","last_fullsized":"2024-09-19T11:23:45.002195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:45.629786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.482864+0000","last_change":"2024-09-19T11:23:05.246852+0000","last_active":"2024-09-19T11:23:45.482864+0000","last_peered":"2024-09-19T11:23:45.482864+0000","last_clean":"2024-09-19T11:23:45.482864+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:23:45.482864+0000","last_undegraded":"2024-09-19T11:23:45.482864+0000","last_fullsized":"2024-09-19T11:23:45.482864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.057981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278908+0000","last_change":"2024-09-19T11:23:03.243727+0000","last_active":"2024-09-19T11:23:42.278908+0000","last_peered":"2024-09-19T11:23:42.278908+0000","last_clean":"2024-09-19T11:23:42.278908+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:23:42.278908+0000","last_undegraded":"2024-09-19T11:23:42.278908+0000","last_fullsized":"2024-09-19T11:23:42.278908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:52:42.605160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433129+0000","last_change":"2024-09-19T11:23:05.245754+0000","last_active":"2024-09-19T11:23:44.433129+0000","last_peered":"2024-09-19T11:23:44.433129+0000","last_clean":"2024-09-19T11:23:44.433129+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:23:44.433129+0000","last_undegraded":"2024-09-19T11:23:44.433129+0000","last_fullsized":"2024-09-19T11:23:44.433129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:22.898396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:03.241382+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:01.554767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266030+0000","last_change":"2024-09-19T11:23:05.248307+0000","last_active":"2024-09-19T11:26:12.266030+0000","last_peered":"2024-09-19T11:26:12.266030+0000","last_clean":"2024-09-19T11:26:12.266030+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:12.266030+0000","last_undegraded":"2024-09-19T11:26:12.266030+0000","last_fullsized":"2024-09-19T11:26:12.266030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:44.091824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946371+0000","last_change":"2024-09-19T11:23:03.256127+0000","last_active":"2024-09-19T11:23:42.946371+0000","last_peered":"2024-09-19T11:23:42.946371+0000","last_clean":"2024-09-19T11:23:42.946371+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:23:42.946371+0000","last_undegraded":"2024-09-19T11:23:42.946371+0000","last_fullsized":"2024-09-19T11:23:42.946371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:03.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123790+0000","last_change":"2024-09-19T11:23:05.250503+0000","last_active":"2024-09-19T11:26:12.123790+0000","last_peered":"2024-09-19T11:26:12.123790+0000","last_clean":"2024-09-19T11:26:12.123790+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:26:12.123790+0000","last_undegraded":"2024-09-19T11:26:12.123790+0000","last_fullsized":"2024-09-19T11:26:12.123790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:46.547197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.993579+0000","last_change":"2024-09-19T11:23:03.259032+0000","last_active":"2024-09-19T11:23:44.993579+0000","last_peered":"2024-09-19T11:23:44.993579+0000","last_clean":"2024-09-19T11:23:44.993579+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:23:44.993579+0000","last_undegraded":"2024-09-19T11:23:44.993579+0000","last_fullsized":"2024-09-19T11:23:44.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:06.246922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368312+0000","last_change":"2024-09-19T11:23:05.248701+0000","last_active":"2024-09-19T11:26:12.368312+0000","last_peered":"2024-09-19T11:26:12.368312+0000","last_clean":"2024-09-19T11:26:12.368312+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:12.368312+0000","last_undegraded":"2024-09-19T11:26:12.368312+0000","last_fullsized":"2024-09-19T11:26:12.368312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:20.161230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947585+0000","last_change":"2024-09-19T11:23:03.252640+0000","last_active":"2024-09-19T11:23:42.947585+0000","last_peered":"2024-09-19T11:23:42.947585+0000","last_clean":"2024-09-19T11:23:42.947585+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:23:42.947585+0000","last_undegraded":"2024-09-19T11:23:42.947585+0000","last_fullsized":"2024-09-19T11:23:42.947585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:25.619002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811813+0000","last_change":"2024-09-19T11:23:03.256515+0000","last_active":"2024-09-19T11:25:26.811813+0000","last_peered":"2024-09-19T11:25:26.811813+0000","last_clean":"2024-09-19T11:25:26.811813+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:25:26.811813+0000","last_undegraded":"2024-09-19T11:25:26.811813+0000","last_fullsized":"2024-09-19T11:25:26.811813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:08.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123873+0000","last_change":"2024-09-19T11:23:05.248691+0000","last_active":"2024-09-19T11:26:12.123873+0000","last_peered":"2024-09-19T11:26:12.123873+0000","last_clean":"2024-09-19T11:26:12.123873+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:12.123873+0000","last_undegraded":"2024-09-19T11:26:12.123873+0000","last_fullsized":"2024-09-19T11:26:12.123873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:20.035436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278475+0000","last_change":"2024-09-19T11:23:03.243676+0000","last_active":"2024-09-19T11:23:42.278475+0000","last_peered":"2024-09-19T11:23:42.278475+0000","last_clean":"2024-09-19T11:23:42.278475+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:23:42.278475+0000","last_undegraded":"2024-09-19T11:23:42.278475+0000","last_fullsized":"2024-09-19T11:23:42.278475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:37.230042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510844+0000","last_change":"2024-09-19T11:23:05.245906+0000","last_active":"2024-09-19T11:25:11.510844+0000","last_peered":"2024-09-19T11:25:11.510844+0000","last_clean":"2024-09-19T11:25:11.510844+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:25:11.510844+0000","last_undegraded":"2024-09-19T11:25:11.510844+0000","last_fullsized":"2024-09-19T11:25:11.510844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:19.236446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":182,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":38719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673005,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40096,"kb_used_data":13072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":13385728,"data_stored":13057068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673005,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":5408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":5537792,"data_stored":5255453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40300,"kb_used_data":13276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331540,"statfs":{"total":96636764160,"available":96595496960,"internally_reserved":0,"allocated":13594624,"data_stored":13323694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40128,"kb_used_data":8944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331712,"statfs":{"total":96636764160,"available":96595673088,"internally_reserved":0,"allocated":9158656,"data_stored":8862463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4517888,"data_stored":4517888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12906496,"data_stored":12906496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:26:19.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:26:19.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b8033640 1 Processor -- start 2024-09-19T11:26:19.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b8033640 1 -- start start 2024-09-19T11:26:19.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b8033640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f11b012cba0 0x7f11b01315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b8033640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11b0099e20 con 0x7f11b012c0b0 2024-09-19T11:26:19.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b8033640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f11b00715a0 con 0x7f11b012cba0 2024-09-19T11:26:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b55ce640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f11b012cba0 0x7f11b01315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b5dcf640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f11b012c0b0 0x7f11b012c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41940/0 (socket says 172.21.15.22:41940) 2024-09-19T11:26:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.725+0000 7f11b5dcf640 1 -- 172.21.15.22:0/3070108548 learned_addr learned my addr 172.21.15.22:0/3070108548 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.726+0000 7f11b55ce640 1 -- 172.21.15.22:0/3070108548 >> v1:172.21.15.22:6789/0 conn(0x7f11b012c0b0 legacy=0x7f11b012c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.726+0000 7f11b55ce640 1 -- 172.21.15.22:0/3070108548 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b00705b0 con 0x7f11b012cba0 2024-09-19T11:26:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.726+0000 7f11b55ce640 1 --2- 172.21.15.22:0/3070108548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012cba0 0x7f11b01315e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009750 tx=0x7f11a002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca75e97cdca752a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b4dcd640 1 -- 172.21.15.22:0/3070108548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a0037070 con 0x7f11b012cba0 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b4dcd640 1 -- 172.21.15.22:0/3070108548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11a002ad80 con 0x7f11b012cba0 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b4dcd640 1 -- 172.21.15.22:0/3070108548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11a0032590 con 0x7f11b012cba0 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b8033640 1 -- 172.21.15.22:0/3070108548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012cba0 msgr2=0x7f11b01315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b8033640 1 --2- 172.21.15.22:0/3070108548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012cba0 0x7f11b01315e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009750 tx=0x7f11a002a140 comp rx=0 tx=0).stop 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b8033640 1 -- 172.21.15.22:0/3070108548 shutdown_connections 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b8033640 1 --2- 172.21.15.22:0/3070108548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012cba0 0x7f11b01315e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b8033640 1 -- 172.21.15.22:0/3070108548 >> 172.21.15.22:0/3070108548 conn(0x7f11b009abc0 msgr2=0x7f11b0098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b8033640 1 -- 172.21.15.22:0/3070108548 shutdown_connections 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.727+0000 7f11b8033640 1 -- 172.21.15.22:0/3070108548 wait complete. 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.728+0000 7f11b8033640 1 Processor -- start 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.728+0000 7f11b8033640 1 -- start start 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.728+0000 7f11b8033640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012c0b0 0x7f11b0128c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.728+0000 7f11b8033640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11b0131b20 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11b5dcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012c0b0 0x7f11b0128c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11b5dcf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012c0b0 0x7f11b0128c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35094/0 (socket says 172.21.15.22:35094) 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11b5dcf640 1 -- 172.21.15.22:0/2686901058 learned_addr learned my addr 172.21.15.22:0/2686901058 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11b5dcf640 1 -- 172.21.15.22:0/2686901058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b0074090 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11b5dcf640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012c0b0 0x7f11b0128c10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c480 tx=0x7f11ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11a67fc640 1 -- 172.21.15.22:0/2686901058 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac017070 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11a67fc640 1 -- 172.21.15.22:0/2686901058 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac005c10 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11a67fc640 1 -- 172.21.15.22:0/2686901058 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac007070 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b01287d0 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.729+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b0126e20 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.730+0000 7f11a67fc640 1 -- 172.21.15.22:0/2686901058 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f11ac01a070 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.730+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b012c490 con 0x7f11b012c0b0 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.730+0000 7f11a67fc640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f119403e710 0x7f1194040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:19.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.730+0000 7f11a67fc640 1 -- 172.21.15.22:0/2686901058 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f11ac052a80 con 0x7f11b012c0b0 2024-09-19T11:26:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.731+0000 7f11b55ce640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f119403e710 0x7f1194040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.731+0000 7f11b55ce640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f119403e710 0x7f1194040bd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009750 tx=0x7f11a002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:19.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.735+0000 7f11a67fc640 1 -- 172.21.15.22:0/2686901058 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f11b0074090 con 0x7f11b012c0b0 2024-09-19T11:26:19.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.918+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11b00008d0 con 0x7f119403e710 2024-09-19T11:26:19.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.929+0000 7f11a67fc640 1 -- 172.21.15.22:0/2686901058 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665813 (secure 0 0 0) 0x7f11b00008d0 con 0x7f119403e710 2024-09-19T11:26:19.942 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:26:19.949 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:26:19.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f119403e710 msgr2=0x7f1194040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f119403e710 0x7f1194040bd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009750 tx=0x7f11a002e000 comp rx=0 tx=0).stop 2024-09-19T11:26:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012c0b0 msgr2=0x7f11b0128c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012c0b0 0x7f11b0128c10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c480 tx=0x7f11ac00c940 comp rx=0 tx=0).stop 2024-09-19T11:26:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 shutdown_connections 2024-09-19T11:26:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f119403e710 0x7f1194040bd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 --2- 172.21.15.22:0/2686901058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11b012c0b0 0x7f11b0128c10 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 >> 172.21.15.22:0/2686901058 conn(0x7f11b009abc0 msgr2=0x7f11b006dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 shutdown_connections 2024-09-19T11:26:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:19.934+0000 7f11b8033640 1 -- 172.21.15.22:0/2686901058 wait complete. 2024-09-19T11:26:19.955 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-19T11:26:18.841537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412817,"num_objects":229,"num_object_clones":0,"num_object_copies":465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":229,"num_whiteouts":0,"num_read":4973,"num_read_kb":4511,"num_write":2048,"num_write_kb":60586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4434,"ondisk_log_size":4434,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":152956,"kb_used_data":40700,"kb_used_omap":6,"kb_used_meta":112185,"kb_avail":377334404,"statfs":{"total":386547056640,"available":386390429696,"internally_reserved":0,"allocated":41676800,"data_stored":40498678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":114878230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811688,"num_objects":9,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":477,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940271+0000","last_change":"2024-09-19T11:23:03.259044+0000","last_active":"2024-09-19T11:23:43.940271+0000","last_peered":"2024-09-19T11:23:43.940271+0000","last_clean":"2024-09-19T11:23:43.940271+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:23:43.940271+0000","last_undegraded":"2024-09-19T11:23:43.940271+0000","last_fullsized":"2024-09-19T11:23:43.940271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:41.435159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333097+0000","last_change":"2024-09-19T11:23:05.248495+0000","last_active":"2024-09-19T11:26:12.333097+0000","last_peered":"2024-09-19T11:26:12.333097+0000","last_clean":"2024-09-19T11:26:12.333097+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:12.333097+0000","last_undegraded":"2024-09-19T11:26:12.333097+0000","last_fullsized":"2024-09-19T11:26:12.333097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:27.929797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.237576+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:20:44.729040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333061+0000","last_change":"2024-09-19T11:23:05.242656+0000","last_active":"2024-09-19T11:26:12.333061+0000","last_peered":"2024-09-19T11:26:12.333061+0000","last_clean":"2024-09-19T11:26:12.333061+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:26:12.333061+0000","last_undegraded":"2024-09-19T11:26:12.333061+0000","last_fullsized":"2024-09-19T11:26:12.333061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.791517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.022905+0000","last_change":"2024-09-19T11:23:03.252002+0000","last_active":"2024-09-19T11:25:08.022905+0000","last_peered":"2024-09-19T11:25:08.022905+0000","last_clean":"2024-09-19T11:25:08.022905+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:25:08.022905+0000","last_undegraded":"2024-09-19T11:25:08.022905+0000","last_fullsized":"2024-09-19T11:25:08.022905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:04:47.977517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'11","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470023+0000","last_change":"2024-09-19T11:23:42.285084+0000","last_active":"2024-09-19T11:26:12.470023+0000","last_peered":"2024-09-19T11:26:12.470023+0000","last_clean":"2024-09-19T11:26:12.470023+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:12.470023+0000","last_undegraded":"2024-09-19T11:26:12.470023+0000","last_fullsized":"2024-09-19T11:26:12.470023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:31.524562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.939807+0000","last_change":"2024-09-19T11:23:03.257377+0000","last_active":"2024-09-19T11:23:43.939807+0000","last_peered":"2024-09-19T11:23:43.939807+0000","last_clean":"2024-09-19T11:23:43.939807+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:23:43.939807+0000","last_undegraded":"2024-09-19T11:23:43.939807+0000","last_fullsized":"2024-09-19T11:23:43.939807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:06.788652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944750+0000","last_change":"2024-09-19T11:23:05.233162+0000","last_active":"2024-09-19T11:23:42.944750+0000","last_peered":"2024-09-19T11:23:42.944750+0000","last_clean":"2024-09-19T11:23:42.944750+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:23:42.944750+0000","last_undegraded":"2024-09-19T11:23:42.944750+0000","last_fullsized":"2024-09-19T11:23:42.944750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:49.026520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:36.813838+0000","last_change":"2024-09-19T11:23:03.254504+0000","last_active":"2024-09-19T11:25:36.813838+0000","last_peered":"2024-09-19T11:25:36.813838+0000","last_clean":"2024-09-19T11:25:36.813838+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:25:36.813838+0000","last_undegraded":"2024-09-19T11:25:36.813838+0000","last_fullsized":"2024-09-19T11:25:36.813838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:05.223629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'11","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.128818+0000","last_change":"2024-09-19T11:23:05.247147+0000","last_active":"2024-09-19T11:26:12.128818+0000","last_peered":"2024-09-19T11:26:12.128818+0000","last_clean":"2024-09-19T11:26:12.128818+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:12.128818+0000","last_undegraded":"2024-09-19T11:26:12.128818+0000","last_fullsized":"2024-09-19T11:26:12.128818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:25.162001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276791+0000","last_change":"2024-09-19T11:23:03.256689+0000","last_active":"2024-09-19T11:23:42.276791+0000","last_peered":"2024-09-19T11:23:42.276791+0000","last_clean":"2024-09-19T11:23:42.276791+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:23:42.276791+0000","last_undegraded":"2024-09-19T11:23:42.276791+0000","last_fullsized":"2024-09-19T11:23:42.276791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:09.931084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374918+0000","last_change":"2024-09-19T11:23:05.254262+0000","last_active":"2024-09-19T11:26:12.374918+0000","last_peered":"2024-09-19T11:26:12.374918+0000","last_clean":"2024-09-19T11:26:12.374918+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:26:12.374918+0000","last_undegraded":"2024-09-19T11:26:12.374918+0000","last_fullsized":"2024-09-19T11:26:12.374918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:58.352519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.252828+0000","last_change":"2024-09-19T11:23:03.239664+0000","last_active":"2024-09-19T11:26:12.252828+0000","last_peered":"2024-09-19T11:26:12.252828+0000","last_clean":"2024-09-19T11:26:12.252828+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:12.252828+0000","last_undegraded":"2024-09-19T11:26:12.252828+0000","last_fullsized":"2024-09-19T11:26:12.252828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:43.733405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333282+0000","last_change":"2024-09-19T11:23:05.250554+0000","last_active":"2024-09-19T11:26:12.333282+0000","last_peered":"2024-09-19T11:26:12.333282+0000","last_clean":"2024-09-19T11:26:12.333282+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:12.333282+0000","last_undegraded":"2024-09-19T11:26:12.333282+0000","last_fullsized":"2024-09-19T11:26:12.333282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278484+0000","last_change":"2024-09-19T11:23:03.237565+0000","last_active":"2024-09-19T11:23:42.278484+0000","last_peered":"2024-09-19T11:23:42.278484+0000","last_clean":"2024-09-19T11:23:42.278484+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:23:42.278484+0000","last_undegraded":"2024-09-19T11:23:42.278484+0000","last_fullsized":"2024-09-19T11:23:42.278484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:24:40.441642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.012417+0000","last_change":"2024-09-19T11:23:05.246773+0000","last_active":"2024-09-19T11:23:45.012417+0000","last_peered":"2024-09-19T11:23:45.012417+0000","last_clean":"2024-09-19T11:23:45.012417+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:23:45.012417+0000","last_undegraded":"2024-09-19T11:23:45.012417+0000","last_fullsized":"2024-09-19T11:23:45.012417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:05:02.041530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:05.241849+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:04.673024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283475+0000","last_change":"2024-09-19T11:23:03.250062+0000","last_active":"2024-09-19T11:23:42.283475+0000","last_peered":"2024-09-19T11:23:42.283475+0000","last_clean":"2024-09-19T11:23:42.283475+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:23:42.283475+0000","last_undegraded":"2024-09-19T11:23:42.283475+0000","last_fullsized":"2024-09-19T11:23:42.283475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:32.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.483130+0000","last_change":"2024-09-19T11:23:05.246402+0000","last_active":"2024-09-19T11:23:45.483130+0000","last_peered":"2024-09-19T11:23:45.483130+0000","last_clean":"2024-09-19T11:23:45.483130+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:23:45.483130+0000","last_undegraded":"2024-09-19T11:23:45.483130+0000","last_fullsized":"2024-09-19T11:23:45.483130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:44.280015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281849+0000","last_change":"2024-09-19T11:23:03.259838+0000","last_active":"2024-09-19T11:23:42.281849+0000","last_peered":"2024-09-19T11:23:42.281849+0000","last_clean":"2024-09-19T11:23:42.281849+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:23:42.281849+0000","last_undegraded":"2024-09-19T11:23:42.281849+0000","last_fullsized":"2024-09-19T11:23:42.281849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342502+0000","last_change":"2024-09-19T11:23:05.248326+0000","last_active":"2024-09-19T11:24:27.342502+0000","last_peered":"2024-09-19T11:24:27.342502+0000","last_clean":"2024-09-19T11:24:27.342502+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:24:27.342502+0000","last_undegraded":"2024-09-19T11:24:27.342502+0000","last_fullsized":"2024-09-19T11:24:27.342502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:28.400630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.969817+0000","last_change":"2024-09-19T11:23:03.250122+0000","last_active":"2024-09-19T11:26:11.969817+0000","last_peered":"2024-09-19T11:26:11.969817+0000","last_clean":"2024-09-19T11:26:11.969817+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:11.969817+0000","last_undegraded":"2024-09-19T11:26:11.969817+0000","last_fullsized":"2024-09-19T11:26:11.969817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:13.455767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123608+0000","last_change":"2024-09-19T11:23:05.240918+0000","last_active":"2024-09-19T11:26:12.123608+0000","last_peered":"2024-09-19T11:26:12.123608+0000","last_clean":"2024-09-19T11:26:12.123608+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:12.123608+0000","last_undegraded":"2024-09-19T11:26:12.123608+0000","last_fullsized":"2024-09-19T11:26:12.123608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:19.590735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282416+0000","last_change":"2024-09-19T11:23:03.245456+0000","last_active":"2024-09-19T11:23:42.282416+0000","last_peered":"2024-09-19T11:23:42.282416+0000","last_clean":"2024-09-19T11:23:42.282416+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:23:42.282416+0000","last_undegraded":"2024-09-19T11:23:42.282416+0000","last_fullsized":"2024-09-19T11:23:42.282416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:14.954640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266363+0000","last_change":"2024-09-19T11:23:05.245486+0000","last_active":"2024-09-19T11:26:12.266363+0000","last_peered":"2024-09-19T11:26:12.266363+0000","last_clean":"2024-09-19T11:26:12.266363+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:12.266363+0000","last_undegraded":"2024-09-19T11:26:12.266363+0000","last_fullsized":"2024-09-19T11:26:12.266363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.454541+0000","last_change":"2024-09-19T11:23:03.243483+0000","last_active":"2024-09-19T11:25:11.454541+0000","last_peered":"2024-09-19T11:25:11.454541+0000","last_clean":"2024-09-19T11:25:11.454541+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:25:11.454541+0000","last_undegraded":"2024-09-19T11:25:11.454541+0000","last_fullsized":"2024-09-19T11:25:11.454541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:10.061252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266347+0000","last_change":"2024-09-19T11:23:05.248431+0000","last_active":"2024-09-19T11:26:12.266347+0000","last_peered":"2024-09-19T11:26:12.266347+0000","last_clean":"2024-09-19T11:26:12.266347+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:12.266347+0000","last_undegraded":"2024-09-19T11:26:12.266347+0000","last_fullsized":"2024-09-19T11:26:12.266347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:17.337349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945193+0000","last_change":"2024-09-19T11:23:03.240547+0000","last_active":"2024-09-19T11:23:42.945193+0000","last_peered":"2024-09-19T11:23:42.945193+0000","last_clean":"2024-09-19T11:23:42.945193+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:23:42.945193+0000","last_undegraded":"2024-09-19T11:23:42.945193+0000","last_fullsized":"2024-09-19T11:23:42.945193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:55.218589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'37","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.305388+0000","last_change":"2024-09-19T11:23:13.275524+0000","last_active":"2024-09-19T11:26:12.305388+0000","last_peered":"2024-09-19T11:26:12.305388+0000","last_clean":"2024-09-19T11:26:12.305388+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:26:12.305388+0000","last_undegraded":"2024-09-19T11:26:12.305388+0000","last_fullsized":"2024-09-19T11:26:12.305388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.670701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.018412+0000","last_change":"2024-09-19T11:23:42.283032+0000","last_active":"2024-09-19T11:26:12.018412+0000","last_peered":"2024-09-19T11:26:12.018412+0000","last_clean":"2024-09-19T11:26:12.018412+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:12.018412+0000","last_undegraded":"2024-09-19T11:26:12.018412+0000","last_fullsized":"2024-09-19T11:26:12.018412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.760502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811976+0000","last_change":"2024-09-19T11:23:03.258428+0000","last_active":"2024-09-19T11:25:26.811976+0000","last_peered":"2024-09-19T11:25:26.811976+0000","last_clean":"2024-09-19T11:25:26.811976+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:25:26.811976+0000","last_undegraded":"2024-09-19T11:25:26.811976+0000","last_fullsized":"2024-09-19T11:25:26.811976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:51.654373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945113+0000","last_change":"2024-09-19T11:23:07.245040+0000","last_active":"2024-09-19T11:23:42.945113+0000","last_peered":"2024-09-19T11:23:42.945113+0000","last_clean":"2024-09-19T11:23:42.945113+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:23:42.945113+0000","last_undegraded":"2024-09-19T11:23:42.945113+0000","last_fullsized":"2024-09-19T11:23:42.945113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.056714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.749025+0000","last_change":"2024-09-19T11:23:09.256765+0000","last_active":"2024-09-19T11:25:09.749025+0000","last_peered":"2024-09-19T11:25:09.749025+0000","last_clean":"2024-09-19T11:25:09.749025+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:25:09.749025+0000","last_undegraded":"2024-09-19T11:25:09.749025+0000","last_fullsized":"2024-09-19T11:25:09.749025+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:39:32.451376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283356+0000","last_change":"2024-09-19T11:23:11.267021+0000","last_active":"2024-09-19T11:23:42.283356+0000","last_peered":"2024-09-19T11:23:42.283356+0000","last_clean":"2024-09-19T11:23:42.283356+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:23:42.283356+0000","last_undegraded":"2024-09-19T11:23:42.283356+0000","last_fullsized":"2024-09-19T11:23:42.283356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.806510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.707113+0000","last_change":"2024-09-19T11:23:13.274325+0000","last_active":"2024-09-19T11:26:11.707113+0000","last_peered":"2024-09-19T11:26:11.707113+0000","last_clean":"2024-09-19T11:26:11.707113+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:26:11.707113+0000","last_undegraded":"2024-09-19T11:26:11.707113+0000","last_fullsized":"2024-09-19T11:26:11.707113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:53:22.517816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.332597+0000","last_change":"2024-09-19T11:23:05.237978+0000","last_active":"2024-09-19T11:26:12.332597+0000","last_peered":"2024-09-19T11:26:12.332597+0000","last_clean":"2024-09-19T11:26:12.332597+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:12.332597+0000","last_undegraded":"2024-09-19T11:26:12.332597+0000","last_fullsized":"2024-09-19T11:26:12.332597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:02.964073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.319875+0000","last_change":"2024-09-19T11:23:03.256489+0000","last_active":"2024-09-19T11:25:11.319875+0000","last_peered":"2024-09-19T11:25:11.319875+0000","last_clean":"2024-09-19T11:25:11.319875+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:25:11.319875+0000","last_undegraded":"2024-09-19T11:25:11.319875+0000","last_fullsized":"2024-09-19T11:25:11.319875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:34.312130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279716+0000","last_change":"2024-09-19T11:23:07.245213+0000","last_active":"2024-09-19T11:23:42.279716+0000","last_peered":"2024-09-19T11:23:42.279716+0000","last_clean":"2024-09-19T11:23:42.279716+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:23:42.279716+0000","last_undegraded":"2024-09-19T11:23:42.279716+0000","last_fullsized":"2024-09-19T11:23:42.279716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:56.312564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'7","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.551668+0000","last_change":"2024-09-19T11:23:09.248257+0000","last_active":"2024-09-19T11:24:27.551668+0000","last_peered":"2024-09-19T11:24:27.551668+0000","last_clean":"2024-09-19T11:24:27.551668+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:24:27.551668+0000","last_undegraded":"2024-09-19T11:24:27.551668+0000","last_fullsized":"2024-09-19T11:24:27.551668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:18.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281824+0000","last_change":"2024-09-19T11:23:11.263544+0000","last_active":"2024-09-19T11:23:42.281824+0000","last_peered":"2024-09-19T11:23:42.281824+0000","last_clean":"2024-09-19T11:23:42.281824+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:23:42.281824+0000","last_undegraded":"2024-09-19T11:23:42.281824+0000","last_fullsized":"2024-09-19T11:23:42.281824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:44.652949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278313+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:23:42.278313+0000","last_peered":"2024-09-19T11:23:42.278313+0000","last_clean":"2024-09-19T11:23:42.278313+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:23:42.278313+0000","last_undegraded":"2024-09-19T11:23:42.278313+0000","last_fullsized":"2024-09-19T11:23:42.278313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:33:33.618068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510054+0000","last_change":"2024-09-19T11:23:05.236597+0000","last_active":"2024-09-19T11:25:11.510054+0000","last_peered":"2024-09-19T11:25:11.510054+0000","last_clean":"2024-09-19T11:25:11.510054+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:25:11.510054+0000","last_undegraded":"2024-09-19T11:25:11.510054+0000","last_fullsized":"2024-09-19T11:25:11.510054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:11.702021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.486679+0000","last_change":"2024-09-19T11:23:03.257697+0000","last_active":"2024-09-19T11:25:11.486679+0000","last_peered":"2024-09-19T11:25:11.486679+0000","last_clean":"2024-09-19T11:25:11.486679+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:25:11.486679+0000","last_undegraded":"2024-09-19T11:25:11.486679+0000","last_fullsized":"2024-09-19T11:25:11.486679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:55.338804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796210+0000","last_change":"2024-09-19T11:23:42.284049+0000","last_active":"2024-09-19T11:23:44.796210+0000","last_peered":"2024-09-19T11:23:44.796210+0000","last_clean":"2024-09-19T11:23:44.796210+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:23:44.796210+0000","last_undegraded":"2024-09-19T11:23:44.796210+0000","last_fullsized":"2024-09-19T11:23:44.796210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279160+0000","last_change":"2024-09-19T11:23:09.253591+0000","last_active":"2024-09-19T11:23:42.279160+0000","last_peered":"2024-09-19T11:23:42.279160+0000","last_clean":"2024-09-19T11:23:42.279160+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:23:42.279160+0000","last_undegraded":"2024-09-19T11:23:42.279160+0000","last_fullsized":"2024-09-19T11:23:42.279160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:52.970909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'282","reported_seq":386,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.615383+0000","last_change":"2024-09-19T11:23:11.271056+0000","last_active":"2024-09-19T11:26:12.615383+0000","last_peered":"2024-09-19T11:26:12.615383+0000","last_clean":"2024-09-19T11:26:12.615383+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:26:12.615383+0000","last_undegraded":"2024-09-19T11:26:12.615383+0000","last_fullsized":"2024-09-19T11:26:12.615383+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:07.822761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.312802+0000","last_change":"2024-09-19T11:23:13.270157+0000","last_active":"2024-09-19T11:26:12.312802+0000","last_peered":"2024-09-19T11:26:12.312802+0000","last_clean":"2024-09-19T11:26:12.312802+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:26:12.312802+0000","last_undegraded":"2024-09-19T11:26:12.312802+0000","last_fullsized":"2024-09-19T11:26:12.312802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:32.371537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.781965+0000","last_change":"2024-09-19T11:23:05.248712+0000","last_active":"2024-09-19T11:23:44.781965+0000","last_peered":"2024-09-19T11:23:44.781965+0000","last_clean":"2024-09-19T11:23:44.781965+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:23:44.781965+0000","last_undegraded":"2024-09-19T11:23:44.781965+0000","last_fullsized":"2024-09-19T11:23:44.781965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:01:31.288073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279237+0000","last_change":"2024-09-19T11:23:03.250291+0000","last_active":"2024-09-19T11:23:42.279237+0000","last_peered":"2024-09-19T11:23:42.279237+0000","last_clean":"2024-09-19T11:23:42.279237+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:23:42.279237+0000","last_undegraded":"2024-09-19T11:23:42.279237+0000","last_fullsized":"2024-09-19T11:23:42.279237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:27.615110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947691+0000","last_change":"2024-09-19T11:23:07.248024+0000","last_active":"2024-09-19T11:23:42.947691+0000","last_peered":"2024-09-19T11:23:42.947691+0000","last_clean":"2024-09-19T11:23:42.947691+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:23:42.947691+0000","last_undegraded":"2024-09-19T11:23:42.947691+0000","last_fullsized":"2024-09-19T11:23:42.947691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:02.812050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.743749+0000","last_change":"2024-09-19T11:23:09.260866+0000","last_active":"2024-09-19T11:25:09.743749+0000","last_peered":"2024-09-19T11:25:09.743749+0000","last_clean":"2024-09-19T11:25:09.743749+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:25:09.743749+0000","last_undegraded":"2024-09-19T11:25:09.743749+0000","last_fullsized":"2024-09-19T11:25:09.743749+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:22.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'282","reported_seq":402,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.371212+0000","last_change":"2024-09-19T11:23:11.260474+0000","last_active":"2024-09-19T11:26:12.371212+0000","last_peered":"2024-09-19T11:26:12.371212+0000","last_clean":"2024-09-19T11:26:12.371212+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:26:12.371212+0000","last_undegraded":"2024-09-19T11:26:12.371212+0000","last_fullsized":"2024-09-19T11:26:12.371212+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:32.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032873+0000","last_change":"2024-09-19T11:23:05.253843+0000","last_active":"2024-09-19T11:25:08.032873+0000","last_peered":"2024-09-19T11:25:08.032873+0000","last_clean":"2024-09-19T11:25:08.032873+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:25:08.032873+0000","last_undegraded":"2024-09-19T11:25:08.032873+0000","last_fullsized":"2024-09-19T11:25:08.032873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:22.232407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279601+0000","last_change":"2024-09-19T11:23:03.256377+0000","last_active":"2024-09-19T11:23:42.279601+0000","last_peered":"2024-09-19T11:23:42.279601+0000","last_clean":"2024-09-19T11:23:42.279601+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:23:42.279601+0000","last_undegraded":"2024-09-19T11:23:42.279601+0000","last_fullsized":"2024-09-19T11:23:42.279601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:29.305136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281170+0000","last_change":"2024-09-19T11:23:07.244371+0000","last_active":"2024-09-19T11:23:42.281170+0000","last_peered":"2024-09-19T11:23:42.281170+0000","last_clean":"2024-09-19T11:23:42.281170+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:23:42.281170+0000","last_undegraded":"2024-09-19T11:23:42.281170+0000","last_fullsized":"2024-09-19T11:23:42.281170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:53.267571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'40","reported_seq":88,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.753372+0000","last_change":"2024-09-19T11:23:09.249075+0000","last_active":"2024-09-19T11:25:09.753372+0000","last_peered":"2024-09-19T11:25:09.753372+0000","last_clean":"2024-09-19T11:25:09.753372+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:25:09.753372+0000","last_undegraded":"2024-09-19T11:25:09.753372+0000","last_fullsized":"2024-09-19T11:25:09.753372+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:03.176195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281316+0000","last_change":"2024-09-19T11:23:11.258275+0000","last_active":"2024-09-19T11:23:42.281316+0000","last_peered":"2024-09-19T11:23:42.281316+0000","last_clean":"2024-09-19T11:23:42.281316+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:23:42.281316+0000","last_undegraded":"2024-09-19T11:23:42.281316+0000","last_fullsized":"2024-09-19T11:23:42.281316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:14.017912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.349482+0000","last_change":"2024-09-19T11:23:13.274689+0000","last_active":"2024-09-19T11:24:27.349482+0000","last_peered":"2024-09-19T11:24:27.349482+0000","last_clean":"2024-09-19T11:24:27.349482+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:24:27.349482+0000","last_undegraded":"2024-09-19T11:24:27.349482+0000","last_fullsized":"2024-09-19T11:24:27.349482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:05.210803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281462+0000","last_change":"2024-09-19T11:23:05.245370+0000","last_active":"2024-09-19T11:23:42.281462+0000","last_peered":"2024-09-19T11:23:42.281462+0000","last_clean":"2024-09-19T11:23:42.281462+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:23:42.281462+0000","last_undegraded":"2024-09-19T11:23:42.281462+0000","last_fullsized":"2024-09-19T11:23:42.281462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:55:45.570918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280712+0000","last_change":"2024-09-19T11:23:03.257588+0000","last_active":"2024-09-19T11:23:42.280712+0000","last_peered":"2024-09-19T11:23:42.280712+0000","last_clean":"2024-09-19T11:23:42.280712+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:23:42.280712+0000","last_undegraded":"2024-09-19T11:23:42.280712+0000","last_fullsized":"2024-09-19T11:23:42.280712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.928791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278703+0000","last_change":"2024-09-19T11:23:07.247386+0000","last_active":"2024-09-19T11:23:42.278703+0000","last_peered":"2024-09-19T11:23:42.278703+0000","last_clean":"2024-09-19T11:23:42.278703+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:23:42.278703+0000","last_undegraded":"2024-09-19T11:23:42.278703+0000","last_fullsized":"2024-09-19T11:23:42.278703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:53.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'30","reported_seq":73,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.733747+0000","last_change":"2024-09-19T11:23:09.262668+0000","last_active":"2024-09-19T11:25:09.733747+0000","last_peered":"2024-09-19T11:25:09.733747+0000","last_clean":"2024-09-19T11:25:09.733747+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:25:09.733747+0000","last_undegraded":"2024-09-19T11:25:09.733747+0000","last_fullsized":"2024-09-19T11:25:09.733747+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:42.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278485+0000","last_change":"2024-09-19T11:23:11.265649+0000","last_active":"2024-09-19T11:23:42.278485+0000","last_peered":"2024-09-19T11:23:42.278485+0000","last_clean":"2024-09-19T11:23:42.278485+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:23:42.278485+0000","last_undegraded":"2024-09-19T11:23:42.278485+0000","last_fullsized":"2024-09-19T11:23:42.278485+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:22.946739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'5","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.124970+0000","last_change":"2024-09-19T11:23:13.268845+0000","last_active":"2024-09-19T11:26:12.124970+0000","last_peered":"2024-09-19T11:26:12.124970+0000","last_clean":"2024-09-19T11:26:12.124970+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:26:12.124970+0000","last_undegraded":"2024-09-19T11:26:12.124970+0000","last_fullsized":"2024-09-19T11:26:12.124970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:30.688522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342674+0000","last_change":"2024-09-19T11:23:05.251302+0000","last_active":"2024-09-19T11:24:27.342674+0000","last_peered":"2024-09-19T11:24:27.342674+0000","last_clean":"2024-09-19T11:24:27.342674+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:24:27.342674+0000","last_undegraded":"2024-09-19T11:24:27.342674+0000","last_fullsized":"2024-09-19T11:24:27.342674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:55.904446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'3","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.249995+0000","last_change":"2024-09-19T11:23:03.242612+0000","last_active":"2024-09-19T11:26:12.249995+0000","last_peered":"2024-09-19T11:26:12.249995+0000","last_clean":"2024-09-19T11:26:12.249995+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:12.249995+0000","last_undegraded":"2024-09-19T11:26:12.249995+0000","last_fullsized":"2024-09-19T11:26:12.249995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:48.666399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":11,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281855+0000","last_change":"2024-09-19T11:23:07.247594+0000","last_active":"2024-09-19T11:23:42.281855+0000","last_peered":"2024-09-19T11:23:42.281855+0000","last_clean":"2024-09-19T11:23:42.281855+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:23:42.281855+0000","last_undegraded":"2024-09-19T11:23:42.281855+0000","last_fullsized":"2024-09-19T11:23:42.281855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:41.108832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.755394+0000","last_change":"2024-09-19T11:23:09.259476+0000","last_active":"2024-09-19T11:25:09.755394+0000","last_peered":"2024-09-19T11:25:09.755394+0000","last_clean":"2024-09-19T11:25:09.755394+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:25:09.755394+0000","last_undegraded":"2024-09-19T11:25:09.755394+0000","last_fullsized":"2024-09-19T11:25:09.755394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:21.364894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'282","reported_seq":349,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.641675+0000","last_change":"2024-09-19T11:23:11.267050+0000","last_active":"2024-09-19T11:26:12.641675+0000","last_peered":"2024-09-19T11:26:12.641675+0000","last_clean":"2024-09-19T11:26:12.641675+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:26:12.641675+0000","last_undegraded":"2024-09-19T11:26:12.641675+0000","last_fullsized":"2024-09-19T11:26:12.641675+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.330204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.334604+0000","last_change":"2024-09-19T11:23:13.269663+0000","last_active":"2024-09-19T11:26:12.334604+0000","last_peered":"2024-09-19T11:26:12.334604+0000","last_clean":"2024-09-19T11:26:12.334604+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:26:12.334604+0000","last_undegraded":"2024-09-19T11:26:12.334604+0000","last_fullsized":"2024-09-19T11:26:12.334604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:57:11.742213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368223+0000","last_change":"2024-09-19T11:23:05.235632+0000","last_active":"2024-09-19T11:26:12.368223+0000","last_peered":"2024-09-19T11:26:12.368223+0000","last_clean":"2024-09-19T11:26:12.368223+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:12.368223+0000","last_undegraded":"2024-09-19T11:26:12.368223+0000","last_fullsized":"2024-09-19T11:26:12.368223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945253+0000","last_change":"2024-09-19T11:23:03.257570+0000","last_active":"2024-09-19T11:23:42.945253+0000","last_peered":"2024-09-19T11:23:42.945253+0000","last_clean":"2024-09-19T11:23:42.945253+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:23:42.945253+0000","last_undegraded":"2024-09-19T11:23:42.945253+0000","last_fullsized":"2024-09-19T11:23:42.945253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:41:29.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279784+0000","last_change":"2024-09-19T11:23:07.247239+0000","last_active":"2024-09-19T11:23:42.279784+0000","last_peered":"2024-09-19T11:23:42.279784+0000","last_clean":"2024-09-19T11:23:42.279784+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:23:42.279784+0000","last_undegraded":"2024-09-19T11:23:42.279784+0000","last_fullsized":"2024-09-19T11:23:42.279784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:21.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'55","reported_seq":112,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.512267+0000","last_change":"2024-09-19T11:23:09.259576+0000","last_active":"2024-09-19T11:25:11.512267+0000","last_peered":"2024-09-19T11:25:11.512267+0000","last_clean":"2024-09-19T11:25:11.512267+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:25:11.512267+0000","last_undegraded":"2024-09-19T11:25:11.512267+0000","last_fullsized":"2024-09-19T11:25:11.512267+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.556001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278323+0000","last_change":"2024-09-19T11:23:11.260403+0000","last_active":"2024-09-19T11:23:42.278323+0000","last_peered":"2024-09-19T11:23:42.278323+0000","last_clean":"2024-09-19T11:23:42.278323+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:23:42.278323+0000","last_undegraded":"2024-09-19T11:23:42.278323+0000","last_fullsized":"2024-09-19T11:23:42.278323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:11.335945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'18","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585256+0000","last_change":"2024-09-19T11:23:13.273720+0000","last_active":"2024-09-19T11:26:12.585256+0000","last_peered":"2024-09-19T11:26:12.585256+0000","last_clean":"2024-09-19T11:26:12.585256+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:26:12.585256+0000","last_undegraded":"2024-09-19T11:26:12.585256+0000","last_fullsized":"2024-09-19T11:26:12.585256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:46.547393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:21.810844+0000","last_change":"2024-09-19T11:23:05.247032+0000","last_active":"2024-09-19T11:25:21.810844+0000","last_peered":"2024-09-19T11:25:21.810844+0000","last_clean":"2024-09-19T11:25:21.810844+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:25:21.810844+0000","last_undegraded":"2024-09-19T11:25:21.810844+0000","last_fullsized":"2024-09-19T11:25:21.810844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:24.496941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.255786+0000","last_change":"2024-09-19T11:23:03.234223+0000","last_active":"2024-09-19T11:26:12.255786+0000","last_peered":"2024-09-19T11:26:12.255786+0000","last_clean":"2024-09-19T11:26:12.255786+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:12.255786+0000","last_undegraded":"2024-09-19T11:26:12.255786+0000","last_fullsized":"2024-09-19T11:26:12.255786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278469+0000","last_change":"2024-09-19T11:23:07.244885+0000","last_active":"2024-09-19T11:23:42.278469+0000","last_peered":"2024-09-19T11:23:42.278469+0000","last_clean":"2024-09-19T11:23:42.278469+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:23:42.278469+0000","last_undegraded":"2024-09-19T11:23:42.278469+0000","last_fullsized":"2024-09-19T11:23:42.278469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:45:02.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'33","reported_seq":77,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811791+0000","last_change":"2024-09-19T11:23:09.248190+0000","last_active":"2024-09-19T11:25:26.811791+0000","last_peered":"2024-09-19T11:25:26.811791+0000","last_clean":"2024-09-19T11:25:26.811791+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:25:26.811791+0000","last_undegraded":"2024-09-19T11:25:26.811791+0000","last_fullsized":"2024-09-19T11:25:26.811791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:38.180379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946607+0000","last_change":"2024-09-19T11:23:11.270709+0000","last_active":"2024-09-19T11:23:42.946607+0000","last_peered":"2024-09-19T11:23:42.946607+0000","last_clean":"2024-09-19T11:23:42.946607+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:23:42.946607+0000","last_undegraded":"2024-09-19T11:23:42.946607+0000","last_fullsized":"2024-09-19T11:23:42.946607+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:39.157590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809883+0000","last_change":"2024-09-19T11:23:13.272542+0000","last_active":"2024-09-19T11:25:16.809883+0000","last_peered":"2024-09-19T11:25:16.809883+0000","last_clean":"2024-09-19T11:25:16.809883+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:25:16.809883+0000","last_undegraded":"2024-09-19T11:25:16.809883+0000","last_fullsized":"2024-09-19T11:25:16.809883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:52.793325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433382+0000","last_change":"2024-09-19T11:23:05.254593+0000","last_active":"2024-09-19T11:23:44.433382+0000","last_peered":"2024-09-19T11:23:44.433382+0000","last_clean":"2024-09-19T11:23:44.433382+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:23:44.433382+0000","last_undegraded":"2024-09-19T11:23:44.433382+0000","last_fullsized":"2024-09-19T11:23:44.433382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:27.816142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945690+0000","last_change":"2024-09-19T11:23:03.241691+0000","last_active":"2024-09-19T11:23:42.945690+0000","last_peered":"2024-09-19T11:23:42.945690+0000","last_clean":"2024-09-19T11:23:42.945690+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:23:42.945690+0000","last_undegraded":"2024-09-19T11:23:42.945690+0000","last_fullsized":"2024-09-19T11:23:42.945690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:21.286478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279479+0000","last_change":"2024-09-19T11:23:07.247411+0000","last_active":"2024-09-19T11:23:42.279479+0000","last_peered":"2024-09-19T11:23:42.279479+0000","last_clean":"2024-09-19T11:23:42.279479+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:23:42.279479+0000","last_undegraded":"2024-09-19T11:23:42.279479+0000","last_fullsized":"2024-09-19T11:23:42.279479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:20:41.691053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.763568+0000","last_change":"2024-09-19T11:23:09.246515+0000","last_active":"2024-09-19T11:25:09.763568+0000","last_peered":"2024-09-19T11:25:09.763568+0000","last_clean":"2024-09-19T11:25:09.763568+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:25:09.763568+0000","last_undegraded":"2024-09-19T11:25:09.763568+0000","last_fullsized":"2024-09-19T11:25:09.763568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:54.969396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'282","reported_seq":404,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.589475+0000","last_change":"2024-09-19T11:23:11.263693+0000","last_active":"2024-09-19T11:26:12.589475+0000","last_peered":"2024-09-19T11:26:12.589475+0000","last_clean":"2024-09-19T11:26:12.589475+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:26:12.589475+0000","last_undegraded":"2024-09-19T11:26:12.589475+0000","last_fullsized":"2024-09-19T11:26:12.589475+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:29.980859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'14","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.475120+0000","last_change":"2024-09-19T11:23:13.265074+0000","last_active":"2024-09-19T11:26:12.475120+0000","last_peered":"2024-09-19T11:26:12.475120+0000","last_clean":"2024-09-19T11:26:12.475120+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:26:12.475120+0000","last_undegraded":"2024-09-19T11:26:12.475120+0000","last_fullsized":"2024-09-19T11:26:12.475120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:47.118156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368115+0000","last_change":"2024-09-19T11:23:05.250216+0000","last_active":"2024-09-19T11:26:12.368115+0000","last_peered":"2024-09-19T11:26:12.368115+0000","last_clean":"2024-09-19T11:26:12.368115+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:12.368115+0000","last_undegraded":"2024-09-19T11:26:12.368115+0000","last_fullsized":"2024-09-19T11:26:12.368115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.941399+0000","last_change":"2024-09-19T11:23:03.255564+0000","last_active":"2024-09-19T11:23:43.941399+0000","last_peered":"2024-09-19T11:23:43.941399+0000","last_clean":"2024-09-19T11:23:43.941399+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:23:43.941399+0000","last_undegraded":"2024-09-19T11:23:43.941399+0000","last_fullsized":"2024-09-19T11:23:43.941399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:10:03.443220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'3","reported_seq":414,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.866528+0000","last_change":"2024-09-19T11:23:07.242151+0000","last_active":"2024-09-19T11:25:10.866528+0000","last_peered":"2024-09-19T11:25:10.866528+0000","last_clean":"2024-09-19T11:25:10.866528+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:25:10.866528+0000","last_undegraded":"2024-09-19T11:25:10.866528+0000","last_fullsized":"2024-09-19T11:25:10.866528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:15.211742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946255+0000","last_change":"2024-09-19T11:23:09.255928+0000","last_active":"2024-09-19T11:23:42.946255+0000","last_peered":"2024-09-19T11:23:42.946255+0000","last_clean":"2024-09-19T11:23:42.946255+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:23:42.946255+0000","last_undegraded":"2024-09-19T11:23:42.946255+0000","last_fullsized":"2024-09-19T11:23:42.946255+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:04.689951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'282","reported_seq":466,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.473914+0000","last_change":"2024-09-19T11:23:11.255933+0000","last_active":"2024-09-19T11:26:12.473914+0000","last_peered":"2024-09-19T11:26:12.473914+0000","last_clean":"2024-09-19T11:26:12.473914+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:26:12.473914+0000","last_undegraded":"2024-09-19T11:26:12.473914+0000","last_fullsized":"2024-09-19T11:26:12.473914+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:01.160503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.141728+0000","last_change":"2024-09-19T11:23:13.266957+0000","last_active":"2024-09-19T11:25:11.141728+0000","last_peered":"2024-09-19T11:25:11.141728+0000","last_clean":"2024-09-19T11:25:11.141728+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:25:11.141728+0000","last_undegraded":"2024-09-19T11:25:11.141728+0000","last_fullsized":"2024-09-19T11:25:11.141728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:59.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":101,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338578+0000","last_change":"2024-09-19T11:23:05.255344+0000","last_active":"2024-09-19T11:26:12.338578+0000","last_peered":"2024-09-19T11:26:12.338578+0000","last_clean":"2024-09-19T11:26:12.338578+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:12.338578+0000","last_undegraded":"2024-09-19T11:26:12.338578+0000","last_fullsized":"2024-09-19T11:26:12.338578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:22.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944743+0000","last_change":"2024-09-19T11:23:03.254989+0000","last_active":"2024-09-19T11:23:42.944743+0000","last_peered":"2024-09-19T11:23:42.944743+0000","last_clean":"2024-09-19T11:23:42.944743+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:23:42.944743+0000","last_undegraded":"2024-09-19T11:23:42.944743+0000","last_fullsized":"2024-09-19T11:23:42.944743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:39.840111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":416,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.866830+0000","last_change":"2024-09-19T11:23:07.250416+0000","last_active":"2024-09-19T11:25:10.866830+0000","last_peered":"2024-09-19T11:25:10.866830+0000","last_clean":"2024-09-19T11:25:10.866830+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:25:10.866830+0000","last_undegraded":"2024-09-19T11:25:10.866830+0000","last_fullsized":"2024-09-19T11:25:10.866830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":580,"num_read_kb":580,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'38","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.750105+0000","last_change":"2024-09-19T11:23:09.247733+0000","last_active":"2024-09-19T11:25:09.750105+0000","last_peered":"2024-09-19T11:25:09.750105+0000","last_clean":"2024-09-19T11:25:09.750105+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:25:09.750105+0000","last_undegraded":"2024-09-19T11:25:09.750105+0000","last_fullsized":"2024-09-19T11:25:09.750105+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:38.921761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279060+0000","last_change":"2024-09-19T11:23:11.266008+0000","last_active":"2024-09-19T11:23:42.279060+0000","last_peered":"2024-09-19T11:23:42.279060+0000","last_clean":"2024-09-19T11:23:42.279060+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:23:42.279060+0000","last_undegraded":"2024-09-19T11:23:42.279060+0000","last_fullsized":"2024-09-19T11:23:42.279060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:38:43.871781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.426809+0000","last_change":"2024-09-19T11:23:13.263896+0000","last_active":"2024-09-19T11:26:12.426809+0000","last_peered":"2024-09-19T11:26:12.426809+0000","last_clean":"2024-09-19T11:26:12.426809+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:26:12.426809+0000","last_undegraded":"2024-09-19T11:26:12.426809+0000","last_fullsized":"2024-09-19T11:26:12.426809+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:35.364893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'7","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.273438+0000","last_change":"2024-09-19T11:23:05.254167+0000","last_active":"2024-09-19T11:26:12.273438+0000","last_peered":"2024-09-19T11:26:12.273438+0000","last_clean":"2024-09-19T11:26:12.273438+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:12.273438+0000","last_undegraded":"2024-09-19T11:26:12.273438+0000","last_fullsized":"2024-09-19T11:26:12.273438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:44.425539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":146,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279138+0000","last_change":"2024-09-19T11:22:46.832286+0000","last_active":"2024-09-19T11:23:42.279138+0000","last_peered":"2024-09-19T11:23:42.279138+0000","last_clean":"2024-09-19T11:23:42.279138+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:23:42.279138+0000","last_undegraded":"2024-09-19T11:23:42.279138+0000","last_fullsized":"2024-09-19T11:23:42.279138+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_clean_scrub_stamp":"2024-09-19T11:22:45.784999+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:24.575130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944159+0000","last_change":"2024-09-19T11:22:55.817718+0000","last_active":"2024-09-19T11:23:42.944159+0000","last_peered":"2024-09-19T11:23:42.944159+0000","last_clean":"2024-09-19T11:23:42.944159+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:23:42.944159+0000","last_undegraded":"2024-09-19T11:23:42.944159+0000","last_fullsized":"2024-09-19T11:23:42.944159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:33.890847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944557+0000","last_change":"2024-09-19T11:23:03.258130+0000","last_active":"2024-09-19T11:23:42.944557+0000","last_peered":"2024-09-19T11:23:42.944557+0000","last_clean":"2024-09-19T11:23:42.944557+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:23:42.944557+0000","last_undegraded":"2024-09-19T11:23:42.944557+0000","last_fullsized":"2024-09-19T11:23:42.944557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:11.381818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281740+0000","last_change":"2024-09-19T11:23:07.238893+0000","last_active":"2024-09-19T11:23:42.281740+0000","last_peered":"2024-09-19T11:23:42.281740+0000","last_clean":"2024-09-19T11:23:42.281740+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:23:42.281740+0000","last_undegraded":"2024-09-19T11:23:42.281740+0000","last_fullsized":"2024-09-19T11:23:42.281740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:35.034936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'76","reported_seq":146,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.761411+0000","last_change":"2024-09-19T11:23:09.260843+0000","last_active":"2024-09-19T11:25:09.761411+0000","last_peered":"2024-09-19T11:25:09.761411+0000","last_clean":"2024-09-19T11:25:09.761411+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:25:09.761411+0000","last_undegraded":"2024-09-19T11:25:09.761411+0000","last_fullsized":"2024-09-19T11:25:09.761411+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:17.846685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280169+0000","last_change":"2024-09-19T11:23:11.266314+0000","last_active":"2024-09-19T11:23:42.280169+0000","last_peered":"2024-09-19T11:23:42.280169+0000","last_clean":"2024-09-19T11:23:42.280169+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:23:42.280169+0000","last_undegraded":"2024-09-19T11:23:42.280169+0000","last_fullsized":"2024-09-19T11:23:42.280169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:28.832170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":163,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.377387+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:26:12.377387+0000","last_peered":"2024-09-19T11:26:12.377387+0000","last_clean":"2024-09-19T11:26:12.377387+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:26:12.377387+0000","last_undegraded":"2024-09-19T11:26:12.377387+0000","last_fullsized":"2024-09-19T11:26:12.377387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:51.720881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'5","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.810212+0000","last_change":"2024-09-19T11:23:05.233784+0000","last_active":"2024-09-19T11:25:16.810212+0000","last_peered":"2024-09-19T11:25:16.810212+0000","last_clean":"2024-09-19T11:25:16.810212+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:25:16.810212+0000","last_undegraded":"2024-09-19T11:25:16.810212+0000","last_fullsized":"2024-09-19T11:25:16.810212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943897+0000","last_change":"2024-09-19T11:22:55.817444+0000","last_active":"2024-09-19T11:23:42.943897+0000","last_peered":"2024-09-19T11:23:42.943897+0000","last_clean":"2024-09-19T11:23:42.943897+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:23:42.943897+0000","last_undegraded":"2024-09-19T11:23:42.943897+0000","last_fullsized":"2024-09-19T11:23:42.943897+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:39.661543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.805962+0000","last_change":"2024-09-19T11:23:03.236740+0000","last_active":"2024-09-19T11:25:10.805962+0000","last_peered":"2024-09-19T11:25:10.805962+0000","last_clean":"2024-09-19T11:25:10.805962+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:25:10.805962+0000","last_undegraded":"2024-09-19T11:25:10.805962+0000","last_fullsized":"2024-09-19T11:25:10.805962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:40.283716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948430+0000","last_change":"2024-09-19T11:23:07.246124+0000","last_active":"2024-09-19T11:23:42.948430+0000","last_peered":"2024-09-19T11:23:42.948430+0000","last_clean":"2024-09-19T11:23:42.948430+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:23:42.948430+0000","last_undegraded":"2024-09-19T11:23:42.948430+0000","last_fullsized":"2024-09-19T11:23:42.948430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'273","reported_seq":477,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327313+0000","last_change":"2024-09-19T11:23:09.257240+0000","last_active":"2024-09-19T11:26:12.327313+0000","last_peered":"2024-09-19T11:26:12.327313+0000","last_clean":"2024-09-19T11:26:12.327313+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:26:12.327313+0000","last_undegraded":"2024-09-19T11:26:12.327313+0000","last_fullsized":"2024-09-19T11:26:12.327313+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:14.514960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944824+0000","last_change":"2024-09-19T11:23:11.270807+0000","last_active":"2024-09-19T11:23:42.944824+0000","last_peered":"2024-09-19T11:23:42.944824+0000","last_clean":"2024-09-19T11:23:42.944824+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:23:42.944824+0000","last_undegraded":"2024-09-19T11:23:42.944824+0000","last_fullsized":"2024-09-19T11:23:42.944824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:23.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.346118+0000","last_change":"2024-09-19T11:23:13.269927+0000","last_active":"2024-09-19T11:24:27.346118+0000","last_peered":"2024-09-19T11:24:27.346118+0000","last_clean":"2024-09-19T11:24:27.346118+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:24:27.346118+0000","last_undegraded":"2024-09-19T11:24:27.346118+0000","last_fullsized":"2024-09-19T11:24:27.346118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:59.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123437+0000","last_change":"2024-09-19T11:23:05.248320+0000","last_active":"2024-09-19T11:26:12.123437+0000","last_peered":"2024-09-19T11:26:12.123437+0000","last_clean":"2024-09-19T11:26:12.123437+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:12.123437+0000","last_undegraded":"2024-09-19T11:26:12.123437+0000","last_fullsized":"2024-09-19T11:26:12.123437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:59.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276942+0000","last_change":"2024-09-19T11:22:55.886390+0000","last_active":"2024-09-19T11:23:42.276942+0000","last_peered":"2024-09-19T11:23:42.276942+0000","last_clean":"2024-09-19T11:23:42.276942+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:23:42.276942+0000","last_undegraded":"2024-09-19T11:23:42.276942+0000","last_fullsized":"2024-09-19T11:23:42.276942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:42.873153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.275364+0000","last_change":"2024-09-19T11:23:03.244094+0000","last_active":"2024-09-19T11:25:11.275364+0000","last_peered":"2024-09-19T11:25:11.275364+0000","last_clean":"2024-09-19T11:25:11.275364+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:25:11.275364+0000","last_undegraded":"2024-09-19T11:25:11.275364+0000","last_fullsized":"2024-09-19T11:25:11.275364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280552+0000","last_change":"2024-09-19T11:23:07.245296+0000","last_active":"2024-09-19T11:23:42.280552+0000","last_peered":"2024-09-19T11:23:42.280552+0000","last_clean":"2024-09-19T11:23:42.280552+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:23:42.280552+0000","last_undegraded":"2024-09-19T11:23:42.280552+0000","last_fullsized":"2024-09-19T11:23:42.280552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:23:05.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'16","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327294+0000","last_change":"2024-09-19T11:23:09.249984+0000","last_active":"2024-09-19T11:26:12.327294+0000","last_peered":"2024-09-19T11:26:12.327294+0000","last_clean":"2024-09-19T11:26:12.327294+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:12.327294+0000","last_undegraded":"2024-09-19T11:26:12.327294+0000","last_fullsized":"2024-09-19T11:26:12.327294+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:47.233637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280391+0000","last_change":"2024-09-19T11:23:11.265633+0000","last_active":"2024-09-19T11:23:42.280391+0000","last_peered":"2024-09-19T11:23:42.280391+0000","last_clean":"2024-09-19T11:23:42.280391+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:23:42.280391+0000","last_undegraded":"2024-09-19T11:23:42.280391+0000","last_fullsized":"2024-09-19T11:23:42.280391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:16.181121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943722+0000","last_change":"2024-09-19T11:23:13.273373+0000","last_active":"2024-09-19T11:23:42.943722+0000","last_peered":"2024-09-19T11:23:42.943722+0000","last_clean":"2024-09-19T11:23:42.943722+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:23:42.943722+0000","last_undegraded":"2024-09-19T11:23:42.943722+0000","last_fullsized":"2024-09-19T11:23:42.943722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:11:13.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368226+0000","last_change":"2024-09-19T11:23:05.238227+0000","last_active":"2024-09-19T11:26:12.368226+0000","last_peered":"2024-09-19T11:26:12.368226+0000","last_clean":"2024-09-19T11:26:12.368226+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:12.368226+0000","last_undegraded":"2024-09-19T11:26:12.368226+0000","last_fullsized":"2024-09-19T11:26:12.368226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:05.469042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944111+0000","last_change":"2024-09-19T11:22:55.817947+0000","last_active":"2024-09-19T11:23:42.944111+0000","last_peered":"2024-09-19T11:23:42.944111+0000","last_clean":"2024-09-19T11:23:42.944111+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:23:42.944111+0000","last_undegraded":"2024-09-19T11:23:42.944111+0000","last_fullsized":"2024-09-19T11:23:42.944111+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:52.495235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.363564+0000","last_change":"2024-09-19T11:23:03.247494+0000","last_active":"2024-09-19T11:26:12.363564+0000","last_peered":"2024-09-19T11:26:12.363564+0000","last_clean":"2024-09-19T11:26:12.363564+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:12.363564+0000","last_undegraded":"2024-09-19T11:26:12.363564+0000","last_fullsized":"2024-09-19T11:26:12.363564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:58.764280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279722+0000","last_change":"2024-09-19T11:23:07.243946+0000","last_active":"2024-09-19T11:23:42.279722+0000","last_peered":"2024-09-19T11:23:42.279722+0000","last_clean":"2024-09-19T11:23:42.279722+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:23:42.279722+0000","last_undegraded":"2024-09-19T11:23:42.279722+0000","last_fullsized":"2024-09-19T11:23:42.279722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:02.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'13","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.530475+0000","last_change":"2024-09-19T11:23:09.254499+0000","last_active":"2024-09-19T11:26:11.530475+0000","last_peered":"2024-09-19T11:26:11.530475+0000","last_clean":"2024-09-19T11:26:11.530475+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:26:11.530475+0000","last_undegraded":"2024-09-19T11:26:11.530475+0000","last_fullsized":"2024-09-19T11:26:11.530475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:18.957107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947826+0000","last_change":"2024-09-19T11:23:11.271073+0000","last_active":"2024-09-19T11:23:42.947826+0000","last_peered":"2024-09-19T11:23:42.947826+0000","last_clean":"2024-09-19T11:23:42.947826+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:23:42.947826+0000","last_undegraded":"2024-09-19T11:23:42.947826+0000","last_fullsized":"2024-09-19T11:23:42.947826+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:52.079937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.088603+0000","last_change":"2024-09-19T11:23:13.273910+0000","last_active":"2024-09-19T11:25:11.088603+0000","last_peered":"2024-09-19T11:25:11.088603+0000","last_clean":"2024-09-19T11:25:11.088603+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:25:11.088603+0000","last_undegraded":"2024-09-19T11:25:11.088603+0000","last_fullsized":"2024-09-19T11:25:11.088603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:02.581618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470157+0000","last_change":"2024-09-19T11:23:05.247999+0000","last_active":"2024-09-19T11:26:12.470157+0000","last_peered":"2024-09-19T11:26:12.470157+0000","last_clean":"2024-09-19T11:26:12.470157+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:12.470157+0000","last_undegraded":"2024-09-19T11:26:12.470157+0000","last_fullsized":"2024-09-19T11:26:12.470157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.575444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277923+0000","last_change":"2024-09-19T11:23:03.251364+0000","last_active":"2024-09-19T11:23:42.277923+0000","last_peered":"2024-09-19T11:23:42.277923+0000","last_clean":"2024-09-19T11:23:42.277923+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:23:42.277923+0000","last_undegraded":"2024-09-19T11:23:42.277923+0000","last_fullsized":"2024-09-19T11:23:42.277923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279906+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.279906+0000","last_peered":"2024-09-19T11:23:42.279906+0000","last_clean":"2024-09-19T11:23:42.279906+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:23:42.279906+0000","last_undegraded":"2024-09-19T11:23:42.279906+0000","last_fullsized":"2024-09-19T11:23:42.279906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:41.618944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:15.806655+0000","last_change":"2024-09-19T11:23:07.247913+0000","last_active":"2024-09-19T11:25:15.806655+0000","last_peered":"2024-09-19T11:25:15.806655+0000","last_clean":"2024-09-19T11:25:15.806655+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:25:15.806655+0000","last_undegraded":"2024-09-19T11:25:15.806655+0000","last_fullsized":"2024-09-19T11:25:15.806655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:03.208183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'62","reported_seq":122,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.805937+0000","last_change":"2024-09-19T11:23:09.257853+0000","last_active":"2024-09-19T11:25:10.805937+0000","last_peered":"2024-09-19T11:25:10.805937+0000","last_clean":"2024-09-19T11:25:10.805937+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:25:10.805937+0000","last_undegraded":"2024-09-19T11:25:10.805937+0000","last_fullsized":"2024-09-19T11:25:10.805937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:02.366161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277438+0000","last_change":"2024-09-19T11:23:11.268600+0000","last_active":"2024-09-19T11:23:42.277438+0000","last_peered":"2024-09-19T11:23:42.277438+0000","last_clean":"2024-09-19T11:23:42.277438+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:23:42.277438+0000","last_undegraded":"2024-09-19T11:23:42.277438+0000","last_fullsized":"2024-09-19T11:23:42.277438+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:50.540232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.422953+0000","last_change":"2024-09-19T11:23:13.267518+0000","last_active":"2024-09-19T11:25:11.422953+0000","last_peered":"2024-09-19T11:25:11.422953+0000","last_clean":"2024-09-19T11:25:11.422953+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:25:11.422953+0000","last_undegraded":"2024-09-19T11:25:11.422953+0000","last_fullsized":"2024-09-19T11:25:11.422953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:35.030653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338573+0000","last_change":"2024-09-19T11:23:05.247369+0000","last_active":"2024-09-19T11:26:12.338573+0000","last_peered":"2024-09-19T11:26:12.338573+0000","last_clean":"2024-09-19T11:26:12.338573+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:12.338573+0000","last_undegraded":"2024-09-19T11:26:12.338573+0000","last_fullsized":"2024-09-19T11:26:12.338573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:05.530837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.806030+0000","last_change":"2024-09-19T11:23:03.236746+0000","last_active":"2024-09-19T11:25:10.806030+0000","last_peered":"2024-09-19T11:25:10.806030+0000","last_clean":"2024-09-19T11:25:10.806030+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:25:10.806030+0000","last_undegraded":"2024-09-19T11:25:10.806030+0000","last_fullsized":"2024-09-19T11:25:10.806030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:23.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945428+0000","last_change":"2024-09-19T11:22:55.818302+0000","last_active":"2024-09-19T11:23:42.945428+0000","last_peered":"2024-09-19T11:23:42.945428+0000","last_clean":"2024-09-19T11:23:42.945428+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:23:42.945428+0000","last_undegraded":"2024-09-19T11:23:42.945428+0000","last_fullsized":"2024-09-19T11:23:42.945428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:51.911098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943947+0000","last_change":"2024-09-19T11:23:07.243184+0000","last_active":"2024-09-19T11:23:42.943947+0000","last_peered":"2024-09-19T11:23:42.943947+0000","last_clean":"2024-09-19T11:23:42.943947+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:23:42.943947+0000","last_undegraded":"2024-09-19T11:23:42.943947+0000","last_fullsized":"2024-09-19T11:23:42.943947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:32:59.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'15","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809866+0000","last_change":"2024-09-19T11:23:09.261039+0000","last_active":"2024-09-19T11:25:16.809866+0000","last_peered":"2024-09-19T11:25:16.809866+0000","last_clean":"2024-09-19T11:25:16.809866+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:25:16.809866+0000","last_undegraded":"2024-09-19T11:25:16.809866+0000","last_fullsized":"2024-09-19T11:25:16.809866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:42.183684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'282","reported_seq":356,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327172+0000","last_change":"2024-09-19T11:23:11.263125+0000","last_active":"2024-09-19T11:26:12.327172+0000","last_peered":"2024-09-19T11:26:12.327172+0000","last_clean":"2024-09-19T11:26:12.327172+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:26:12.327172+0000","last_undegraded":"2024-09-19T11:26:12.327172+0000","last_fullsized":"2024-09-19T11:26:12.327172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:10.328770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946621+0000","last_change":"2024-09-19T11:23:13.269759+0000","last_active":"2024-09-19T11:23:42.946621+0000","last_peered":"2024-09-19T11:23:42.946621+0000","last_clean":"2024-09-19T11:23:42.946621+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:23:42.946621+0000","last_undegraded":"2024-09-19T11:23:42.946621+0000","last_fullsized":"2024-09-19T11:23:42.946621+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:46.820239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'11","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.025445+0000","last_change":"2024-09-19T11:23:05.254850+0000","last_active":"2024-09-19T11:26:12.025445+0000","last_peered":"2024-09-19T11:26:12.025445+0000","last_clean":"2024-09-19T11:26:12.025445+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:12.025445+0000","last_undegraded":"2024-09-19T11:26:12.025445+0000","last_fullsized":"2024-09-19T11:26:12.025445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:44.121192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948269+0000","last_change":"2024-09-19T11:23:03.241499+0000","last_active":"2024-09-19T11:23:42.948269+0000","last_peered":"2024-09-19T11:23:42.948269+0000","last_clean":"2024-09-19T11:23:42.948269+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:23:42.948269+0000","last_undegraded":"2024-09-19T11:23:42.948269+0000","last_fullsized":"2024-09-19T11:23:42.948269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:40.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280554+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.280554+0000","last_peered":"2024-09-19T11:23:42.280554+0000","last_clean":"2024-09-19T11:23:42.280554+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:23:42.280554+0000","last_undegraded":"2024-09-19T11:23:42.280554+0000","last_fullsized":"2024-09-19T11:23:42.280554+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:08:18.682837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946584+0000","last_change":"2024-09-19T11:23:07.245138+0000","last_active":"2024-09-19T11:23:42.946584+0000","last_peered":"2024-09-19T11:23:42.946584+0000","last_clean":"2024-09-19T11:23:42.946584+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:23:42.946584+0000","last_undegraded":"2024-09-19T11:23:42.946584+0000","last_fullsized":"2024-09-19T11:23:42.946584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:31.680535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'11","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945354+0000","last_change":"2024-09-19T11:23:09.256698+0000","last_active":"2024-09-19T11:23:42.945354+0000","last_peered":"2024-09-19T11:23:42.945354+0000","last_clean":"2024-09-19T11:23:42.945354+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:23:42.945354+0000","last_undegraded":"2024-09-19T11:23:42.945354+0000","last_fullsized":"2024-09-19T11:23:42.945354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:21.852554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'282","reported_seq":430,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585906+0000","last_change":"2024-09-19T11:23:11.258279+0000","last_active":"2024-09-19T11:26:12.585906+0000","last_peered":"2024-09-19T11:26:12.585906+0000","last_clean":"2024-09-19T11:26:12.585906+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:26:12.585906+0000","last_undegraded":"2024-09-19T11:26:12.585906+0000","last_fullsized":"2024-09-19T11:26:12.585906+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:24.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281484+0000","last_change":"2024-09-19T11:23:13.274470+0000","last_active":"2024-09-19T11:23:42.281484+0000","last_peered":"2024-09-19T11:23:42.281484+0000","last_clean":"2024-09-19T11:23:42.281484+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:23:42.281484+0000","last_undegraded":"2024-09-19T11:23:42.281484+0000","last_fullsized":"2024-09-19T11:23:42.281484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:33.500152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550363+0000","last_change":"2024-09-19T11:23:05.250199+0000","last_active":"2024-09-19T11:24:27.550363+0000","last_peered":"2024-09-19T11:24:27.550363+0000","last_clean":"2024-09-19T11:24:27.550363+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:24:27.550363+0000","last_undegraded":"2024-09-19T11:24:27.550363+0000","last_fullsized":"2024-09-19T11:24:27.550363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:20.188966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282786+0000","last_change":"2024-09-19T11:23:03.243876+0000","last_active":"2024-09-19T11:23:42.282786+0000","last_peered":"2024-09-19T11:23:42.282786+0000","last_clean":"2024-09-19T11:23:42.282786+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:23:42.282786+0000","last_undegraded":"2024-09-19T11:23:42.282786+0000","last_fullsized":"2024-09-19T11:23:42.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:15.709376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946831+0000","last_change":"2024-09-19T11:22:55.818050+0000","last_active":"2024-09-19T11:23:42.946831+0000","last_peered":"2024-09-19T11:23:42.946831+0000","last_clean":"2024-09-19T11:23:42.946831+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:23:42.946831+0000","last_undegraded":"2024-09-19T11:23:42.946831+0000","last_fullsized":"2024-09-19T11:23:42.946831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.933095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945462+0000","last_change":"2024-09-19T11:23:07.248324+0000","last_active":"2024-09-19T11:23:42.945462+0000","last_peered":"2024-09-19T11:23:42.945462+0000","last_clean":"2024-09-19T11:23:42.945462+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:23:42.945462+0000","last_undegraded":"2024-09-19T11:23:42.945462+0000","last_fullsized":"2024-09-19T11:23:42.945462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.767188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'82","reported_seq":155,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.748098+0000","last_change":"2024-09-19T11:23:09.258292+0000","last_active":"2024-09-19T11:25:09.748098+0000","last_peered":"2024-09-19T11:25:09.748098+0000","last_clean":"2024-09-19T11:25:09.748098+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:25:09.748098+0000","last_undegraded":"2024-09-19T11:25:09.748098+0000","last_fullsized":"2024-09-19T11:25:09.748098+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:51.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280109+0000","last_change":"2024-09-19T11:23:11.263614+0000","last_active":"2024-09-19T11:23:42.280109+0000","last_peered":"2024-09-19T11:23:42.280109+0000","last_clean":"2024-09-19T11:23:42.280109+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:23:42.280109+0000","last_undegraded":"2024-09-19T11:23:42.280109+0000","last_fullsized":"2024-09-19T11:23:42.280109+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.842570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278855+0000","last_change":"2024-09-19T11:23:13.265949+0000","last_active":"2024-09-19T11:23:42.278855+0000","last_peered":"2024-09-19T11:23:42.278855+0000","last_clean":"2024-09-19T11:23:42.278855+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:23:42.278855+0000","last_undegraded":"2024-09-19T11:23:42.278855+0000","last_fullsized":"2024-09-19T11:23:42.278855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:37.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278953+0000","last_change":"2024-09-19T11:23:13.271066+0000","last_active":"2024-09-19T11:23:42.278953+0000","last_peered":"2024-09-19T11:23:42.278953+0000","last_clean":"2024-09-19T11:23:42.278953+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:23:42.278953+0000","last_undegraded":"2024-09-19T11:23:42.278953+0000","last_fullsized":"2024-09-19T11:23:42.278953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:34.307629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277914+0000","last_change":"2024-09-19T11:23:05.245620+0000","last_active":"2024-09-19T11:23:42.277914+0000","last_peered":"2024-09-19T11:23:42.277914+0000","last_clean":"2024-09-19T11:23:42.277914+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:23:42.277914+0000","last_undegraded":"2024-09-19T11:23:42.277914+0000","last_fullsized":"2024-09-19T11:23:42.277914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.990710+0000","last_change":"2024-09-19T11:23:03.254634+0000","last_active":"2024-09-19T11:23:44.990710+0000","last_peered":"2024-09-19T11:23:44.990710+0000","last_clean":"2024-09-19T11:23:44.990710+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:23:44.990710+0000","last_undegraded":"2024-09-19T11:23:44.990710+0000","last_fullsized":"2024-09-19T11:23:44.990710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.416384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282381+0000","last_change":"2024-09-19T11:23:07.240696+0000","last_active":"2024-09-19T11:23:42.282381+0000","last_peered":"2024-09-19T11:23:42.282381+0000","last_clean":"2024-09-19T11:23:42.282381+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:23:42.282381+0000","last_undegraded":"2024-09-19T11:23:42.282381+0000","last_fullsized":"2024-09-19T11:23:42.282381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:02.659138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'32","reported_seq":76,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.754473+0000","last_change":"2024-09-19T11:23:09.259148+0000","last_active":"2024-09-19T11:25:09.754473+0000","last_peered":"2024-09-19T11:25:09.754473+0000","last_clean":"2024-09-19T11:25:09.754473+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:25:09.754473+0000","last_undegraded":"2024-09-19T11:25:09.754473+0000","last_fullsized":"2024-09-19T11:25:09.754473+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:29.000573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945264+0000","last_change":"2024-09-19T11:23:11.271128+0000","last_active":"2024-09-19T11:23:42.945264+0000","last_peered":"2024-09-19T11:23:42.945264+0000","last_clean":"2024-09-19T11:23:42.945264+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:23:42.945264+0000","last_undegraded":"2024-09-19T11:23:42.945264+0000","last_fullsized":"2024-09-19T11:23:42.945264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:35.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.427118+0000","last_change":"2024-09-19T11:23:13.273576+0000","last_active":"2024-09-19T11:25:09.427118+0000","last_peered":"2024-09-19T11:25:09.427118+0000","last_clean":"2024-09-19T11:25:09.427118+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:25:09.427118+0000","last_undegraded":"2024-09-19T11:25:09.427118+0000","last_fullsized":"2024-09-19T11:25:09.427118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:43.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.782138+0000","last_change":"2024-09-19T11:23:42.285785+0000","last_active":"2024-09-19T11:23:44.782138+0000","last_peered":"2024-09-19T11:23:44.782138+0000","last_clean":"2024-09-19T11:23:44.782138+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:23:44.782138+0000","last_undegraded":"2024-09-19T11:23:44.782138+0000","last_fullsized":"2024-09-19T11:23:44.782138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:15.907367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277946+0000","last_change":"2024-09-19T11:23:03.255902+0000","last_active":"2024-09-19T11:23:42.277946+0000","last_peered":"2024-09-19T11:23:42.277946+0000","last_clean":"2024-09-19T11:23:42.277946+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:23:42.277946+0000","last_undegraded":"2024-09-19T11:23:42.277946+0000","last_fullsized":"2024-09-19T11:23:42.277946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:41.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945177+0000","last_change":"2024-09-19T11:23:07.248908+0000","last_active":"2024-09-19T11:23:42.945177+0000","last_peered":"2024-09-19T11:23:42.945177+0000","last_clean":"2024-09-19T11:23:42.945177+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:23:42.945177+0000","last_undegraded":"2024-09-19T11:23:42.945177+0000","last_fullsized":"2024-09-19T11:23:42.945177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:57.835222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'5","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.535761+0000","last_change":"2024-09-19T11:23:09.244959+0000","last_active":"2024-09-19T11:26:11.535761+0000","last_peered":"2024-09-19T11:26:11.535761+0000","last_clean":"2024-09-19T11:26:11.535761+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:26:11.535761+0000","last_undegraded":"2024-09-19T11:26:11.535761+0000","last_fullsized":"2024-09-19T11:26:11.535761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:13.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277093+0000","last_change":"2024-09-19T11:23:11.268038+0000","last_active":"2024-09-19T11:23:42.277093+0000","last_peered":"2024-09-19T11:23:42.277093+0000","last_clean":"2024-09-19T11:23:42.277093+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:23:42.277093+0000","last_undegraded":"2024-09-19T11:23:42.277093+0000","last_fullsized":"2024-09-19T11:23:42.277093+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:40.232592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279462+0000","last_change":"2024-09-19T11:23:13.267238+0000","last_active":"2024-09-19T11:23:42.279462+0000","last_peered":"2024-09-19T11:23:42.279462+0000","last_clean":"2024-09-19T11:23:42.279462+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:23:42.279462+0000","last_undegraded":"2024-09-19T11:23:42.279462+0000","last_fullsized":"2024-09-19T11:23:42.279462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:36.252981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123419+0000","last_change":"2024-09-19T11:23:05.240162+0000","last_active":"2024-09-19T11:26:12.123419+0000","last_peered":"2024-09-19T11:26:12.123419+0000","last_clean":"2024-09-19T11:26:12.123419+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:12.123419+0000","last_undegraded":"2024-09-19T11:26:12.123419+0000","last_fullsized":"2024-09-19T11:26:12.123419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:45.738004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.261555+0000","last_change":"2024-09-19T11:23:03.257719+0000","last_active":"2024-09-19T11:26:12.261555+0000","last_peered":"2024-09-19T11:26:12.261555+0000","last_clean":"2024-09-19T11:26:12.261555+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:12.261555+0000","last_undegraded":"2024-09-19T11:26:12.261555+0000","last_fullsized":"2024-09-19T11:26:12.261555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:46.873725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277308+0000","last_change":"2024-09-19T11:23:07.243467+0000","last_active":"2024-09-19T11:23:42.277308+0000","last_peered":"2024-09-19T11:23:42.277308+0000","last_clean":"2024-09-19T11:23:42.277308+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:23:42.277308+0000","last_undegraded":"2024-09-19T11:23:42.277308+0000","last_fullsized":"2024-09-19T11:23:42.277308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:48.792369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'40","reported_seq":100,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.662362+0000","last_change":"2024-09-19T11:23:09.248715+0000","last_active":"2024-09-19T11:26:11.662362+0000","last_peered":"2024-09-19T11:26:11.662362+0000","last_clean":"2024-09-19T11:26:11.662362+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:26:11.662362+0000","last_undegraded":"2024-09-19T11:26:11.662362+0000","last_fullsized":"2024-09-19T11:26:11.662362+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:51.723342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280357+0000","last_change":"2024-09-19T11:23:11.262690+0000","last_active":"2024-09-19T11:23:42.280357+0000","last_peered":"2024-09-19T11:23:42.280357+0000","last_clean":"2024-09-19T11:23:42.280357+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:23:42.280357+0000","last_undegraded":"2024-09-19T11:23:42.280357+0000","last_fullsized":"2024-09-19T11:23:42.280357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:55.778119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.511055+0000","last_change":"2024-09-19T11:23:05.253736+0000","last_active":"2024-09-19T11:25:11.511055+0000","last_peered":"2024-09-19T11:25:11.511055+0000","last_clean":"2024-09-19T11:25:11.511055+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:25:11.511055+0000","last_undegraded":"2024-09-19T11:25:11.511055+0000","last_fullsized":"2024-09-19T11:25:11.511055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:09.050323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283174+0000","last_change":"2024-09-19T11:23:03.248284+0000","last_active":"2024-09-19T11:23:42.283174+0000","last_peered":"2024-09-19T11:23:42.283174+0000","last_clean":"2024-09-19T11:23:42.283174+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:23:42.283174+0000","last_undegraded":"2024-09-19T11:23:42.283174+0000","last_fullsized":"2024-09-19T11:23:42.283174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:40.686948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944123+0000","last_change":"2024-09-19T11:23:07.245862+0000","last_active":"2024-09-19T11:23:42.944123+0000","last_peered":"2024-09-19T11:23:42.944123+0000","last_clean":"2024-09-19T11:23:42.944123+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:23:42.944123+0000","last_undegraded":"2024-09-19T11:23:42.944123+0000","last_fullsized":"2024-09-19T11:23:42.944123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:09.692396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.739853+0000","last_change":"2024-09-19T11:23:09.247041+0000","last_active":"2024-09-19T11:25:09.739853+0000","last_peered":"2024-09-19T11:25:09.739853+0000","last_clean":"2024-09-19T11:25:09.739853+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:25:09.739853+0000","last_undegraded":"2024-09-19T11:25:09.739853+0000","last_fullsized":"2024-09-19T11:25:09.739853+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:21.727511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277107+0000","last_change":"2024-09-19T11:23:11.253755+0000","last_active":"2024-09-19T11:23:42.277107+0000","last_peered":"2024-09-19T11:23:42.277107+0000","last_clean":"2024-09-19T11:23:42.277107+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:23:42.277107+0000","last_undegraded":"2024-09-19T11:23:42.277107+0000","last_fullsized":"2024-09-19T11:23:42.277107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:51.209331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510416+0000","last_change":"2024-09-19T11:23:05.236615+0000","last_active":"2024-09-19T11:25:11.510416+0000","last_peered":"2024-09-19T11:25:11.510416+0000","last_clean":"2024-09-19T11:25:11.510416+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:25:11.510416+0000","last_undegraded":"2024-09-19T11:25:11.510416+0000","last_fullsized":"2024-09-19T11:25:11.510416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:36.749643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'3","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:31.812710+0000","last_change":"2024-09-19T11:23:03.255452+0000","last_active":"2024-09-19T11:25:31.812710+0000","last_peered":"2024-09-19T11:25:31.812710+0000","last_clean":"2024-09-19T11:25:31.812710+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:25:31.812710+0000","last_undegraded":"2024-09-19T11:25:31.812710+0000","last_fullsized":"2024-09-19T11:25:31.812710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:34.861274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796174+0000","last_change":"2024-09-19T11:23:42.945035+0000","last_active":"2024-09-19T11:23:44.796174+0000","last_peered":"2024-09-19T11:23:44.796174+0000","last_clean":"2024-09-19T11:23:44.796174+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:23:44.796174+0000","last_undegraded":"2024-09-19T11:23:44.796174+0000","last_fullsized":"2024-09-19T11:23:44.796174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'55","reported_seq":112,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:40.810485+0000","last_change":"2024-09-19T11:23:09.246587+0000","last_active":"2024-09-19T11:25:40.810485+0000","last_peered":"2024-09-19T11:25:40.810485+0000","last_clean":"2024-09-19T11:25:40.810485+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:25:40.810485+0000","last_undegraded":"2024-09-19T11:25:40.810485+0000","last_fullsized":"2024-09-19T11:25:40.810485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:53.040162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":42,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278167+0000","last_change":"2024-09-19T11:23:11.260449+0000","last_active":"2024-09-19T11:23:42.278167+0000","last_peered":"2024-09-19T11:23:42.278167+0000","last_clean":"2024-09-19T11:23:42.278167+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:23:42.278167+0000","last_undegraded":"2024-09-19T11:23:42.278167+0000","last_fullsized":"2024-09-19T11:23:42.278167+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:30.983027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.419187+0000","last_change":"2024-09-19T11:23:13.268105+0000","last_active":"2024-09-19T11:25:11.419187+0000","last_peered":"2024-09-19T11:25:11.419187+0000","last_clean":"2024-09-19T11:25:11.419187+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:25:11.419187+0000","last_undegraded":"2024-09-19T11:25:11.419187+0000","last_fullsized":"2024-09-19T11:25:11.419187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:21.113571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368061+0000","last_change":"2024-09-19T11:23:05.246352+0000","last_active":"2024-09-19T11:26:12.368061+0000","last_peered":"2024-09-19T11:26:12.368061+0000","last_clean":"2024-09-19T11:26:12.368061+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:12.368061+0000","last_undegraded":"2024-09-19T11:26:12.368061+0000","last_fullsized":"2024-09-19T11:26:12.368061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.248495+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.959521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279359+0000","last_change":"2024-09-19T11:23:07.243165+0000","last_active":"2024-09-19T11:23:42.279359+0000","last_peered":"2024-09-19T11:23:42.279359+0000","last_clean":"2024-09-19T11:23:42.279359+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:23:42.279359+0000","last_undegraded":"2024-09-19T11:23:42.279359+0000","last_fullsized":"2024-09-19T11:23:42.279359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:21.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.740822+0000","last_change":"2024-09-19T11:23:09.256642+0000","last_active":"2024-09-19T11:25:09.740822+0000","last_peered":"2024-09-19T11:25:09.740822+0000","last_clean":"2024-09-19T11:25:09.740822+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:25:09.740822+0000","last_undegraded":"2024-09-19T11:25:09.740822+0000","last_fullsized":"2024-09-19T11:25:09.740822+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:42.797325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282300+0000","last_change":"2024-09-19T11:23:11.266817+0000","last_active":"2024-09-19T11:23:42.282300+0000","last_peered":"2024-09-19T11:23:42.282300+0000","last_clean":"2024-09-19T11:23:42.282300+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:23:42.282300+0000","last_undegraded":"2024-09-19T11:23:42.282300+0000","last_fullsized":"2024-09-19T11:23:42.282300+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:41:28.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947653+0000","last_change":"2024-09-19T11:23:05.254589+0000","last_active":"2024-09-19T11:23:42.947653+0000","last_peered":"2024-09-19T11:23:42.947653+0000","last_clean":"2024-09-19T11:23:42.947653+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:23:42.947653+0000","last_undegraded":"2024-09-19T11:23:42.947653+0000","last_fullsized":"2024-09-19T11:23:42.947653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:11.078441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280548+0000","last_change":"2024-09-19T11:23:03.244404+0000","last_active":"2024-09-19T11:23:42.280548+0000","last_peered":"2024-09-19T11:23:42.280548+0000","last_clean":"2024-09-19T11:23:42.280548+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:23:42.280548+0000","last_undegraded":"2024-09-19T11:23:42.280548+0000","last_fullsized":"2024-09-19T11:23:42.280548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:07.948011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278448+0000","last_change":"2024-09-19T11:23:07.242009+0000","last_active":"2024-09-19T11:23:42.278448+0000","last_peered":"2024-09-19T11:23:42.278448+0000","last_clean":"2024-09-19T11:23:42.278448+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:23:42.278448+0000","last_undegraded":"2024-09-19T11:23:42.278448+0000","last_fullsized":"2024-09-19T11:23:42.278448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:37.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'34","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.758462+0000","last_change":"2024-09-19T11:23:09.258190+0000","last_active":"2024-09-19T11:25:09.758462+0000","last_peered":"2024-09-19T11:25:09.758462+0000","last_clean":"2024-09-19T11:25:09.758462+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:25:09.758462+0000","last_undegraded":"2024-09-19T11:25:09.758462+0000","last_fullsized":"2024-09-19T11:25:09.758462+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:31.989630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":46,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280329+0000","last_change":"2024-09-19T11:23:11.261698+0000","last_active":"2024-09-19T11:23:42.280329+0000","last_peered":"2024-09-19T11:23:42.280329+0000","last_clean":"2024-09-19T11:23:42.280329+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:23:42.280329+0000","last_undegraded":"2024-09-19T11:23:42.280329+0000","last_fullsized":"2024-09-19T11:23:42.280329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:48.537146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032904+0000","last_change":"2024-09-19T11:23:05.238234+0000","last_active":"2024-09-19T11:25:08.032904+0000","last_peered":"2024-09-19T11:25:08.032904+0000","last_clean":"2024-09-19T11:25:08.032904+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:25:08.032904+0000","last_undegraded":"2024-09-19T11:25:08.032904+0000","last_fullsized":"2024-09-19T11:25:08.032904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:03:26.967357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944417+0000","last_change":"2024-09-19T11:23:03.253512+0000","last_active":"2024-09-19T11:23:42.944417+0000","last_peered":"2024-09-19T11:23:42.944417+0000","last_clean":"2024-09-19T11:23:42.944417+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:23:42.944417+0000","last_undegraded":"2024-09-19T11:23:42.944417+0000","last_fullsized":"2024-09-19T11:23:42.944417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:17.009054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.286788+0000","last_change":"2024-09-19T11:23:42.286200+0000","last_active":"2024-09-19T11:23:42.286788+0000","last_peered":"2024-09-19T11:23:42.286788+0000","last_clean":"2024-09-19T11:23:42.286788+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:23:42.286788+0000","last_undegraded":"2024-09-19T11:23:42.286788+0000","last_fullsized":"2024-09-19T11:23:42.286788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.397616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946537+0000","last_change":"2024-09-19T11:23:09.257194+0000","last_active":"2024-09-19T11:23:42.946537+0000","last_peered":"2024-09-19T11:23:42.946537+0000","last_clean":"2024-09-19T11:23:42.946537+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:23:42.946537+0000","last_undegraded":"2024-09-19T11:23:42.946537+0000","last_fullsized":"2024-09-19T11:23:42.946537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:10.690183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'282","reported_seq":418,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.372744+0000","last_change":"2024-09-19T11:23:11.268055+0000","last_active":"2024-09-19T11:26:12.372744+0000","last_peered":"2024-09-19T11:26:12.372744+0000","last_clean":"2024-09-19T11:26:12.372744+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:26:12.372744+0000","last_undegraded":"2024-09-19T11:26:12.372744+0000","last_fullsized":"2024-09-19T11:26:12.372744+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:33.997140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943313+0000","last_change":"2024-09-19T11:23:13.272264+0000","last_active":"2024-09-19T11:23:42.943313+0000","last_peered":"2024-09-19T11:23:42.943313+0000","last_clean":"2024-09-19T11:23:42.943313+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:23:42.943313+0000","last_undegraded":"2024-09-19T11:23:42.943313+0000","last_fullsized":"2024-09-19T11:23:42.943313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:34.821219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123209+0000","last_change":"2024-09-19T11:23:05.235764+0000","last_active":"2024-09-19T11:26:12.123209+0000","last_peered":"2024-09-19T11:26:12.123209+0000","last_clean":"2024-09-19T11:26:12.123209+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:12.123209+0000","last_undegraded":"2024-09-19T11:26:12.123209+0000","last_fullsized":"2024-09-19T11:26:12.123209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:41.326878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282378+0000","last_change":"2024-09-19T11:23:03.245357+0000","last_active":"2024-09-19T11:23:42.282378+0000","last_peered":"2024-09-19T11:23:42.282378+0000","last_clean":"2024-09-19T11:23:42.282378+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:23:42.282378+0000","last_undegraded":"2024-09-19T11:23:42.282378+0000","last_fullsized":"2024-09-19T11:23:42.282378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:23.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281352+0000","last_change":"2024-09-19T11:23:07.247890+0000","last_active":"2024-09-19T11:23:42.281352+0000","last_peered":"2024-09-19T11:23:42.281352+0000","last_clean":"2024-09-19T11:23:42.281352+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:23:42.281352+0000","last_undegraded":"2024-09-19T11:23:42.281352+0000","last_fullsized":"2024-09-19T11:23:42.281352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.412033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.731864+0000","last_change":"2024-09-19T11:23:09.246755+0000","last_active":"2024-09-19T11:25:09.731864+0000","last_peered":"2024-09-19T11:25:09.731864+0000","last_clean":"2024-09-19T11:25:09.731864+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:25:09.731864+0000","last_undegraded":"2024-09-19T11:25:09.731864+0000","last_fullsized":"2024-09-19T11:25:09.731864+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.229678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278129+0000","last_change":"2024-09-19T11:23:11.266713+0000","last_active":"2024-09-19T11:23:42.278129+0000","last_peered":"2024-09-19T11:23:42.278129+0000","last_clean":"2024-09-19T11:23:42.278129+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:23:42.278129+0000","last_undegraded":"2024-09-19T11:23:42.278129+0000","last_fullsized":"2024-09-19T11:23:42.278129+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:38.162577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283516+0000","last_change":"2024-09-19T11:23:13.268771+0000","last_active":"2024-09-19T11:23:42.283516+0000","last_peered":"2024-09-19T11:23:42.283516+0000","last_clean":"2024-09-19T11:23:42.283516+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:23:42.283516+0000","last_undegraded":"2024-09-19T11:23:42.283516+0000","last_fullsized":"2024-09-19T11:23:42.283516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:19.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266078+0000","last_change":"2024-09-19T11:23:05.236610+0000","last_active":"2024-09-19T11:26:12.266078+0000","last_peered":"2024-09-19T11:26:12.266078+0000","last_clean":"2024-09-19T11:26:12.266078+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:12.266078+0000","last_undegraded":"2024-09-19T11:26:12.266078+0000","last_fullsized":"2024-09-19T11:26:12.266078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:44.218259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946253+0000","last_change":"2024-09-19T11:23:03.258006+0000","last_active":"2024-09-19T11:23:42.946253+0000","last_peered":"2024-09-19T11:23:42.946253+0000","last_clean":"2024-09-19T11:23:42.946253+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:23:42.946253+0000","last_undegraded":"2024-09-19T11:23:42.946253+0000","last_fullsized":"2024-09-19T11:23:42.946253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:55.934419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278151+0000","last_change":"2024-09-19T11:23:07.247875+0000","last_active":"2024-09-19T11:23:42.278151+0000","last_peered":"2024-09-19T11:23:42.278151+0000","last_clean":"2024-09-19T11:23:42.278151+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:23:42.278151+0000","last_undegraded":"2024-09-19T11:23:42.278151+0000","last_fullsized":"2024-09-19T11:23:42.278151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:20.420374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.732852+0000","last_change":"2024-09-19T11:23:09.257850+0000","last_active":"2024-09-19T11:25:09.732852+0000","last_peered":"2024-09-19T11:25:09.732852+0000","last_clean":"2024-09-19T11:25:09.732852+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:25:09.732852+0000","last_undegraded":"2024-09-19T11:25:09.732852+0000","last_fullsized":"2024-09-19T11:25:09.732852+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:48.134834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943735+0000","last_change":"2024-09-19T11:23:11.271462+0000","last_active":"2024-09-19T11:23:42.943735+0000","last_peered":"2024-09-19T11:23:42.943735+0000","last_clean":"2024-09-19T11:23:42.943735+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:23:42.943735+0000","last_undegraded":"2024-09-19T11:23:42.943735+0000","last_fullsized":"2024-09-19T11:23:42.943735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.194246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368608+0000","last_change":"2024-09-19T11:23:42.282153+0000","last_active":"2024-09-19T11:26:12.368608+0000","last_peered":"2024-09-19T11:26:12.368608+0000","last_clean":"2024-09-19T11:26:12.368608+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:12.368608+0000","last_undegraded":"2024-09-19T11:26:12.368608+0000","last_fullsized":"2024-09-19T11:26:12.368608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.828536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.999397+0000","last_change":"2024-09-19T11:23:03.237858+0000","last_active":"2024-09-19T11:23:44.999397+0000","last_peered":"2024-09-19T11:23:44.999397+0000","last_clean":"2024-09-19T11:23:44.999397+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:23:44.999397+0000","last_undegraded":"2024-09-19T11:23:44.999397+0000","last_fullsized":"2024-09-19T11:23:44.999397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:38.094142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280358+0000","last_change":"2024-09-19T11:23:07.245080+0000","last_active":"2024-09-19T11:23:42.280358+0000","last_peered":"2024-09-19T11:23:42.280358+0000","last_clean":"2024-09-19T11:23:42.280358+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:23:42.280358+0000","last_undegraded":"2024-09-19T11:23:42.280358+0000","last_fullsized":"2024-09-19T11:23:42.280358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:41.815514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'26","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.756320+0000","last_change":"2024-09-19T11:23:09.258685+0000","last_active":"2024-09-19T11:25:09.756320+0000","last_peered":"2024-09-19T11:25:09.756320+0000","last_clean":"2024-09-19T11:25:09.756320+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:25:09.756320+0000","last_undegraded":"2024-09-19T11:25:09.756320+0000","last_fullsized":"2024-09-19T11:25:09.756320+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:33.508526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944317+0000","last_change":"2024-09-19T11:23:11.267904+0000","last_active":"2024-09-19T11:23:42.944317+0000","last_peered":"2024-09-19T11:23:42.944317+0000","last_clean":"2024-09-19T11:23:42.944317+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:23:42.944317+0000","last_undegraded":"2024-09-19T11:23:42.944317+0000","last_fullsized":"2024-09-19T11:23:42.944317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:16.874318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032486+0000","last_change":"2024-09-19T11:23:05.235573+0000","last_active":"2024-09-19T11:25:08.032486+0000","last_peered":"2024-09-19T11:25:08.032486+0000","last_clean":"2024-09-19T11:25:08.032486+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:25:08.032486+0000","last_undegraded":"2024-09-19T11:25:08.032486+0000","last_fullsized":"2024-09-19T11:25:08.032486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:07.062938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":46,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.996626+0000","last_change":"2024-09-19T11:23:03.249699+0000","last_active":"2024-09-19T11:23:44.996626+0000","last_peered":"2024-09-19T11:23:44.996626+0000","last_clean":"2024-09-19T11:23:44.996626+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:23:44.996626+0000","last_undegraded":"2024-09-19T11:23:44.996626+0000","last_fullsized":"2024-09-19T11:23:44.996626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:36.434981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'2","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.365932+0000","last_change":"2024-09-19T11:23:07.247476+0000","last_active":"2024-09-19T11:25:08.365932+0000","last_peered":"2024-09-19T11:25:08.365932+0000","last_clean":"2024-09-19T11:25:08.365932+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:25:08.365932+0000","last_undegraded":"2024-09-19T11:25:08.365932+0000","last_fullsized":"2024-09-19T11:25:08.365932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:03.848364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'34","reported_seq":79,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.561308+0000","last_change":"2024-09-19T11:23:09.255745+0000","last_active":"2024-09-19T11:26:11.561308+0000","last_peered":"2024-09-19T11:26:11.561308+0000","last_clean":"2024-09-19T11:26:11.561308+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:26:11.561308+0000","last_undegraded":"2024-09-19T11:26:11.561308+0000","last_fullsized":"2024-09-19T11:26:11.561308+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:35.538204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279071+0000","last_change":"2024-09-19T11:23:11.266117+0000","last_active":"2024-09-19T11:23:42.279071+0000","last_peered":"2024-09-19T11:23:42.279071+0000","last_clean":"2024-09-19T11:23:42.279071+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:23:42.279071+0000","last_undegraded":"2024-09-19T11:23:42.279071+0000","last_fullsized":"2024-09-19T11:23:42.279071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:22.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550414+0000","last_change":"2024-09-19T11:23:05.252243+0000","last_active":"2024-09-19T11:24:27.550414+0000","last_peered":"2024-09-19T11:24:27.550414+0000","last_clean":"2024-09-19T11:24:27.550414+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:24:27.550414+0000","last_undegraded":"2024-09-19T11:24:27.550414+0000","last_fullsized":"2024-09-19T11:24:27.550414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:18.179391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276311+0000","last_change":"2024-09-19T11:23:03.257178+0000","last_active":"2024-09-19T11:23:42.276311+0000","last_peered":"2024-09-19T11:23:42.276311+0000","last_clean":"2024-09-19T11:23:42.276311+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:23:42.276311+0000","last_undegraded":"2024-09-19T11:23:42.276311+0000","last_fullsized":"2024-09-19T11:23:42.276311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:13.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.024977+0000","last_change":"2024-09-19T11:23:05.238242+0000","last_active":"2024-09-19T11:26:12.024977+0000","last_peered":"2024-09-19T11:26:12.024977+0000","last_clean":"2024-09-19T11:26:12.024977+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:12.024977+0000","last_undegraded":"2024-09-19T11:26:12.024977+0000","last_fullsized":"2024-09-19T11:26:12.024977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278230+0000","last_change":"2024-09-19T11:23:03.238163+0000","last_active":"2024-09-19T11:23:42.278230+0000","last_peered":"2024-09-19T11:23:42.278230+0000","last_clean":"2024-09-19T11:23:42.278230+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:23:42.278230+0000","last_undegraded":"2024-09-19T11:23:42.278230+0000","last_fullsized":"2024-09-19T11:23:42.278230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:49:17.258919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342000+0000","last_change":"2024-09-19T11:23:05.237372+0000","last_active":"2024-09-19T11:24:27.342000+0000","last_peered":"2024-09-19T11:24:27.342000+0000","last_clean":"2024-09-19T11:24:27.342000+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:24:27.342000+0000","last_undegraded":"2024-09-19T11:24:27.342000+0000","last_fullsized":"2024-09-19T11:24:27.342000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:27.975988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940912+0000","last_change":"2024-09-19T11:23:03.256664+0000","last_active":"2024-09-19T11:23:43.940912+0000","last_peered":"2024-09-19T11:23:43.940912+0000","last_clean":"2024-09-19T11:23:43.940912+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:23:43.940912+0000","last_undegraded":"2024-09-19T11:23:43.940912+0000","last_fullsized":"2024-09-19T11:23:43.940912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:24.307449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.011182+0000","last_change":"2024-09-19T11:23:05.237761+0000","last_active":"2024-09-19T11:23:45.011182+0000","last_peered":"2024-09-19T11:23:45.011182+0000","last_clean":"2024-09-19T11:23:45.011182+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:23:45.011182+0000","last_undegraded":"2024-09-19T11:23:45.011182+0000","last_fullsized":"2024-09-19T11:23:45.011182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:29:24.776293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281499+0000","last_change":"2024-09-19T11:23:03.243433+0000","last_active":"2024-09-19T11:23:42.281499+0000","last_peered":"2024-09-19T11:23:42.281499+0000","last_clean":"2024-09-19T11:23:42.281499+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:23:42.281499+0000","last_undegraded":"2024-09-19T11:23:42.281499+0000","last_fullsized":"2024-09-19T11:23:42.281499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:45.982872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338525+0000","last_change":"2024-09-19T11:23:05.240645+0000","last_active":"2024-09-19T11:26:12.338525+0000","last_peered":"2024-09-19T11:26:12.338525+0000","last_clean":"2024-09-19T11:26:12.338525+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:12.338525+0000","last_undegraded":"2024-09-19T11:26:12.338525+0000","last_fullsized":"2024-09-19T11:26:12.338525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.997259+0000","last_change":"2024-09-19T11:23:03.234311+0000","last_active":"2024-09-19T11:26:11.997259+0000","last_peered":"2024-09-19T11:26:11.997259+0000","last_clean":"2024-09-19T11:26:11.997259+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:11.997259+0000","last_undegraded":"2024-09-19T11:26:11.997259+0000","last_fullsized":"2024-09-19T11:26:11.997259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:10.195241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342109+0000","last_change":"2024-09-19T11:23:05.251796+0000","last_active":"2024-09-19T11:24:27.342109+0000","last_peered":"2024-09-19T11:24:27.342109+0000","last_clean":"2024-09-19T11:24:27.342109+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:24:27.342109+0000","last_undegraded":"2024-09-19T11:24:27.342109+0000","last_fullsized":"2024-09-19T11:24:27.342109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:53.953637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281786+0000","last_change":"2024-09-19T11:23:03.248461+0000","last_active":"2024-09-19T11:23:42.281786+0000","last_peered":"2024-09-19T11:23:42.281786+0000","last_clean":"2024-09-19T11:23:42.281786+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:23:42.281786+0000","last_undegraded":"2024-09-19T11:23:42.281786+0000","last_fullsized":"2024-09-19T11:23:42.281786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:40.877834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279051+0000","last_change":"2024-09-19T11:23:05.248747+0000","last_active":"2024-09-19T11:23:42.279051+0000","last_peered":"2024-09-19T11:23:42.279051+0000","last_clean":"2024-09-19T11:23:42.279051+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:23:42.279051+0000","last_undegraded":"2024-09-19T11:23:42.279051+0000","last_fullsized":"2024-09-19T11:23:42.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.258514+0000","last_change":"2024-09-19T11:23:03.257927+0000","last_active":"2024-09-19T11:26:12.258514+0000","last_peered":"2024-09-19T11:26:12.258514+0000","last_clean":"2024-09-19T11:26:12.258514+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:12.258514+0000","last_undegraded":"2024-09-19T11:26:12.258514+0000","last_fullsized":"2024-09-19T11:26:12.258514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374459+0000","last_change":"2024-09-19T11:23:05.248818+0000","last_active":"2024-09-19T11:26:12.374459+0000","last_peered":"2024-09-19T11:26:12.374459+0000","last_clean":"2024-09-19T11:26:12.374459+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:12.374459+0000","last_undegraded":"2024-09-19T11:26:12.374459+0000","last_fullsized":"2024-09-19T11:26:12.374459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:41.343464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280250+0000","last_change":"2024-09-19T11:23:03.238591+0000","last_active":"2024-09-19T11:23:42.280250+0000","last_peered":"2024-09-19T11:23:42.280250+0000","last_clean":"2024-09-19T11:23:42.280250+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:23:42.280250+0000","last_undegraded":"2024-09-19T11:23:42.280250+0000","last_fullsized":"2024-09-19T11:23:42.280250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:42.869904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342282+0000","last_change":"2024-09-19T11:23:05.248696+0000","last_active":"2024-09-19T11:24:27.342282+0000","last_peered":"2024-09-19T11:24:27.342282+0000","last_clean":"2024-09-19T11:24:27.342282+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:24:27.342282+0000","last_undegraded":"2024-09-19T11:24:27.342282+0000","last_fullsized":"2024-09-19T11:24:27.342282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:37:55.133352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944801+0000","last_change":"2024-09-19T11:23:03.255783+0000","last_active":"2024-09-19T11:23:42.944801+0000","last_peered":"2024-09-19T11:23:42.944801+0000","last_clean":"2024-09-19T11:23:42.944801+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:23:42.944801+0000","last_undegraded":"2024-09-19T11:23:42.944801+0000","last_fullsized":"2024-09-19T11:23:42.944801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:24.578577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368100+0000","last_change":"2024-09-19T11:23:05.243797+0000","last_active":"2024-09-19T11:26:12.368100+0000","last_peered":"2024-09-19T11:26:12.368100+0000","last_clean":"2024-09-19T11:26:12.368100+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:26:12.368100+0000","last_undegraded":"2024-09-19T11:26:12.368100+0000","last_fullsized":"2024-09-19T11:26:12.368100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:33.524342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278358+0000","last_change":"2024-09-19T11:23:03.243321+0000","last_active":"2024-09-19T11:23:42.278358+0000","last_peered":"2024-09-19T11:23:42.278358+0000","last_clean":"2024-09-19T11:23:42.278358+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:23:42.278358+0000","last_undegraded":"2024-09-19T11:23:42.278358+0000","last_fullsized":"2024-09-19T11:23:42.278358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342015+0000","last_change":"2024-09-19T11:23:05.241366+0000","last_active":"2024-09-19T11:24:27.342015+0000","last_peered":"2024-09-19T11:24:27.342015+0000","last_clean":"2024-09-19T11:24:27.342015+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:24:27.342015+0000","last_undegraded":"2024-09-19T11:24:27.342015+0000","last_fullsized":"2024-09-19T11:24:27.342015+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.002195+0000","last_change":"2024-09-19T11:23:03.249920+0000","last_active":"2024-09-19T11:23:45.002195+0000","last_peered":"2024-09-19T11:23:45.002195+0000","last_clean":"2024-09-19T11:23:45.002195+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:23:45.002195+0000","last_undegraded":"2024-09-19T11:23:45.002195+0000","last_fullsized":"2024-09-19T11:23:45.002195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:45.629786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.482864+0000","last_change":"2024-09-19T11:23:05.246852+0000","last_active":"2024-09-19T11:23:45.482864+0000","last_peered":"2024-09-19T11:23:45.482864+0000","last_clean":"2024-09-19T11:23:45.482864+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:23:45.482864+0000","last_undegraded":"2024-09-19T11:23:45.482864+0000","last_fullsized":"2024-09-19T11:23:45.482864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.057981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278908+0000","last_change":"2024-09-19T11:23:03.243727+0000","last_active":"2024-09-19T11:23:42.278908+0000","last_peered":"2024-09-19T11:23:42.278908+0000","last_clean":"2024-09-19T11:23:42.278908+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:23:42.278908+0000","last_undegraded":"2024-09-19T11:23:42.278908+0000","last_fullsized":"2024-09-19T11:23:42.278908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:52:42.605160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433129+0000","last_change":"2024-09-19T11:23:05.245754+0000","last_active":"2024-09-19T11:23:44.433129+0000","last_peered":"2024-09-19T11:23:44.433129+0000","last_clean":"2024-09-19T11:23:44.433129+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:23:44.433129+0000","last_undegraded":"2024-09-19T11:23:44.433129+0000","last_fullsized":"2024-09-19T11:23:44.433129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:22.898396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:03.241382+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:01.554767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266030+0000","last_change":"2024-09-19T11:23:05.248307+0000","last_active":"2024-09-19T11:26:12.266030+0000","last_peered":"2024-09-19T11:26:12.266030+0000","last_clean":"2024-09-19T11:26:12.266030+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:12.266030+0000","last_undegraded":"2024-09-19T11:26:12.266030+0000","last_fullsized":"2024-09-19T11:26:12.266030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:44.091824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946371+0000","last_change":"2024-09-19T11:23:03.256127+0000","last_active":"2024-09-19T11:23:42.946371+0000","last_peered":"2024-09-19T11:23:42.946371+0000","last_clean":"2024-09-19T11:23:42.946371+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:23:42.946371+0000","last_undegraded":"2024-09-19T11:23:42.946371+0000","last_fullsized":"2024-09-19T11:23:42.946371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:03.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123790+0000","last_change":"2024-09-19T11:23:05.250503+0000","last_active":"2024-09-19T11:26:12.123790+0000","last_peered":"2024-09-19T11:26:12.123790+0000","last_clean":"2024-09-19T11:26:12.123790+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:26:12.123790+0000","last_undegraded":"2024-09-19T11:26:12.123790+0000","last_fullsized":"2024-09-19T11:26:12.123790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:46.547197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.993579+0000","last_change":"2024-09-19T11:23:03.259032+0000","last_active":"2024-09-19T11:23:44.993579+0000","last_peered":"2024-09-19T11:23:44.993579+0000","last_clean":"2024-09-19T11:23:44.993579+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:23:44.993579+0000","last_undegraded":"2024-09-19T11:23:44.993579+0000","last_fullsized":"2024-09-19T11:23:44.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:06.246922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368312+0000","last_change":"2024-09-19T11:23:05.248701+0000","last_active":"2024-09-19T11:26:12.368312+0000","last_peered":"2024-09-19T11:26:12.368312+0000","last_clean":"2024-09-19T11:26:12.368312+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:12.368312+0000","last_undegraded":"2024-09-19T11:26:12.368312+0000","last_fullsized":"2024-09-19T11:26:12.368312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:20.161230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947585+0000","last_change":"2024-09-19T11:23:03.252640+0000","last_active":"2024-09-19T11:23:42.947585+0000","last_peered":"2024-09-19T11:23:42.947585+0000","last_clean":"2024-09-19T11:23:42.947585+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:23:42.947585+0000","last_undegraded":"2024-09-19T11:23:42.947585+0000","last_fullsized":"2024-09-19T11:23:42.947585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:25.619002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811813+0000","last_change":"2024-09-19T11:23:03.256515+0000","last_active":"2024-09-19T11:25:26.811813+0000","last_peered":"2024-09-19T11:25:26.811813+0000","last_clean":"2024-09-19T11:25:26.811813+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:25:26.811813+0000","last_undegraded":"2024-09-19T11:25:26.811813+0000","last_fullsized":"2024-09-19T11:25:26.811813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:08.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123873+0000","last_change":"2024-09-19T11:23:05.248691+0000","last_active":"2024-09-19T11:26:12.123873+0000","last_peered":"2024-09-19T11:26:12.123873+0000","last_clean":"2024-09-19T11:26:12.123873+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:12.123873+0000","last_undegraded":"2024-09-19T11:26:12.123873+0000","last_fullsized":"2024-09-19T11:26:12.123873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:20.035436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278475+0000","last_change":"2024-09-19T11:23:03.243676+0000","last_active":"2024-09-19T11:23:42.278475+0000","last_peered":"2024-09-19T11:23:42.278475+0000","last_clean":"2024-09-19T11:23:42.278475+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:23:42.278475+0000","last_undegraded":"2024-09-19T11:23:42.278475+0000","last_fullsized":"2024-09-19T11:23:42.278475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:37.230042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510844+0000","last_change":"2024-09-19T11:23:05.245906+0000","last_active":"2024-09-19T11:25:11.510844+0000","last_peered":"2024-09-19T11:25:11.510844+0000","last_clean":"2024-09-19T11:25:11.510844+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:25:11.510844+0000","last_undegraded":"2024-09-19T11:25:11.510844+0000","last_fullsized":"2024-09-19T11:25:11.510844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:19.236446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":182,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":38719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673005,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40096,"kb_used_data":13072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":13385728,"data_stored":13057068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673005,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":5408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":5537792,"data_stored":5255453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40300,"kb_used_data":13276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331540,"statfs":{"total":96636764160,"available":96595496960,"internally_reserved":0,"allocated":13594624,"data_stored":13323694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40128,"kb_used_data":8944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331712,"statfs":{"total":96636764160,"available":96595673088,"internally_reserved":0,"allocated":9158656,"data_stored":8862463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4517888,"data_stored":4517888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12906496,"data_stored":12906496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:26:19.966 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T11:26:19.966 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:26:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd96d9b640 1 Processor -- start 2024-09-19T11:26:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd96d9b640 1 -- start start 2024-09-19T11:26:20.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd96d9b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efd9012cba0 0x7efd901315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd96d9b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd9009a290 con 0x7efd9012c0b0 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd96d9b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7efd90071860 con 0x7efd9012cba0 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd8ffff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7efd9012cba0 0x7efd901315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd94b37640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7efd9012c0b0 0x7efd9012c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41952/0 (socket says 172.21.15.22:41952) 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.087+0000 7efd94b37640 1 -- 172.21.15.22:0/1013433392 learned_addr learned my addr 172.21.15.22:0/1013433392 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.088+0000 7efd8ffff640 1 -- 172.21.15.22:0/1013433392 >> v1:172.21.15.22:6789/0 conn(0x7efd9012c0b0 legacy=0x7efd9012c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.088+0000 7efd8ffff640 1 -- 172.21.15.22:0/1013433392 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd90099e20 con 0x7efd9012cba0 2024-09-19T11:26:20.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.088+0000 7efd8f7fe640 1 -- 172.21.15.22:0/1013433392 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4061212678 0 0) 0x7efd9009a290 con 0x7efd9012c0b0 2024-09-19T11:26:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.088+0000 7efd8ffff640 1 --2- 172.21.15.22:0/1013433392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012cba0 0x7efd901315e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009750 tx=0x7efd7c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6954c910556702e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.088+0000 7efd8f7fe640 1 -- 172.21.15.22:0/1013433392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd7c037070 con 0x7efd9012cba0 2024-09-19T11:26:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd8f7fe640 1 -- 172.21.15.22:0/1013433392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c02ad80 con 0x7efd9012cba0 2024-09-19T11:26:20.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd8f7fe640 1 -- 172.21.15.22:0/1013433392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd7c032590 con 0x7efd9012cba0 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd96d9b640 1 -- 172.21.15.22:0/1013433392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012cba0 msgr2=0x7efd901315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd96d9b640 1 --2- 172.21.15.22:0/1013433392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012cba0 0x7efd901315e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009750 tx=0x7efd7c02a140 comp rx=0 tx=0).stop 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd96d9b640 1 -- 172.21.15.22:0/1013433392 shutdown_connections 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd96d9b640 1 --2- 172.21.15.22:0/1013433392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012cba0 0x7efd901315e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd96d9b640 1 -- 172.21.15.22:0/1013433392 >> 172.21.15.22:0/1013433392 conn(0x7efd9009abc0 msgr2=0x7efd90098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd96d9b640 1 -- 172.21.15.22:0/1013433392 shutdown_connections 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.089+0000 7efd96d9b640 1 -- 172.21.15.22:0/1013433392 wait complete. 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.090+0000 7efd96d9b640 1 Processor -- start 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.090+0000 7efd96d9b640 1 -- start start 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.090+0000 7efd96d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012c0b0 0x7efd9012f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.090+0000 7efd96d9b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efd90070c40 con 0x7efd9012c0b0 2024-09-19T11:26:20.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd94b37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012c0b0 0x7efd9012f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd94b37640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012c0b0 0x7efd9012f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35110/0 (socket says 172.21.15.22:35110) 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd94b37640 1 -- 172.21.15.22:0/1900345895 learned_addr learned my addr 172.21.15.22:0/1900345895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd94b37640 1 -- 172.21.15.22:0/1900345895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9008d440 con 0x7efd9012c0b0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd94b37640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012c0b0 0x7efd9012f870 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c3d0 tx=0x7efd8400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd8d7fa640 1 -- 172.21.15.22:0/1900345895 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd840074d0 con 0x7efd9012c0b0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd8d7fa640 1 -- 172.21.15.22:0/1900345895 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd840047e0 con 0x7efd9012c0b0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd8d7fa640 1 -- 172.21.15.22:0/1900345895 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd84007670 con 0x7efd9012c0b0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd90074090 con 0x7efd9012c0b0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.091+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9008d770 con 0x7efd9012c0b0 2024-09-19T11:26:20.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.092+0000 7efd8d7fa640 1 -- 172.21.15.22:0/1900345895 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7efd84011070 con 0x7efd9012c0b0 2024-09-19T11:26:20.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.092+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9012c490 con 0x7efd9012c0b0 2024-09-19T11:26:20.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.092+0000 7efd8d7fa640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efd7403e710 0x7efd74040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:20.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.092+0000 7efd8d7fa640 1 -- 172.21.15.22:0/1900345895 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7efd84053410 con 0x7efd9012c0b0 2024-09-19T11:26:20.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.093+0000 7efd8ffff640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efd7403e710 0x7efd74040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:20.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.093+0000 7efd8ffff640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efd7403e710 0x7efd74040bd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efd7c034000 tx=0x7efd7c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:20.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.097+0000 7efd8d7fa640 1 -- 172.21.15.22:0/1900345895 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7efd9008d440 con 0x7efd9012c0b0 2024-09-19T11:26:20.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.281+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd90131270 con 0x7efd7403e710 2024-09-19T11:26:20.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.292+0000 7efd8d7fa640 1 -- 172.21.15.22:0/1900345895 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665813 (secure 0 0 0) 0x7efd7c0326c0 con 0x7efd7403e710 2024-09-19T11:26:20.300 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:26:20.305 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.296+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efd7403e710 msgr2=0x7efd74040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.296+0000 7efd96d9b640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efd7403e710 0x7efd74040bd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efd7c034000 tx=0x7efd7c02e000 comp rx=0 tx=0).stop 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.296+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012c0b0 msgr2=0x7efd9012f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.296+0000 7efd96d9b640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012c0b0 0x7efd9012f870 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7efd8400c3d0 tx=0x7efd8400c890 comp rx=0 tx=0).stop 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.297+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 shutdown_connections 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.297+0000 7efd96d9b640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7efd7403e710 0x7efd74040bd0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.297+0000 7efd96d9b640 1 --2- 172.21.15.22:0/1900345895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efd9012c0b0 0x7efd9012f870 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:20.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.297+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 >> 172.21.15.22:0/1900345895 conn(0x7efd9009abc0 msgr2=0x7efd90129eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:20.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.297+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 shutdown_connections 2024-09-19T11:26:20.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.297+0000 7efd96d9b640 1 -- 172.21.15.22:0/1900345895 wait complete. 2024-09-19T11:26:20.311 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-19T11:26:18.841537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412817,"num_objects":229,"num_object_clones":0,"num_object_copies":465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":229,"num_whiteouts":0,"num_read":4973,"num_read_kb":4511,"num_write":2048,"num_write_kb":60586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4434,"ondisk_log_size":4434,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":152956,"kb_used_data":40700,"kb_used_omap":6,"kb_used_meta":112185,"kb_avail":377334404,"statfs":{"total":386547056640,"available":386390429696,"internally_reserved":0,"allocated":41676800,"data_stored":40498678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":114878230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811688,"num_objects":9,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":477,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004170"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940271+0000","last_change":"2024-09-19T11:23:03.259044+0000","last_active":"2024-09-19T11:23:43.940271+0000","last_peered":"2024-09-19T11:23:43.940271+0000","last_clean":"2024-09-19T11:23:43.940271+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:23:43.940271+0000","last_undegraded":"2024-09-19T11:23:43.940271+0000","last_fullsized":"2024-09-19T11:23:43.940271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:41.435159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333097+0000","last_change":"2024-09-19T11:23:05.248495+0000","last_active":"2024-09-19T11:26:12.333097+0000","last_peered":"2024-09-19T11:26:12.333097+0000","last_clean":"2024-09-19T11:26:12.333097+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:12.333097+0000","last_undegraded":"2024-09-19T11:26:12.333097+0000","last_fullsized":"2024-09-19T11:26:12.333097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:27.929797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.237576+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:20:44.729040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333061+0000","last_change":"2024-09-19T11:23:05.242656+0000","last_active":"2024-09-19T11:26:12.333061+0000","last_peered":"2024-09-19T11:26:12.333061+0000","last_clean":"2024-09-19T11:26:12.333061+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:26:12.333061+0000","last_undegraded":"2024-09-19T11:26:12.333061+0000","last_fullsized":"2024-09-19T11:26:12.333061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.791517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.022905+0000","last_change":"2024-09-19T11:23:03.252002+0000","last_active":"2024-09-19T11:25:08.022905+0000","last_peered":"2024-09-19T11:25:08.022905+0000","last_clean":"2024-09-19T11:25:08.022905+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:25:08.022905+0000","last_undegraded":"2024-09-19T11:25:08.022905+0000","last_fullsized":"2024-09-19T11:25:08.022905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:04:47.977517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'11","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470023+0000","last_change":"2024-09-19T11:23:42.285084+0000","last_active":"2024-09-19T11:26:12.470023+0000","last_peered":"2024-09-19T11:26:12.470023+0000","last_clean":"2024-09-19T11:26:12.470023+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:12.470023+0000","last_undegraded":"2024-09-19T11:26:12.470023+0000","last_fullsized":"2024-09-19T11:26:12.470023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:31.524562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.939807+0000","last_change":"2024-09-19T11:23:03.257377+0000","last_active":"2024-09-19T11:23:43.939807+0000","last_peered":"2024-09-19T11:23:43.939807+0000","last_clean":"2024-09-19T11:23:43.939807+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:23:43.939807+0000","last_undegraded":"2024-09-19T11:23:43.939807+0000","last_fullsized":"2024-09-19T11:23:43.939807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:06.788652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944750+0000","last_change":"2024-09-19T11:23:05.233162+0000","last_active":"2024-09-19T11:23:42.944750+0000","last_peered":"2024-09-19T11:23:42.944750+0000","last_clean":"2024-09-19T11:23:42.944750+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:23:42.944750+0000","last_undegraded":"2024-09-19T11:23:42.944750+0000","last_fullsized":"2024-09-19T11:23:42.944750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:49.026520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:36.813838+0000","last_change":"2024-09-19T11:23:03.254504+0000","last_active":"2024-09-19T11:25:36.813838+0000","last_peered":"2024-09-19T11:25:36.813838+0000","last_clean":"2024-09-19T11:25:36.813838+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:25:36.813838+0000","last_undegraded":"2024-09-19T11:25:36.813838+0000","last_fullsized":"2024-09-19T11:25:36.813838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:05.223629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'11","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.128818+0000","last_change":"2024-09-19T11:23:05.247147+0000","last_active":"2024-09-19T11:26:12.128818+0000","last_peered":"2024-09-19T11:26:12.128818+0000","last_clean":"2024-09-19T11:26:12.128818+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:12.128818+0000","last_undegraded":"2024-09-19T11:26:12.128818+0000","last_fullsized":"2024-09-19T11:26:12.128818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:25.162001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276791+0000","last_change":"2024-09-19T11:23:03.256689+0000","last_active":"2024-09-19T11:23:42.276791+0000","last_peered":"2024-09-19T11:23:42.276791+0000","last_clean":"2024-09-19T11:23:42.276791+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:23:42.276791+0000","last_undegraded":"2024-09-19T11:23:42.276791+0000","last_fullsized":"2024-09-19T11:23:42.276791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:09.931084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374918+0000","last_change":"2024-09-19T11:23:05.254262+0000","last_active":"2024-09-19T11:26:12.374918+0000","last_peered":"2024-09-19T11:26:12.374918+0000","last_clean":"2024-09-19T11:26:12.374918+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:26:12.374918+0000","last_undegraded":"2024-09-19T11:26:12.374918+0000","last_fullsized":"2024-09-19T11:26:12.374918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:58.352519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.252828+0000","last_change":"2024-09-19T11:23:03.239664+0000","last_active":"2024-09-19T11:26:12.252828+0000","last_peered":"2024-09-19T11:26:12.252828+0000","last_clean":"2024-09-19T11:26:12.252828+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:12.252828+0000","last_undegraded":"2024-09-19T11:26:12.252828+0000","last_fullsized":"2024-09-19T11:26:12.252828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:43.733405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333282+0000","last_change":"2024-09-19T11:23:05.250554+0000","last_active":"2024-09-19T11:26:12.333282+0000","last_peered":"2024-09-19T11:26:12.333282+0000","last_clean":"2024-09-19T11:26:12.333282+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:12.333282+0000","last_undegraded":"2024-09-19T11:26:12.333282+0000","last_fullsized":"2024-09-19T11:26:12.333282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278484+0000","last_change":"2024-09-19T11:23:03.237565+0000","last_active":"2024-09-19T11:23:42.278484+0000","last_peered":"2024-09-19T11:23:42.278484+0000","last_clean":"2024-09-19T11:23:42.278484+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:23:42.278484+0000","last_undegraded":"2024-09-19T11:23:42.278484+0000","last_fullsized":"2024-09-19T11:23:42.278484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:24:40.441642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.012417+0000","last_change":"2024-09-19T11:23:05.246773+0000","last_active":"2024-09-19T11:23:45.012417+0000","last_peered":"2024-09-19T11:23:45.012417+0000","last_clean":"2024-09-19T11:23:45.012417+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:23:45.012417+0000","last_undegraded":"2024-09-19T11:23:45.012417+0000","last_fullsized":"2024-09-19T11:23:45.012417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:05:02.041530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:05.241849+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:04.673024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283475+0000","last_change":"2024-09-19T11:23:03.250062+0000","last_active":"2024-09-19T11:23:42.283475+0000","last_peered":"2024-09-19T11:23:42.283475+0000","last_clean":"2024-09-19T11:23:42.283475+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:23:42.283475+0000","last_undegraded":"2024-09-19T11:23:42.283475+0000","last_fullsized":"2024-09-19T11:23:42.283475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:32.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.483130+0000","last_change":"2024-09-19T11:23:05.246402+0000","last_active":"2024-09-19T11:23:45.483130+0000","last_peered":"2024-09-19T11:23:45.483130+0000","last_clean":"2024-09-19T11:23:45.483130+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:23:45.483130+0000","last_undegraded":"2024-09-19T11:23:45.483130+0000","last_fullsized":"2024-09-19T11:23:45.483130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:44.280015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281849+0000","last_change":"2024-09-19T11:23:03.259838+0000","last_active":"2024-09-19T11:23:42.281849+0000","last_peered":"2024-09-19T11:23:42.281849+0000","last_clean":"2024-09-19T11:23:42.281849+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:23:42.281849+0000","last_undegraded":"2024-09-19T11:23:42.281849+0000","last_fullsized":"2024-09-19T11:23:42.281849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342502+0000","last_change":"2024-09-19T11:23:05.248326+0000","last_active":"2024-09-19T11:24:27.342502+0000","last_peered":"2024-09-19T11:24:27.342502+0000","last_clean":"2024-09-19T11:24:27.342502+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:24:27.342502+0000","last_undegraded":"2024-09-19T11:24:27.342502+0000","last_fullsized":"2024-09-19T11:24:27.342502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:28.400630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.969817+0000","last_change":"2024-09-19T11:23:03.250122+0000","last_active":"2024-09-19T11:26:11.969817+0000","last_peered":"2024-09-19T11:26:11.969817+0000","last_clean":"2024-09-19T11:26:11.969817+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:11.969817+0000","last_undegraded":"2024-09-19T11:26:11.969817+0000","last_fullsized":"2024-09-19T11:26:11.969817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:13.455767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123608+0000","last_change":"2024-09-19T11:23:05.240918+0000","last_active":"2024-09-19T11:26:12.123608+0000","last_peered":"2024-09-19T11:26:12.123608+0000","last_clean":"2024-09-19T11:26:12.123608+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:12.123608+0000","last_undegraded":"2024-09-19T11:26:12.123608+0000","last_fullsized":"2024-09-19T11:26:12.123608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:19.590735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282416+0000","last_change":"2024-09-19T11:23:03.245456+0000","last_active":"2024-09-19T11:23:42.282416+0000","last_peered":"2024-09-19T11:23:42.282416+0000","last_clean":"2024-09-19T11:23:42.282416+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:23:42.282416+0000","last_undegraded":"2024-09-19T11:23:42.282416+0000","last_fullsized":"2024-09-19T11:23:42.282416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:14.954640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266363+0000","last_change":"2024-09-19T11:23:05.245486+0000","last_active":"2024-09-19T11:26:12.266363+0000","last_peered":"2024-09-19T11:26:12.266363+0000","last_clean":"2024-09-19T11:26:12.266363+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:12.266363+0000","last_undegraded":"2024-09-19T11:26:12.266363+0000","last_fullsized":"2024-09-19T11:26:12.266363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.454541+0000","last_change":"2024-09-19T11:23:03.243483+0000","last_active":"2024-09-19T11:25:11.454541+0000","last_peered":"2024-09-19T11:25:11.454541+0000","last_clean":"2024-09-19T11:25:11.454541+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:25:11.454541+0000","last_undegraded":"2024-09-19T11:25:11.454541+0000","last_fullsized":"2024-09-19T11:25:11.454541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:10.061252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266347+0000","last_change":"2024-09-19T11:23:05.248431+0000","last_active":"2024-09-19T11:26:12.266347+0000","last_peered":"2024-09-19T11:26:12.266347+0000","last_clean":"2024-09-19T11:26:12.266347+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:12.266347+0000","last_undegraded":"2024-09-19T11:26:12.266347+0000","last_fullsized":"2024-09-19T11:26:12.266347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:17.337349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945193+0000","last_change":"2024-09-19T11:23:03.240547+0000","last_active":"2024-09-19T11:23:42.945193+0000","last_peered":"2024-09-19T11:23:42.945193+0000","last_clean":"2024-09-19T11:23:42.945193+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:23:42.945193+0000","last_undegraded":"2024-09-19T11:23:42.945193+0000","last_fullsized":"2024-09-19T11:23:42.945193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:55.218589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'37","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.305388+0000","last_change":"2024-09-19T11:23:13.275524+0000","last_active":"2024-09-19T11:26:12.305388+0000","last_peered":"2024-09-19T11:26:12.305388+0000","last_clean":"2024-09-19T11:26:12.305388+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:26:12.305388+0000","last_undegraded":"2024-09-19T11:26:12.305388+0000","last_fullsized":"2024-09-19T11:26:12.305388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.670701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.018412+0000","last_change":"2024-09-19T11:23:42.283032+0000","last_active":"2024-09-19T11:26:12.018412+0000","last_peered":"2024-09-19T11:26:12.018412+0000","last_clean":"2024-09-19T11:26:12.018412+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:12.018412+0000","last_undegraded":"2024-09-19T11:26:12.018412+0000","last_fullsized":"2024-09-19T11:26:12.018412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.760502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811976+0000","last_change":"2024-09-19T11:23:03.258428+0000","last_active":"2024-09-19T11:25:26.811976+0000","last_peered":"2024-09-19T11:25:26.811976+0000","last_clean":"2024-09-19T11:25:26.811976+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:25:26.811976+0000","last_undegraded":"2024-09-19T11:25:26.811976+0000","last_fullsized":"2024-09-19T11:25:26.811976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:51.654373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945113+0000","last_change":"2024-09-19T11:23:07.245040+0000","last_active":"2024-09-19T11:23:42.945113+0000","last_peered":"2024-09-19T11:23:42.945113+0000","last_clean":"2024-09-19T11:23:42.945113+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:23:42.945113+0000","last_undegraded":"2024-09-19T11:23:42.945113+0000","last_fullsized":"2024-09-19T11:23:42.945113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.056714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.749025+0000","last_change":"2024-09-19T11:23:09.256765+0000","last_active":"2024-09-19T11:25:09.749025+0000","last_peered":"2024-09-19T11:25:09.749025+0000","last_clean":"2024-09-19T11:25:09.749025+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:25:09.749025+0000","last_undegraded":"2024-09-19T11:25:09.749025+0000","last_fullsized":"2024-09-19T11:25:09.749025+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:39:32.451376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283356+0000","last_change":"2024-09-19T11:23:11.267021+0000","last_active":"2024-09-19T11:23:42.283356+0000","last_peered":"2024-09-19T11:23:42.283356+0000","last_clean":"2024-09-19T11:23:42.283356+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:23:42.283356+0000","last_undegraded":"2024-09-19T11:23:42.283356+0000","last_fullsized":"2024-09-19T11:23:42.283356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.806510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.707113+0000","last_change":"2024-09-19T11:23:13.274325+0000","last_active":"2024-09-19T11:26:11.707113+0000","last_peered":"2024-09-19T11:26:11.707113+0000","last_clean":"2024-09-19T11:26:11.707113+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:26:11.707113+0000","last_undegraded":"2024-09-19T11:26:11.707113+0000","last_fullsized":"2024-09-19T11:26:11.707113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:53:22.517816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.332597+0000","last_change":"2024-09-19T11:23:05.237978+0000","last_active":"2024-09-19T11:26:12.332597+0000","last_peered":"2024-09-19T11:26:12.332597+0000","last_clean":"2024-09-19T11:26:12.332597+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:12.332597+0000","last_undegraded":"2024-09-19T11:26:12.332597+0000","last_fullsized":"2024-09-19T11:26:12.332597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:02.964073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.319875+0000","last_change":"2024-09-19T11:23:03.256489+0000","last_active":"2024-09-19T11:25:11.319875+0000","last_peered":"2024-09-19T11:25:11.319875+0000","last_clean":"2024-09-19T11:25:11.319875+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:25:11.319875+0000","last_undegraded":"2024-09-19T11:25:11.319875+0000","last_fullsized":"2024-09-19T11:25:11.319875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:34.312130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279716+0000","last_change":"2024-09-19T11:23:07.245213+0000","last_active":"2024-09-19T11:23:42.279716+0000","last_peered":"2024-09-19T11:23:42.279716+0000","last_clean":"2024-09-19T11:23:42.279716+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:23:42.279716+0000","last_undegraded":"2024-09-19T11:23:42.279716+0000","last_fullsized":"2024-09-19T11:23:42.279716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:56.312564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'7","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.551668+0000","last_change":"2024-09-19T11:23:09.248257+0000","last_active":"2024-09-19T11:24:27.551668+0000","last_peered":"2024-09-19T11:24:27.551668+0000","last_clean":"2024-09-19T11:24:27.551668+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:24:27.551668+0000","last_undegraded":"2024-09-19T11:24:27.551668+0000","last_fullsized":"2024-09-19T11:24:27.551668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:18.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281824+0000","last_change":"2024-09-19T11:23:11.263544+0000","last_active":"2024-09-19T11:23:42.281824+0000","last_peered":"2024-09-19T11:23:42.281824+0000","last_clean":"2024-09-19T11:23:42.281824+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:23:42.281824+0000","last_undegraded":"2024-09-19T11:23:42.281824+0000","last_fullsized":"2024-09-19T11:23:42.281824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:44.652949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278313+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:23:42.278313+0000","last_peered":"2024-09-19T11:23:42.278313+0000","last_clean":"2024-09-19T11:23:42.278313+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:23:42.278313+0000","last_undegraded":"2024-09-19T11:23:42.278313+0000","last_fullsized":"2024-09-19T11:23:42.278313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:33:33.618068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510054+0000","last_change":"2024-09-19T11:23:05.236597+0000","last_active":"2024-09-19T11:25:11.510054+0000","last_peered":"2024-09-19T11:25:11.510054+0000","last_clean":"2024-09-19T11:25:11.510054+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:25:11.510054+0000","last_undegraded":"2024-09-19T11:25:11.510054+0000","last_fullsized":"2024-09-19T11:25:11.510054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:11.702021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.486679+0000","last_change":"2024-09-19T11:23:03.257697+0000","last_active":"2024-09-19T11:25:11.486679+0000","last_peered":"2024-09-19T11:25:11.486679+0000","last_clean":"2024-09-19T11:25:11.486679+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:25:11.486679+0000","last_undegraded":"2024-09-19T11:25:11.486679+0000","last_fullsized":"2024-09-19T11:25:11.486679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:55.338804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796210+0000","last_change":"2024-09-19T11:23:42.284049+0000","last_active":"2024-09-19T11:23:44.796210+0000","last_peered":"2024-09-19T11:23:44.796210+0000","last_clean":"2024-09-19T11:23:44.796210+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:23:44.796210+0000","last_undegraded":"2024-09-19T11:23:44.796210+0000","last_fullsized":"2024-09-19T11:23:44.796210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279160+0000","last_change":"2024-09-19T11:23:09.253591+0000","last_active":"2024-09-19T11:23:42.279160+0000","last_peered":"2024-09-19T11:23:42.279160+0000","last_clean":"2024-09-19T11:23:42.279160+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:23:42.279160+0000","last_undegraded":"2024-09-19T11:23:42.279160+0000","last_fullsized":"2024-09-19T11:23:42.279160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:52.970909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'282","reported_seq":386,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.615383+0000","last_change":"2024-09-19T11:23:11.271056+0000","last_active":"2024-09-19T11:26:12.615383+0000","last_peered":"2024-09-19T11:26:12.615383+0000","last_clean":"2024-09-19T11:26:12.615383+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:26:12.615383+0000","last_undegraded":"2024-09-19T11:26:12.615383+0000","last_fullsized":"2024-09-19T11:26:12.615383+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:07.822761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.312802+0000","last_change":"2024-09-19T11:23:13.270157+0000","last_active":"2024-09-19T11:26:12.312802+0000","last_peered":"2024-09-19T11:26:12.312802+0000","last_clean":"2024-09-19T11:26:12.312802+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:26:12.312802+0000","last_undegraded":"2024-09-19T11:26:12.312802+0000","last_fullsized":"2024-09-19T11:26:12.312802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:32.371537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.781965+0000","last_change":"2024-09-19T11:23:05.248712+0000","last_active":"2024-09-19T11:23:44.781965+0000","last_peered":"2024-09-19T11:23:44.781965+0000","last_clean":"2024-09-19T11:23:44.781965+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:23:44.781965+0000","last_undegraded":"2024-09-19T11:23:44.781965+0000","last_fullsized":"2024-09-19T11:23:44.781965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:01:31.288073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279237+0000","last_change":"2024-09-19T11:23:03.250291+0000","last_active":"2024-09-19T11:23:42.279237+0000","last_peered":"2024-09-19T11:23:42.279237+0000","last_clean":"2024-09-19T11:23:42.279237+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:23:42.279237+0000","last_undegraded":"2024-09-19T11:23:42.279237+0000","last_fullsized":"2024-09-19T11:23:42.279237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:27.615110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947691+0000","last_change":"2024-09-19T11:23:07.248024+0000","last_active":"2024-09-19T11:23:42.947691+0000","last_peered":"2024-09-19T11:23:42.947691+0000","last_clean":"2024-09-19T11:23:42.947691+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:23:42.947691+0000","last_undegraded":"2024-09-19T11:23:42.947691+0000","last_fullsized":"2024-09-19T11:23:42.947691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:02.812050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.743749+0000","last_change":"2024-09-19T11:23:09.260866+0000","last_active":"2024-09-19T11:25:09.743749+0000","last_peered":"2024-09-19T11:25:09.743749+0000","last_clean":"2024-09-19T11:25:09.743749+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:25:09.743749+0000","last_undegraded":"2024-09-19T11:25:09.743749+0000","last_fullsized":"2024-09-19T11:25:09.743749+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:22.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'282","reported_seq":402,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.371212+0000","last_change":"2024-09-19T11:23:11.260474+0000","last_active":"2024-09-19T11:26:12.371212+0000","last_peered":"2024-09-19T11:26:12.371212+0000","last_clean":"2024-09-19T11:26:12.371212+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:26:12.371212+0000","last_undegraded":"2024-09-19T11:26:12.371212+0000","last_fullsized":"2024-09-19T11:26:12.371212+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:32.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032873+0000","last_change":"2024-09-19T11:23:05.253843+0000","last_active":"2024-09-19T11:25:08.032873+0000","last_peered":"2024-09-19T11:25:08.032873+0000","last_clean":"2024-09-19T11:25:08.032873+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:25:08.032873+0000","last_undegraded":"2024-09-19T11:25:08.032873+0000","last_fullsized":"2024-09-19T11:25:08.032873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:22.232407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279601+0000","last_change":"2024-09-19T11:23:03.256377+0000","last_active":"2024-09-19T11:23:42.279601+0000","last_peered":"2024-09-19T11:23:42.279601+0000","last_clean":"2024-09-19T11:23:42.279601+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:23:42.279601+0000","last_undegraded":"2024-09-19T11:23:42.279601+0000","last_fullsized":"2024-09-19T11:23:42.279601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:29.305136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281170+0000","last_change":"2024-09-19T11:23:07.244371+0000","last_active":"2024-09-19T11:23:42.281170+0000","last_peered":"2024-09-19T11:23:42.281170+0000","last_clean":"2024-09-19T11:23:42.281170+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:23:42.281170+0000","last_undegraded":"2024-09-19T11:23:42.281170+0000","last_fullsized":"2024-09-19T11:23:42.281170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:53.267571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'40","reported_seq":88,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.753372+0000","last_change":"2024-09-19T11:23:09.249075+0000","last_active":"2024-09-19T11:25:09.753372+0000","last_peered":"2024-09-19T11:25:09.753372+0000","last_clean":"2024-09-19T11:25:09.753372+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:25:09.753372+0000","last_undegraded":"2024-09-19T11:25:09.753372+0000","last_fullsized":"2024-09-19T11:25:09.753372+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:03.176195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281316+0000","last_change":"2024-09-19T11:23:11.258275+0000","last_active":"2024-09-19T11:23:42.281316+0000","last_peered":"2024-09-19T11:23:42.281316+0000","last_clean":"2024-09-19T11:23:42.281316+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:23:42.281316+0000","last_undegraded":"2024-09-19T11:23:42.281316+0000","last_fullsized":"2024-09-19T11:23:42.281316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:14.017912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.349482+0000","last_change":"2024-09-19T11:23:13.274689+0000","last_active":"2024-09-19T11:24:27.349482+0000","last_peered":"2024-09-19T11:24:27.349482+0000","last_clean":"2024-09-19T11:24:27.349482+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:24:27.349482+0000","last_undegraded":"2024-09-19T11:24:27.349482+0000","last_fullsized":"2024-09-19T11:24:27.349482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:05.210803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281462+0000","last_change":"2024-09-19T11:23:05.245370+0000","last_active":"2024-09-19T11:23:42.281462+0000","last_peered":"2024-09-19T11:23:42.281462+0000","last_clean":"2024-09-19T11:23:42.281462+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:23:42.281462+0000","last_undegraded":"2024-09-19T11:23:42.281462+0000","last_fullsized":"2024-09-19T11:23:42.281462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:55:45.570918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280712+0000","last_change":"2024-09-19T11:23:03.257588+0000","last_active":"2024-09-19T11:23:42.280712+0000","last_peered":"2024-09-19T11:23:42.280712+0000","last_clean":"2024-09-19T11:23:42.280712+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:23:42.280712+0000","last_undegraded":"2024-09-19T11:23:42.280712+0000","last_fullsized":"2024-09-19T11:23:42.280712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.928791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278703+0000","last_change":"2024-09-19T11:23:07.247386+0000","last_active":"2024-09-19T11:23:42.278703+0000","last_peered":"2024-09-19T11:23:42.278703+0000","last_clean":"2024-09-19T11:23:42.278703+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:23:42.278703+0000","last_undegraded":"2024-09-19T11:23:42.278703+0000","last_fullsized":"2024-09-19T11:23:42.278703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:53.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'30","reported_seq":73,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.733747+0000","last_change":"2024-09-19T11:23:09.262668+0000","last_active":"2024-09-19T11:25:09.733747+0000","last_peered":"2024-09-19T11:25:09.733747+0000","last_clean":"2024-09-19T11:25:09.733747+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:25:09.733747+0000","last_undegraded":"2024-09-19T11:25:09.733747+0000","last_fullsized":"2024-09-19T11:25:09.733747+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:42.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278485+0000","last_change":"2024-09-19T11:23:11.265649+0000","last_active":"2024-09-19T11:23:42.278485+0000","last_peered":"2024-09-19T11:23:42.278485+0000","last_clean":"2024-09-19T11:23:42.278485+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:23:42.278485+0000","last_undegraded":"2024-09-19T11:23:42.278485+0000","last_fullsized":"2024-09-19T11:23:42.278485+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:22.946739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'5","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.124970+0000","last_change":"2024-09-19T11:23:13.268845+0000","last_active":"2024-09-19T11:26:12.124970+0000","last_peered":"2024-09-19T11:26:12.124970+0000","last_clean":"2024-09-19T11:26:12.124970+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:26:12.124970+0000","last_undegraded":"2024-09-19T11:26:12.124970+0000","last_fullsized":"2024-09-19T11:26:12.124970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:30.688522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342674+0000","last_change":"2024-09-19T11:23:05.251302+0000","last_active":"2024-09-19T11:24:27.342674+0000","last_peered":"2024-09-19T11:24:27.342674+0000","last_clean":"2024-09-19T11:24:27.342674+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:24:27.342674+0000","last_undegraded":"2024-09-19T11:24:27.342674+0000","last_fullsized":"2024-09-19T11:24:27.342674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:55.904446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'3","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.249995+0000","last_change":"2024-09-19T11:23:03.242612+0000","last_active":"2024-09-19T11:26:12.249995+0000","last_peered":"2024-09-19T11:26:12.249995+0000","last_clean":"2024-09-19T11:26:12.249995+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:12.249995+0000","last_undegraded":"2024-09-19T11:26:12.249995+0000","last_fullsized":"2024-09-19T11:26:12.249995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:48.666399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":11,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281855+0000","last_change":"2024-09-19T11:23:07.247594+0000","last_active":"2024-09-19T11:23:42.281855+0000","last_peered":"2024-09-19T11:23:42.281855+0000","last_clean":"2024-09-19T11:23:42.281855+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:23:42.281855+0000","last_undegraded":"2024-09-19T11:23:42.281855+0000","last_fullsized":"2024-09-19T11:23:42.281855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:41.108832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.755394+0000","last_change":"2024-09-19T11:23:09.259476+0000","last_active":"2024-09-19T11:25:09.755394+0000","last_peered":"2024-09-19T11:25:09.755394+0000","last_clean":"2024-09-19T11:25:09.755394+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:25:09.755394+0000","last_undegraded":"2024-09-19T11:25:09.755394+0000","last_fullsized":"2024-09-19T11:25:09.755394+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:21.364894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'282","reported_seq":349,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.641675+0000","last_change":"2024-09-19T11:23:11.267050+0000","last_active":"2024-09-19T11:26:12.641675+0000","last_peered":"2024-09-19T11:26:12.641675+0000","last_clean":"2024-09-19T11:26:12.641675+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:26:12.641675+0000","last_undegraded":"2024-09-19T11:26:12.641675+0000","last_fullsized":"2024-09-19T11:26:12.641675+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.330204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.334604+0000","last_change":"2024-09-19T11:23:13.269663+0000","last_active":"2024-09-19T11:26:12.334604+0000","last_peered":"2024-09-19T11:26:12.334604+0000","last_clean":"2024-09-19T11:26:12.334604+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:26:12.334604+0000","last_undegraded":"2024-09-19T11:26:12.334604+0000","last_fullsized":"2024-09-19T11:26:12.334604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:57:11.742213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368223+0000","last_change":"2024-09-19T11:23:05.235632+0000","last_active":"2024-09-19T11:26:12.368223+0000","last_peered":"2024-09-19T11:26:12.368223+0000","last_clean":"2024-09-19T11:26:12.368223+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:12.368223+0000","last_undegraded":"2024-09-19T11:26:12.368223+0000","last_fullsized":"2024-09-19T11:26:12.368223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945253+0000","last_change":"2024-09-19T11:23:03.257570+0000","last_active":"2024-09-19T11:23:42.945253+0000","last_peered":"2024-09-19T11:23:42.945253+0000","last_clean":"2024-09-19T11:23:42.945253+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:23:42.945253+0000","last_undegraded":"2024-09-19T11:23:42.945253+0000","last_fullsized":"2024-09-19T11:23:42.945253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:41:29.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279784+0000","last_change":"2024-09-19T11:23:07.247239+0000","last_active":"2024-09-19T11:23:42.279784+0000","last_peered":"2024-09-19T11:23:42.279784+0000","last_clean":"2024-09-19T11:23:42.279784+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:23:42.279784+0000","last_undegraded":"2024-09-19T11:23:42.279784+0000","last_fullsized":"2024-09-19T11:23:42.279784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:21.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'55","reported_seq":112,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.512267+0000","last_change":"2024-09-19T11:23:09.259576+0000","last_active":"2024-09-19T11:25:11.512267+0000","last_peered":"2024-09-19T11:25:11.512267+0000","last_clean":"2024-09-19T11:25:11.512267+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:25:11.512267+0000","last_undegraded":"2024-09-19T11:25:11.512267+0000","last_fullsized":"2024-09-19T11:25:11.512267+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.556001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278323+0000","last_change":"2024-09-19T11:23:11.260403+0000","last_active":"2024-09-19T11:23:42.278323+0000","last_peered":"2024-09-19T11:23:42.278323+0000","last_clean":"2024-09-19T11:23:42.278323+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:23:42.278323+0000","last_undegraded":"2024-09-19T11:23:42.278323+0000","last_fullsized":"2024-09-19T11:23:42.278323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:11.335945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'18","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585256+0000","last_change":"2024-09-19T11:23:13.273720+0000","last_active":"2024-09-19T11:26:12.585256+0000","last_peered":"2024-09-19T11:26:12.585256+0000","last_clean":"2024-09-19T11:26:12.585256+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:26:12.585256+0000","last_undegraded":"2024-09-19T11:26:12.585256+0000","last_fullsized":"2024-09-19T11:26:12.585256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:46.547393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:21.810844+0000","last_change":"2024-09-19T11:23:05.247032+0000","last_active":"2024-09-19T11:25:21.810844+0000","last_peered":"2024-09-19T11:25:21.810844+0000","last_clean":"2024-09-19T11:25:21.810844+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:25:21.810844+0000","last_undegraded":"2024-09-19T11:25:21.810844+0000","last_fullsized":"2024-09-19T11:25:21.810844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:24.496941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.255786+0000","last_change":"2024-09-19T11:23:03.234223+0000","last_active":"2024-09-19T11:26:12.255786+0000","last_peered":"2024-09-19T11:26:12.255786+0000","last_clean":"2024-09-19T11:26:12.255786+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:12.255786+0000","last_undegraded":"2024-09-19T11:26:12.255786+0000","last_fullsized":"2024-09-19T11:26:12.255786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278469+0000","last_change":"2024-09-19T11:23:07.244885+0000","last_active":"2024-09-19T11:23:42.278469+0000","last_peered":"2024-09-19T11:23:42.278469+0000","last_clean":"2024-09-19T11:23:42.278469+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:23:42.278469+0000","last_undegraded":"2024-09-19T11:23:42.278469+0000","last_fullsized":"2024-09-19T11:23:42.278469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:45:02.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'33","reported_seq":77,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811791+0000","last_change":"2024-09-19T11:23:09.248190+0000","last_active":"2024-09-19T11:25:26.811791+0000","last_peered":"2024-09-19T11:25:26.811791+0000","last_clean":"2024-09-19T11:25:26.811791+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:25:26.811791+0000","last_undegraded":"2024-09-19T11:25:26.811791+0000","last_fullsized":"2024-09-19T11:25:26.811791+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:38.180379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946607+0000","last_change":"2024-09-19T11:23:11.270709+0000","last_active":"2024-09-19T11:23:42.946607+0000","last_peered":"2024-09-19T11:23:42.946607+0000","last_clean":"2024-09-19T11:23:42.946607+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:23:42.946607+0000","last_undegraded":"2024-09-19T11:23:42.946607+0000","last_fullsized":"2024-09-19T11:23:42.946607+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:39.157590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809883+0000","last_change":"2024-09-19T11:23:13.272542+0000","last_active":"2024-09-19T11:25:16.809883+0000","last_peered":"2024-09-19T11:25:16.809883+0000","last_clean":"2024-09-19T11:25:16.809883+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:25:16.809883+0000","last_undegraded":"2024-09-19T11:25:16.809883+0000","last_fullsized":"2024-09-19T11:25:16.809883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:52.793325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433382+0000","last_change":"2024-09-19T11:23:05.254593+0000","last_active":"2024-09-19T11:23:44.433382+0000","last_peered":"2024-09-19T11:23:44.433382+0000","last_clean":"2024-09-19T11:23:44.433382+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:23:44.433382+0000","last_undegraded":"2024-09-19T11:23:44.433382+0000","last_fullsized":"2024-09-19T11:23:44.433382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:27.816142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945690+0000","last_change":"2024-09-19T11:23:03.241691+0000","last_active":"2024-09-19T11:23:42.945690+0000","last_peered":"2024-09-19T11:23:42.945690+0000","last_clean":"2024-09-19T11:23:42.945690+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:23:42.945690+0000","last_undegraded":"2024-09-19T11:23:42.945690+0000","last_fullsized":"2024-09-19T11:23:42.945690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:21.286478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279479+0000","last_change":"2024-09-19T11:23:07.247411+0000","last_active":"2024-09-19T11:23:42.279479+0000","last_peered":"2024-09-19T11:23:42.279479+0000","last_clean":"2024-09-19T11:23:42.279479+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:23:42.279479+0000","last_undegraded":"2024-09-19T11:23:42.279479+0000","last_fullsized":"2024-09-19T11:23:42.279479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:20:41.691053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.763568+0000","last_change":"2024-09-19T11:23:09.246515+0000","last_active":"2024-09-19T11:25:09.763568+0000","last_peered":"2024-09-19T11:25:09.763568+0000","last_clean":"2024-09-19T11:25:09.763568+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:25:09.763568+0000","last_undegraded":"2024-09-19T11:25:09.763568+0000","last_fullsized":"2024-09-19T11:25:09.763568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:54.969396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'282","reported_seq":404,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.589475+0000","last_change":"2024-09-19T11:23:11.263693+0000","last_active":"2024-09-19T11:26:12.589475+0000","last_peered":"2024-09-19T11:26:12.589475+0000","last_clean":"2024-09-19T11:26:12.589475+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:26:12.589475+0000","last_undegraded":"2024-09-19T11:26:12.589475+0000","last_fullsized":"2024-09-19T11:26:12.589475+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:29.980859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'14","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.475120+0000","last_change":"2024-09-19T11:23:13.265074+0000","last_active":"2024-09-19T11:26:12.475120+0000","last_peered":"2024-09-19T11:26:12.475120+0000","last_clean":"2024-09-19T11:26:12.475120+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:26:12.475120+0000","last_undegraded":"2024-09-19T11:26:12.475120+0000","last_fullsized":"2024-09-19T11:26:12.475120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:47.118156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368115+0000","last_change":"2024-09-19T11:23:05.250216+0000","last_active":"2024-09-19T11:26:12.368115+0000","last_peered":"2024-09-19T11:26:12.368115+0000","last_clean":"2024-09-19T11:26:12.368115+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:12.368115+0000","last_undegraded":"2024-09-19T11:26:12.368115+0000","last_fullsized":"2024-09-19T11:26:12.368115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.941399+0000","last_change":"2024-09-19T11:23:03.255564+0000","last_active":"2024-09-19T11:23:43.941399+0000","last_peered":"2024-09-19T11:23:43.941399+0000","last_clean":"2024-09-19T11:23:43.941399+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:23:43.941399+0000","last_undegraded":"2024-09-19T11:23:43.941399+0000","last_fullsized":"2024-09-19T11:23:43.941399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:10:03.443220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'3","reported_seq":414,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.866528+0000","last_change":"2024-09-19T11:23:07.242151+0000","last_active":"2024-09-19T11:25:10.866528+0000","last_peered":"2024-09-19T11:25:10.866528+0000","last_clean":"2024-09-19T11:25:10.866528+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:25:10.866528+0000","last_undegraded":"2024-09-19T11:25:10.866528+0000","last_fullsized":"2024-09-19T11:25:10.866528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:15.211742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946255+0000","last_change":"2024-09-19T11:23:09.255928+0000","last_active":"2024-09-19T11:23:42.946255+0000","last_peered":"2024-09-19T11:23:42.946255+0000","last_clean":"2024-09-19T11:23:42.946255+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:23:42.946255+0000","last_undegraded":"2024-09-19T11:23:42.946255+0000","last_fullsized":"2024-09-19T11:23:42.946255+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:04.689951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'282","reported_seq":466,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.473914+0000","last_change":"2024-09-19T11:23:11.255933+0000","last_active":"2024-09-19T11:26:12.473914+0000","last_peered":"2024-09-19T11:26:12.473914+0000","last_clean":"2024-09-19T11:26:12.473914+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:26:12.473914+0000","last_undegraded":"2024-09-19T11:26:12.473914+0000","last_fullsized":"2024-09-19T11:26:12.473914+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:01.160503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.141728+0000","last_change":"2024-09-19T11:23:13.266957+0000","last_active":"2024-09-19T11:25:11.141728+0000","last_peered":"2024-09-19T11:25:11.141728+0000","last_clean":"2024-09-19T11:25:11.141728+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:25:11.141728+0000","last_undegraded":"2024-09-19T11:25:11.141728+0000","last_fullsized":"2024-09-19T11:25:11.141728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:59.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":101,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338578+0000","last_change":"2024-09-19T11:23:05.255344+0000","last_active":"2024-09-19T11:26:12.338578+0000","last_peered":"2024-09-19T11:26:12.338578+0000","last_clean":"2024-09-19T11:26:12.338578+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:12.338578+0000","last_undegraded":"2024-09-19T11:26:12.338578+0000","last_fullsized":"2024-09-19T11:26:12.338578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:22.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944743+0000","last_change":"2024-09-19T11:23:03.254989+0000","last_active":"2024-09-19T11:23:42.944743+0000","last_peered":"2024-09-19T11:23:42.944743+0000","last_clean":"2024-09-19T11:23:42.944743+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:23:42.944743+0000","last_undegraded":"2024-09-19T11:23:42.944743+0000","last_fullsized":"2024-09-19T11:23:42.944743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:39.840111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":416,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.866830+0000","last_change":"2024-09-19T11:23:07.250416+0000","last_active":"2024-09-19T11:25:10.866830+0000","last_peered":"2024-09-19T11:25:10.866830+0000","last_clean":"2024-09-19T11:25:10.866830+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:25:10.866830+0000","last_undegraded":"2024-09-19T11:25:10.866830+0000","last_fullsized":"2024-09-19T11:25:10.866830+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":580,"num_read_kb":580,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'38","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.750105+0000","last_change":"2024-09-19T11:23:09.247733+0000","last_active":"2024-09-19T11:25:09.750105+0000","last_peered":"2024-09-19T11:25:09.750105+0000","last_clean":"2024-09-19T11:25:09.750105+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:25:09.750105+0000","last_undegraded":"2024-09-19T11:25:09.750105+0000","last_fullsized":"2024-09-19T11:25:09.750105+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:38.921761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279060+0000","last_change":"2024-09-19T11:23:11.266008+0000","last_active":"2024-09-19T11:23:42.279060+0000","last_peered":"2024-09-19T11:23:42.279060+0000","last_clean":"2024-09-19T11:23:42.279060+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:23:42.279060+0000","last_undegraded":"2024-09-19T11:23:42.279060+0000","last_fullsized":"2024-09-19T11:23:42.279060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:38:43.871781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.426809+0000","last_change":"2024-09-19T11:23:13.263896+0000","last_active":"2024-09-19T11:26:12.426809+0000","last_peered":"2024-09-19T11:26:12.426809+0000","last_clean":"2024-09-19T11:26:12.426809+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:26:12.426809+0000","last_undegraded":"2024-09-19T11:26:12.426809+0000","last_fullsized":"2024-09-19T11:26:12.426809+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:35.364893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'7","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.273438+0000","last_change":"2024-09-19T11:23:05.254167+0000","last_active":"2024-09-19T11:26:12.273438+0000","last_peered":"2024-09-19T11:26:12.273438+0000","last_clean":"2024-09-19T11:26:12.273438+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:12.273438+0000","last_undegraded":"2024-09-19T11:26:12.273438+0000","last_fullsized":"2024-09-19T11:26:12.273438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:44.425539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":146,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279138+0000","last_change":"2024-09-19T11:22:46.832286+0000","last_active":"2024-09-19T11:23:42.279138+0000","last_peered":"2024-09-19T11:23:42.279138+0000","last_clean":"2024-09-19T11:23:42.279138+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:23:42.279138+0000","last_undegraded":"2024-09-19T11:23:42.279138+0000","last_fullsized":"2024-09-19T11:23:42.279138+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_clean_scrub_stamp":"2024-09-19T11:22:45.784999+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:24.575130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944159+0000","last_change":"2024-09-19T11:22:55.817718+0000","last_active":"2024-09-19T11:23:42.944159+0000","last_peered":"2024-09-19T11:23:42.944159+0000","last_clean":"2024-09-19T11:23:42.944159+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:23:42.944159+0000","last_undegraded":"2024-09-19T11:23:42.944159+0000","last_fullsized":"2024-09-19T11:23:42.944159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:33.890847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944557+0000","last_change":"2024-09-19T11:23:03.258130+0000","last_active":"2024-09-19T11:23:42.944557+0000","last_peered":"2024-09-19T11:23:42.944557+0000","last_clean":"2024-09-19T11:23:42.944557+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:23:42.944557+0000","last_undegraded":"2024-09-19T11:23:42.944557+0000","last_fullsized":"2024-09-19T11:23:42.944557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:11.381818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281740+0000","last_change":"2024-09-19T11:23:07.238893+0000","last_active":"2024-09-19T11:23:42.281740+0000","last_peered":"2024-09-19T11:23:42.281740+0000","last_clean":"2024-09-19T11:23:42.281740+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:23:42.281740+0000","last_undegraded":"2024-09-19T11:23:42.281740+0000","last_fullsized":"2024-09-19T11:23:42.281740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:35.034936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'76","reported_seq":146,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.761411+0000","last_change":"2024-09-19T11:23:09.260843+0000","last_active":"2024-09-19T11:25:09.761411+0000","last_peered":"2024-09-19T11:25:09.761411+0000","last_clean":"2024-09-19T11:25:09.761411+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:25:09.761411+0000","last_undegraded":"2024-09-19T11:25:09.761411+0000","last_fullsized":"2024-09-19T11:25:09.761411+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:17.846685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280169+0000","last_change":"2024-09-19T11:23:11.266314+0000","last_active":"2024-09-19T11:23:42.280169+0000","last_peered":"2024-09-19T11:23:42.280169+0000","last_clean":"2024-09-19T11:23:42.280169+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:23:42.280169+0000","last_undegraded":"2024-09-19T11:23:42.280169+0000","last_fullsized":"2024-09-19T11:23:42.280169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:28.832170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":163,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.377387+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:26:12.377387+0000","last_peered":"2024-09-19T11:26:12.377387+0000","last_clean":"2024-09-19T11:26:12.377387+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:26:12.377387+0000","last_undegraded":"2024-09-19T11:26:12.377387+0000","last_fullsized":"2024-09-19T11:26:12.377387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:51.720881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'5","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.810212+0000","last_change":"2024-09-19T11:23:05.233784+0000","last_active":"2024-09-19T11:25:16.810212+0000","last_peered":"2024-09-19T11:25:16.810212+0000","last_clean":"2024-09-19T11:25:16.810212+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:25:16.810212+0000","last_undegraded":"2024-09-19T11:25:16.810212+0000","last_fullsized":"2024-09-19T11:25:16.810212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943897+0000","last_change":"2024-09-19T11:22:55.817444+0000","last_active":"2024-09-19T11:23:42.943897+0000","last_peered":"2024-09-19T11:23:42.943897+0000","last_clean":"2024-09-19T11:23:42.943897+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:23:42.943897+0000","last_undegraded":"2024-09-19T11:23:42.943897+0000","last_fullsized":"2024-09-19T11:23:42.943897+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:39.661543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.805962+0000","last_change":"2024-09-19T11:23:03.236740+0000","last_active":"2024-09-19T11:25:10.805962+0000","last_peered":"2024-09-19T11:25:10.805962+0000","last_clean":"2024-09-19T11:25:10.805962+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:25:10.805962+0000","last_undegraded":"2024-09-19T11:25:10.805962+0000","last_fullsized":"2024-09-19T11:25:10.805962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:40.283716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948430+0000","last_change":"2024-09-19T11:23:07.246124+0000","last_active":"2024-09-19T11:23:42.948430+0000","last_peered":"2024-09-19T11:23:42.948430+0000","last_clean":"2024-09-19T11:23:42.948430+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:23:42.948430+0000","last_undegraded":"2024-09-19T11:23:42.948430+0000","last_fullsized":"2024-09-19T11:23:42.948430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'273","reported_seq":477,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327313+0000","last_change":"2024-09-19T11:23:09.257240+0000","last_active":"2024-09-19T11:26:12.327313+0000","last_peered":"2024-09-19T11:26:12.327313+0000","last_clean":"2024-09-19T11:26:12.327313+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:26:12.327313+0000","last_undegraded":"2024-09-19T11:26:12.327313+0000","last_fullsized":"2024-09-19T11:26:12.327313+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:14.514960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944824+0000","last_change":"2024-09-19T11:23:11.270807+0000","last_active":"2024-09-19T11:23:42.944824+0000","last_peered":"2024-09-19T11:23:42.944824+0000","last_clean":"2024-09-19T11:23:42.944824+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:23:42.944824+0000","last_undegraded":"2024-09-19T11:23:42.944824+0000","last_fullsized":"2024-09-19T11:23:42.944824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:23.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.346118+0000","last_change":"2024-09-19T11:23:13.269927+0000","last_active":"2024-09-19T11:24:27.346118+0000","last_peered":"2024-09-19T11:24:27.346118+0000","last_clean":"2024-09-19T11:24:27.346118+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:24:27.346118+0000","last_undegraded":"2024-09-19T11:24:27.346118+0000","last_fullsized":"2024-09-19T11:24:27.346118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:59.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123437+0000","last_change":"2024-09-19T11:23:05.248320+0000","last_active":"2024-09-19T11:26:12.123437+0000","last_peered":"2024-09-19T11:26:12.123437+0000","last_clean":"2024-09-19T11:26:12.123437+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:12.123437+0000","last_undegraded":"2024-09-19T11:26:12.123437+0000","last_fullsized":"2024-09-19T11:26:12.123437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:59.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276942+0000","last_change":"2024-09-19T11:22:55.886390+0000","last_active":"2024-09-19T11:23:42.276942+0000","last_peered":"2024-09-19T11:23:42.276942+0000","last_clean":"2024-09-19T11:23:42.276942+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:23:42.276942+0000","last_undegraded":"2024-09-19T11:23:42.276942+0000","last_fullsized":"2024-09-19T11:23:42.276942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:42.873153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.275364+0000","last_change":"2024-09-19T11:23:03.244094+0000","last_active":"2024-09-19T11:25:11.275364+0000","last_peered":"2024-09-19T11:25:11.275364+0000","last_clean":"2024-09-19T11:25:11.275364+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:25:11.275364+0000","last_undegraded":"2024-09-19T11:25:11.275364+0000","last_fullsized":"2024-09-19T11:25:11.275364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280552+0000","last_change":"2024-09-19T11:23:07.245296+0000","last_active":"2024-09-19T11:23:42.280552+0000","last_peered":"2024-09-19T11:23:42.280552+0000","last_clean":"2024-09-19T11:23:42.280552+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:23:42.280552+0000","last_undegraded":"2024-09-19T11:23:42.280552+0000","last_fullsized":"2024-09-19T11:23:42.280552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:23:05.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'16","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327294+0000","last_change":"2024-09-19T11:23:09.249984+0000","last_active":"2024-09-19T11:26:12.327294+0000","last_peered":"2024-09-19T11:26:12.327294+0000","last_clean":"2024-09-19T11:26:12.327294+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:12.327294+0000","last_undegraded":"2024-09-19T11:26:12.327294+0000","last_fullsized":"2024-09-19T11:26:12.327294+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:47.233637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280391+0000","last_change":"2024-09-19T11:23:11.265633+0000","last_active":"2024-09-19T11:23:42.280391+0000","last_peered":"2024-09-19T11:23:42.280391+0000","last_clean":"2024-09-19T11:23:42.280391+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:23:42.280391+0000","last_undegraded":"2024-09-19T11:23:42.280391+0000","last_fullsized":"2024-09-19T11:23:42.280391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:16.181121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943722+0000","last_change":"2024-09-19T11:23:13.273373+0000","last_active":"2024-09-19T11:23:42.943722+0000","last_peered":"2024-09-19T11:23:42.943722+0000","last_clean":"2024-09-19T11:23:42.943722+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:23:42.943722+0000","last_undegraded":"2024-09-19T11:23:42.943722+0000","last_fullsized":"2024-09-19T11:23:42.943722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:11:13.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368226+0000","last_change":"2024-09-19T11:23:05.238227+0000","last_active":"2024-09-19T11:26:12.368226+0000","last_peered":"2024-09-19T11:26:12.368226+0000","last_clean":"2024-09-19T11:26:12.368226+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:12.368226+0000","last_undegraded":"2024-09-19T11:26:12.368226+0000","last_fullsized":"2024-09-19T11:26:12.368226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:05.469042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944111+0000","last_change":"2024-09-19T11:22:55.817947+0000","last_active":"2024-09-19T11:23:42.944111+0000","last_peered":"2024-09-19T11:23:42.944111+0000","last_clean":"2024-09-19T11:23:42.944111+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:23:42.944111+0000","last_undegraded":"2024-09-19T11:23:42.944111+0000","last_fullsized":"2024-09-19T11:23:42.944111+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:52.495235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.363564+0000","last_change":"2024-09-19T11:23:03.247494+0000","last_active":"2024-09-19T11:26:12.363564+0000","last_peered":"2024-09-19T11:26:12.363564+0000","last_clean":"2024-09-19T11:26:12.363564+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:12.363564+0000","last_undegraded":"2024-09-19T11:26:12.363564+0000","last_fullsized":"2024-09-19T11:26:12.363564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:58.764280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279722+0000","last_change":"2024-09-19T11:23:07.243946+0000","last_active":"2024-09-19T11:23:42.279722+0000","last_peered":"2024-09-19T11:23:42.279722+0000","last_clean":"2024-09-19T11:23:42.279722+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:23:42.279722+0000","last_undegraded":"2024-09-19T11:23:42.279722+0000","last_fullsized":"2024-09-19T11:23:42.279722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:02.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'13","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.530475+0000","last_change":"2024-09-19T11:23:09.254499+0000","last_active":"2024-09-19T11:26:11.530475+0000","last_peered":"2024-09-19T11:26:11.530475+0000","last_clean":"2024-09-19T11:26:11.530475+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:26:11.530475+0000","last_undegraded":"2024-09-19T11:26:11.530475+0000","last_fullsized":"2024-09-19T11:26:11.530475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:18.957107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947826+0000","last_change":"2024-09-19T11:23:11.271073+0000","last_active":"2024-09-19T11:23:42.947826+0000","last_peered":"2024-09-19T11:23:42.947826+0000","last_clean":"2024-09-19T11:23:42.947826+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:23:42.947826+0000","last_undegraded":"2024-09-19T11:23:42.947826+0000","last_fullsized":"2024-09-19T11:23:42.947826+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:52.079937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.088603+0000","last_change":"2024-09-19T11:23:13.273910+0000","last_active":"2024-09-19T11:25:11.088603+0000","last_peered":"2024-09-19T11:25:11.088603+0000","last_clean":"2024-09-19T11:25:11.088603+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:25:11.088603+0000","last_undegraded":"2024-09-19T11:25:11.088603+0000","last_fullsized":"2024-09-19T11:25:11.088603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:02.581618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470157+0000","last_change":"2024-09-19T11:23:05.247999+0000","last_active":"2024-09-19T11:26:12.470157+0000","last_peered":"2024-09-19T11:26:12.470157+0000","last_clean":"2024-09-19T11:26:12.470157+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:12.470157+0000","last_undegraded":"2024-09-19T11:26:12.470157+0000","last_fullsized":"2024-09-19T11:26:12.470157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.575444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277923+0000","last_change":"2024-09-19T11:23:03.251364+0000","last_active":"2024-09-19T11:23:42.277923+0000","last_peered":"2024-09-19T11:23:42.277923+0000","last_clean":"2024-09-19T11:23:42.277923+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:23:42.277923+0000","last_undegraded":"2024-09-19T11:23:42.277923+0000","last_fullsized":"2024-09-19T11:23:42.277923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279906+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.279906+0000","last_peered":"2024-09-19T11:23:42.279906+0000","last_clean":"2024-09-19T11:23:42.279906+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:23:42.279906+0000","last_undegraded":"2024-09-19T11:23:42.279906+0000","last_fullsized":"2024-09-19T11:23:42.279906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:41.618944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:15.806655+0000","last_change":"2024-09-19T11:23:07.247913+0000","last_active":"2024-09-19T11:25:15.806655+0000","last_peered":"2024-09-19T11:25:15.806655+0000","last_clean":"2024-09-19T11:25:15.806655+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:25:15.806655+0000","last_undegraded":"2024-09-19T11:25:15.806655+0000","last_fullsized":"2024-09-19T11:25:15.806655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:03.208183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'62","reported_seq":122,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.805937+0000","last_change":"2024-09-19T11:23:09.257853+0000","last_active":"2024-09-19T11:25:10.805937+0000","last_peered":"2024-09-19T11:25:10.805937+0000","last_clean":"2024-09-19T11:25:10.805937+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:25:10.805937+0000","last_undegraded":"2024-09-19T11:25:10.805937+0000","last_fullsized":"2024-09-19T11:25:10.805937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:02.366161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277438+0000","last_change":"2024-09-19T11:23:11.268600+0000","last_active":"2024-09-19T11:23:42.277438+0000","last_peered":"2024-09-19T11:23:42.277438+0000","last_clean":"2024-09-19T11:23:42.277438+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:23:42.277438+0000","last_undegraded":"2024-09-19T11:23:42.277438+0000","last_fullsized":"2024-09-19T11:23:42.277438+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:50.540232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.422953+0000","last_change":"2024-09-19T11:23:13.267518+0000","last_active":"2024-09-19T11:25:11.422953+0000","last_peered":"2024-09-19T11:25:11.422953+0000","last_clean":"2024-09-19T11:25:11.422953+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:25:11.422953+0000","last_undegraded":"2024-09-19T11:25:11.422953+0000","last_fullsized":"2024-09-19T11:25:11.422953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:35.030653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338573+0000","last_change":"2024-09-19T11:23:05.247369+0000","last_active":"2024-09-19T11:26:12.338573+0000","last_peered":"2024-09-19T11:26:12.338573+0000","last_clean":"2024-09-19T11:26:12.338573+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:12.338573+0000","last_undegraded":"2024-09-19T11:26:12.338573+0000","last_fullsized":"2024-09-19T11:26:12.338573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:05.530837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:10.806030+0000","last_change":"2024-09-19T11:23:03.236746+0000","last_active":"2024-09-19T11:25:10.806030+0000","last_peered":"2024-09-19T11:25:10.806030+0000","last_clean":"2024-09-19T11:25:10.806030+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:25:10.806030+0000","last_undegraded":"2024-09-19T11:25:10.806030+0000","last_fullsized":"2024-09-19T11:25:10.806030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:23.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945428+0000","last_change":"2024-09-19T11:22:55.818302+0000","last_active":"2024-09-19T11:23:42.945428+0000","last_peered":"2024-09-19T11:23:42.945428+0000","last_clean":"2024-09-19T11:23:42.945428+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:23:42.945428+0000","last_undegraded":"2024-09-19T11:23:42.945428+0000","last_fullsized":"2024-09-19T11:23:42.945428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:51.911098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943947+0000","last_change":"2024-09-19T11:23:07.243184+0000","last_active":"2024-09-19T11:23:42.943947+0000","last_peered":"2024-09-19T11:23:42.943947+0000","last_clean":"2024-09-19T11:23:42.943947+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:23:42.943947+0000","last_undegraded":"2024-09-19T11:23:42.943947+0000","last_fullsized":"2024-09-19T11:23:42.943947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:32:59.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'15","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809866+0000","last_change":"2024-09-19T11:23:09.261039+0000","last_active":"2024-09-19T11:25:16.809866+0000","last_peered":"2024-09-19T11:25:16.809866+0000","last_clean":"2024-09-19T11:25:16.809866+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:25:16.809866+0000","last_undegraded":"2024-09-19T11:25:16.809866+0000","last_fullsized":"2024-09-19T11:25:16.809866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:42.183684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'282","reported_seq":356,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327172+0000","last_change":"2024-09-19T11:23:11.263125+0000","last_active":"2024-09-19T11:26:12.327172+0000","last_peered":"2024-09-19T11:26:12.327172+0000","last_clean":"2024-09-19T11:26:12.327172+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:26:12.327172+0000","last_undegraded":"2024-09-19T11:26:12.327172+0000","last_fullsized":"2024-09-19T11:26:12.327172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:10.328770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946621+0000","last_change":"2024-09-19T11:23:13.269759+0000","last_active":"2024-09-19T11:23:42.946621+0000","last_peered":"2024-09-19T11:23:42.946621+0000","last_clean":"2024-09-19T11:23:42.946621+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:23:42.946621+0000","last_undegraded":"2024-09-19T11:23:42.946621+0000","last_fullsized":"2024-09-19T11:23:42.946621+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:46.820239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'11","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.025445+0000","last_change":"2024-09-19T11:23:05.254850+0000","last_active":"2024-09-19T11:26:12.025445+0000","last_peered":"2024-09-19T11:26:12.025445+0000","last_clean":"2024-09-19T11:26:12.025445+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:12.025445+0000","last_undegraded":"2024-09-19T11:26:12.025445+0000","last_fullsized":"2024-09-19T11:26:12.025445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:44.121192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948269+0000","last_change":"2024-09-19T11:23:03.241499+0000","last_active":"2024-09-19T11:23:42.948269+0000","last_peered":"2024-09-19T11:23:42.948269+0000","last_clean":"2024-09-19T11:23:42.948269+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:23:42.948269+0000","last_undegraded":"2024-09-19T11:23:42.948269+0000","last_fullsized":"2024-09-19T11:23:42.948269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:40.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280554+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.280554+0000","last_peered":"2024-09-19T11:23:42.280554+0000","last_clean":"2024-09-19T11:23:42.280554+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:23:42.280554+0000","last_undegraded":"2024-09-19T11:23:42.280554+0000","last_fullsized":"2024-09-19T11:23:42.280554+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:08:18.682837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946584+0000","last_change":"2024-09-19T11:23:07.245138+0000","last_active":"2024-09-19T11:23:42.946584+0000","last_peered":"2024-09-19T11:23:42.946584+0000","last_clean":"2024-09-19T11:23:42.946584+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:23:42.946584+0000","last_undegraded":"2024-09-19T11:23:42.946584+0000","last_fullsized":"2024-09-19T11:23:42.946584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:31.680535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'11","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945354+0000","last_change":"2024-09-19T11:23:09.256698+0000","last_active":"2024-09-19T11:23:42.945354+0000","last_peered":"2024-09-19T11:23:42.945354+0000","last_clean":"2024-09-19T11:23:42.945354+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:23:42.945354+0000","last_undegraded":"2024-09-19T11:23:42.945354+0000","last_fullsized":"2024-09-19T11:23:42.945354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:21.852554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'282","reported_seq":430,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585906+0000","last_change":"2024-09-19T11:23:11.258279+0000","last_active":"2024-09-19T11:26:12.585906+0000","last_peered":"2024-09-19T11:26:12.585906+0000","last_clean":"2024-09-19T11:26:12.585906+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:26:12.585906+0000","last_undegraded":"2024-09-19T11:26:12.585906+0000","last_fullsized":"2024-09-19T11:26:12.585906+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:24.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281484+0000","last_change":"2024-09-19T11:23:13.274470+0000","last_active":"2024-09-19T11:23:42.281484+0000","last_peered":"2024-09-19T11:23:42.281484+0000","last_clean":"2024-09-19T11:23:42.281484+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:23:42.281484+0000","last_undegraded":"2024-09-19T11:23:42.281484+0000","last_fullsized":"2024-09-19T11:23:42.281484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:33.500152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550363+0000","last_change":"2024-09-19T11:23:05.250199+0000","last_active":"2024-09-19T11:24:27.550363+0000","last_peered":"2024-09-19T11:24:27.550363+0000","last_clean":"2024-09-19T11:24:27.550363+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:24:27.550363+0000","last_undegraded":"2024-09-19T11:24:27.550363+0000","last_fullsized":"2024-09-19T11:24:27.550363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:20.188966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282786+0000","last_change":"2024-09-19T11:23:03.243876+0000","last_active":"2024-09-19T11:23:42.282786+0000","last_peered":"2024-09-19T11:23:42.282786+0000","last_clean":"2024-09-19T11:23:42.282786+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:23:42.282786+0000","last_undegraded":"2024-09-19T11:23:42.282786+0000","last_fullsized":"2024-09-19T11:23:42.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:15.709376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946831+0000","last_change":"2024-09-19T11:22:55.818050+0000","last_active":"2024-09-19T11:23:42.946831+0000","last_peered":"2024-09-19T11:23:42.946831+0000","last_clean":"2024-09-19T11:23:42.946831+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:23:42.946831+0000","last_undegraded":"2024-09-19T11:23:42.946831+0000","last_fullsized":"2024-09-19T11:23:42.946831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.933095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945462+0000","last_change":"2024-09-19T11:23:07.248324+0000","last_active":"2024-09-19T11:23:42.945462+0000","last_peered":"2024-09-19T11:23:42.945462+0000","last_clean":"2024-09-19T11:23:42.945462+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:23:42.945462+0000","last_undegraded":"2024-09-19T11:23:42.945462+0000","last_fullsized":"2024-09-19T11:23:42.945462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.767188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'82","reported_seq":155,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.748098+0000","last_change":"2024-09-19T11:23:09.258292+0000","last_active":"2024-09-19T11:25:09.748098+0000","last_peered":"2024-09-19T11:25:09.748098+0000","last_clean":"2024-09-19T11:25:09.748098+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:25:09.748098+0000","last_undegraded":"2024-09-19T11:25:09.748098+0000","last_fullsized":"2024-09-19T11:25:09.748098+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:51.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280109+0000","last_change":"2024-09-19T11:23:11.263614+0000","last_active":"2024-09-19T11:23:42.280109+0000","last_peered":"2024-09-19T11:23:42.280109+0000","last_clean":"2024-09-19T11:23:42.280109+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:23:42.280109+0000","last_undegraded":"2024-09-19T11:23:42.280109+0000","last_fullsized":"2024-09-19T11:23:42.280109+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.842570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278855+0000","last_change":"2024-09-19T11:23:13.265949+0000","last_active":"2024-09-19T11:23:42.278855+0000","last_peered":"2024-09-19T11:23:42.278855+0000","last_clean":"2024-09-19T11:23:42.278855+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:23:42.278855+0000","last_undegraded":"2024-09-19T11:23:42.278855+0000","last_fullsized":"2024-09-19T11:23:42.278855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:37.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278953+0000","last_change":"2024-09-19T11:23:13.271066+0000","last_active":"2024-09-19T11:23:42.278953+0000","last_peered":"2024-09-19T11:23:42.278953+0000","last_clean":"2024-09-19T11:23:42.278953+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:23:42.278953+0000","last_undegraded":"2024-09-19T11:23:42.278953+0000","last_fullsized":"2024-09-19T11:23:42.278953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:34.307629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277914+0000","last_change":"2024-09-19T11:23:05.245620+0000","last_active":"2024-09-19T11:23:42.277914+0000","last_peered":"2024-09-19T11:23:42.277914+0000","last_clean":"2024-09-19T11:23:42.277914+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:23:42.277914+0000","last_undegraded":"2024-09-19T11:23:42.277914+0000","last_fullsized":"2024-09-19T11:23:42.277914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.990710+0000","last_change":"2024-09-19T11:23:03.254634+0000","last_active":"2024-09-19T11:23:44.990710+0000","last_peered":"2024-09-19T11:23:44.990710+0000","last_clean":"2024-09-19T11:23:44.990710+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:23:44.990710+0000","last_undegraded":"2024-09-19T11:23:44.990710+0000","last_fullsized":"2024-09-19T11:23:44.990710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.416384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282381+0000","last_change":"2024-09-19T11:23:07.240696+0000","last_active":"2024-09-19T11:23:42.282381+0000","last_peered":"2024-09-19T11:23:42.282381+0000","last_clean":"2024-09-19T11:23:42.282381+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:23:42.282381+0000","last_undegraded":"2024-09-19T11:23:42.282381+0000","last_fullsized":"2024-09-19T11:23:42.282381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:02.659138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'32","reported_seq":76,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.754473+0000","last_change":"2024-09-19T11:23:09.259148+0000","last_active":"2024-09-19T11:25:09.754473+0000","last_peered":"2024-09-19T11:25:09.754473+0000","last_clean":"2024-09-19T11:25:09.754473+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:25:09.754473+0000","last_undegraded":"2024-09-19T11:25:09.754473+0000","last_fullsized":"2024-09-19T11:25:09.754473+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:29.000573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945264+0000","last_change":"2024-09-19T11:23:11.271128+0000","last_active":"2024-09-19T11:23:42.945264+0000","last_peered":"2024-09-19T11:23:42.945264+0000","last_clean":"2024-09-19T11:23:42.945264+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:23:42.945264+0000","last_undegraded":"2024-09-19T11:23:42.945264+0000","last_fullsized":"2024-09-19T11:23:42.945264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:35.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.427118+0000","last_change":"2024-09-19T11:23:13.273576+0000","last_active":"2024-09-19T11:25:09.427118+0000","last_peered":"2024-09-19T11:25:09.427118+0000","last_clean":"2024-09-19T11:25:09.427118+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:25:09.427118+0000","last_undegraded":"2024-09-19T11:25:09.427118+0000","last_fullsized":"2024-09-19T11:25:09.427118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:43.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.782138+0000","last_change":"2024-09-19T11:23:42.285785+0000","last_active":"2024-09-19T11:23:44.782138+0000","last_peered":"2024-09-19T11:23:44.782138+0000","last_clean":"2024-09-19T11:23:44.782138+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:23:44.782138+0000","last_undegraded":"2024-09-19T11:23:44.782138+0000","last_fullsized":"2024-09-19T11:23:44.782138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:15.907367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277946+0000","last_change":"2024-09-19T11:23:03.255902+0000","last_active":"2024-09-19T11:23:42.277946+0000","last_peered":"2024-09-19T11:23:42.277946+0000","last_clean":"2024-09-19T11:23:42.277946+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:23:42.277946+0000","last_undegraded":"2024-09-19T11:23:42.277946+0000","last_fullsized":"2024-09-19T11:23:42.277946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:41.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945177+0000","last_change":"2024-09-19T11:23:07.248908+0000","last_active":"2024-09-19T11:23:42.945177+0000","last_peered":"2024-09-19T11:23:42.945177+0000","last_clean":"2024-09-19T11:23:42.945177+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:23:42.945177+0000","last_undegraded":"2024-09-19T11:23:42.945177+0000","last_fullsized":"2024-09-19T11:23:42.945177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:57.835222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'5","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.535761+0000","last_change":"2024-09-19T11:23:09.244959+0000","last_active":"2024-09-19T11:26:11.535761+0000","last_peered":"2024-09-19T11:26:11.535761+0000","last_clean":"2024-09-19T11:26:11.535761+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:26:11.535761+0000","last_undegraded":"2024-09-19T11:26:11.535761+0000","last_fullsized":"2024-09-19T11:26:11.535761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:13.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277093+0000","last_change":"2024-09-19T11:23:11.268038+0000","last_active":"2024-09-19T11:23:42.277093+0000","last_peered":"2024-09-19T11:23:42.277093+0000","last_clean":"2024-09-19T11:23:42.277093+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:23:42.277093+0000","last_undegraded":"2024-09-19T11:23:42.277093+0000","last_fullsized":"2024-09-19T11:23:42.277093+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:40.232592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279462+0000","last_change":"2024-09-19T11:23:13.267238+0000","last_active":"2024-09-19T11:23:42.279462+0000","last_peered":"2024-09-19T11:23:42.279462+0000","last_clean":"2024-09-19T11:23:42.279462+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:23:42.279462+0000","last_undegraded":"2024-09-19T11:23:42.279462+0000","last_fullsized":"2024-09-19T11:23:42.279462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:36.252981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123419+0000","last_change":"2024-09-19T11:23:05.240162+0000","last_active":"2024-09-19T11:26:12.123419+0000","last_peered":"2024-09-19T11:26:12.123419+0000","last_clean":"2024-09-19T11:26:12.123419+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:12.123419+0000","last_undegraded":"2024-09-19T11:26:12.123419+0000","last_fullsized":"2024-09-19T11:26:12.123419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:45.738004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.261555+0000","last_change":"2024-09-19T11:23:03.257719+0000","last_active":"2024-09-19T11:26:12.261555+0000","last_peered":"2024-09-19T11:26:12.261555+0000","last_clean":"2024-09-19T11:26:12.261555+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:12.261555+0000","last_undegraded":"2024-09-19T11:26:12.261555+0000","last_fullsized":"2024-09-19T11:26:12.261555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:46.873725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277308+0000","last_change":"2024-09-19T11:23:07.243467+0000","last_active":"2024-09-19T11:23:42.277308+0000","last_peered":"2024-09-19T11:23:42.277308+0000","last_clean":"2024-09-19T11:23:42.277308+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:23:42.277308+0000","last_undegraded":"2024-09-19T11:23:42.277308+0000","last_fullsized":"2024-09-19T11:23:42.277308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:48.792369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'40","reported_seq":100,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.662362+0000","last_change":"2024-09-19T11:23:09.248715+0000","last_active":"2024-09-19T11:26:11.662362+0000","last_peered":"2024-09-19T11:26:11.662362+0000","last_clean":"2024-09-19T11:26:11.662362+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:26:11.662362+0000","last_undegraded":"2024-09-19T11:26:11.662362+0000","last_fullsized":"2024-09-19T11:26:11.662362+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:51.723342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280357+0000","last_change":"2024-09-19T11:23:11.262690+0000","last_active":"2024-09-19T11:23:42.280357+0000","last_peered":"2024-09-19T11:23:42.280357+0000","last_clean":"2024-09-19T11:23:42.280357+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:23:42.280357+0000","last_undegraded":"2024-09-19T11:23:42.280357+0000","last_fullsized":"2024-09-19T11:23:42.280357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:55.778119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.511055+0000","last_change":"2024-09-19T11:23:05.253736+0000","last_active":"2024-09-19T11:25:11.511055+0000","last_peered":"2024-09-19T11:25:11.511055+0000","last_clean":"2024-09-19T11:25:11.511055+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:25:11.511055+0000","last_undegraded":"2024-09-19T11:25:11.511055+0000","last_fullsized":"2024-09-19T11:25:11.511055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:09.050323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283174+0000","last_change":"2024-09-19T11:23:03.248284+0000","last_active":"2024-09-19T11:23:42.283174+0000","last_peered":"2024-09-19T11:23:42.283174+0000","last_clean":"2024-09-19T11:23:42.283174+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:23:42.283174+0000","last_undegraded":"2024-09-19T11:23:42.283174+0000","last_fullsized":"2024-09-19T11:23:42.283174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:40.686948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944123+0000","last_change":"2024-09-19T11:23:07.245862+0000","last_active":"2024-09-19T11:23:42.944123+0000","last_peered":"2024-09-19T11:23:42.944123+0000","last_clean":"2024-09-19T11:23:42.944123+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:23:42.944123+0000","last_undegraded":"2024-09-19T11:23:42.944123+0000","last_fullsized":"2024-09-19T11:23:42.944123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:09.692396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.739853+0000","last_change":"2024-09-19T11:23:09.247041+0000","last_active":"2024-09-19T11:25:09.739853+0000","last_peered":"2024-09-19T11:25:09.739853+0000","last_clean":"2024-09-19T11:25:09.739853+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:25:09.739853+0000","last_undegraded":"2024-09-19T11:25:09.739853+0000","last_fullsized":"2024-09-19T11:25:09.739853+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:21.727511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277107+0000","last_change":"2024-09-19T11:23:11.253755+0000","last_active":"2024-09-19T11:23:42.277107+0000","last_peered":"2024-09-19T11:23:42.277107+0000","last_clean":"2024-09-19T11:23:42.277107+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:23:42.277107+0000","last_undegraded":"2024-09-19T11:23:42.277107+0000","last_fullsized":"2024-09-19T11:23:42.277107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:51.209331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510416+0000","last_change":"2024-09-19T11:23:05.236615+0000","last_active":"2024-09-19T11:25:11.510416+0000","last_peered":"2024-09-19T11:25:11.510416+0000","last_clean":"2024-09-19T11:25:11.510416+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:25:11.510416+0000","last_undegraded":"2024-09-19T11:25:11.510416+0000","last_fullsized":"2024-09-19T11:25:11.510416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:36.749643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'3","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:31.812710+0000","last_change":"2024-09-19T11:23:03.255452+0000","last_active":"2024-09-19T11:25:31.812710+0000","last_peered":"2024-09-19T11:25:31.812710+0000","last_clean":"2024-09-19T11:25:31.812710+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:25:31.812710+0000","last_undegraded":"2024-09-19T11:25:31.812710+0000","last_fullsized":"2024-09-19T11:25:31.812710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:34.861274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796174+0000","last_change":"2024-09-19T11:23:42.945035+0000","last_active":"2024-09-19T11:23:44.796174+0000","last_peered":"2024-09-19T11:23:44.796174+0000","last_clean":"2024-09-19T11:23:44.796174+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:23:44.796174+0000","last_undegraded":"2024-09-19T11:23:44.796174+0000","last_fullsized":"2024-09-19T11:23:44.796174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'55","reported_seq":112,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:40.810485+0000","last_change":"2024-09-19T11:23:09.246587+0000","last_active":"2024-09-19T11:25:40.810485+0000","last_peered":"2024-09-19T11:25:40.810485+0000","last_clean":"2024-09-19T11:25:40.810485+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:25:40.810485+0000","last_undegraded":"2024-09-19T11:25:40.810485+0000","last_fullsized":"2024-09-19T11:25:40.810485+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:53.040162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":42,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278167+0000","last_change":"2024-09-19T11:23:11.260449+0000","last_active":"2024-09-19T11:23:42.278167+0000","last_peered":"2024-09-19T11:23:42.278167+0000","last_clean":"2024-09-19T11:23:42.278167+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:23:42.278167+0000","last_undegraded":"2024-09-19T11:23:42.278167+0000","last_fullsized":"2024-09-19T11:23:42.278167+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:30.983027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.419187+0000","last_change":"2024-09-19T11:23:13.268105+0000","last_active":"2024-09-19T11:25:11.419187+0000","last_peered":"2024-09-19T11:25:11.419187+0000","last_clean":"2024-09-19T11:25:11.419187+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:25:11.419187+0000","last_undegraded":"2024-09-19T11:25:11.419187+0000","last_fullsized":"2024-09-19T11:25:11.419187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:21.113571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368061+0000","last_change":"2024-09-19T11:23:05.246352+0000","last_active":"2024-09-19T11:26:12.368061+0000","last_peered":"2024-09-19T11:26:12.368061+0000","last_clean":"2024-09-19T11:26:12.368061+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:12.368061+0000","last_undegraded":"2024-09-19T11:26:12.368061+0000","last_fullsized":"2024-09-19T11:26:12.368061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.248495+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.959521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279359+0000","last_change":"2024-09-19T11:23:07.243165+0000","last_active":"2024-09-19T11:23:42.279359+0000","last_peered":"2024-09-19T11:23:42.279359+0000","last_clean":"2024-09-19T11:23:42.279359+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:23:42.279359+0000","last_undegraded":"2024-09-19T11:23:42.279359+0000","last_fullsized":"2024-09-19T11:23:42.279359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:21.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.740822+0000","last_change":"2024-09-19T11:23:09.256642+0000","last_active":"2024-09-19T11:25:09.740822+0000","last_peered":"2024-09-19T11:25:09.740822+0000","last_clean":"2024-09-19T11:25:09.740822+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:25:09.740822+0000","last_undegraded":"2024-09-19T11:25:09.740822+0000","last_fullsized":"2024-09-19T11:25:09.740822+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:42.797325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282300+0000","last_change":"2024-09-19T11:23:11.266817+0000","last_active":"2024-09-19T11:23:42.282300+0000","last_peered":"2024-09-19T11:23:42.282300+0000","last_clean":"2024-09-19T11:23:42.282300+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:23:42.282300+0000","last_undegraded":"2024-09-19T11:23:42.282300+0000","last_fullsized":"2024-09-19T11:23:42.282300+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:41:28.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947653+0000","last_change":"2024-09-19T11:23:05.254589+0000","last_active":"2024-09-19T11:23:42.947653+0000","last_peered":"2024-09-19T11:23:42.947653+0000","last_clean":"2024-09-19T11:23:42.947653+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:23:42.947653+0000","last_undegraded":"2024-09-19T11:23:42.947653+0000","last_fullsized":"2024-09-19T11:23:42.947653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:11.078441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280548+0000","last_change":"2024-09-19T11:23:03.244404+0000","last_active":"2024-09-19T11:23:42.280548+0000","last_peered":"2024-09-19T11:23:42.280548+0000","last_clean":"2024-09-19T11:23:42.280548+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:23:42.280548+0000","last_undegraded":"2024-09-19T11:23:42.280548+0000","last_fullsized":"2024-09-19T11:23:42.280548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:07.948011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278448+0000","last_change":"2024-09-19T11:23:07.242009+0000","last_active":"2024-09-19T11:23:42.278448+0000","last_peered":"2024-09-19T11:23:42.278448+0000","last_clean":"2024-09-19T11:23:42.278448+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:23:42.278448+0000","last_undegraded":"2024-09-19T11:23:42.278448+0000","last_fullsized":"2024-09-19T11:23:42.278448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:37.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'34","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.758462+0000","last_change":"2024-09-19T11:23:09.258190+0000","last_active":"2024-09-19T11:25:09.758462+0000","last_peered":"2024-09-19T11:25:09.758462+0000","last_clean":"2024-09-19T11:25:09.758462+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:25:09.758462+0000","last_undegraded":"2024-09-19T11:25:09.758462+0000","last_fullsized":"2024-09-19T11:25:09.758462+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:31.989630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":46,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280329+0000","last_change":"2024-09-19T11:23:11.261698+0000","last_active":"2024-09-19T11:23:42.280329+0000","last_peered":"2024-09-19T11:23:42.280329+0000","last_clean":"2024-09-19T11:23:42.280329+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:23:42.280329+0000","last_undegraded":"2024-09-19T11:23:42.280329+0000","last_fullsized":"2024-09-19T11:23:42.280329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:48.537146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032904+0000","last_change":"2024-09-19T11:23:05.238234+0000","last_active":"2024-09-19T11:25:08.032904+0000","last_peered":"2024-09-19T11:25:08.032904+0000","last_clean":"2024-09-19T11:25:08.032904+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:25:08.032904+0000","last_undegraded":"2024-09-19T11:25:08.032904+0000","last_fullsized":"2024-09-19T11:25:08.032904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:03:26.967357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944417+0000","last_change":"2024-09-19T11:23:03.253512+0000","last_active":"2024-09-19T11:23:42.944417+0000","last_peered":"2024-09-19T11:23:42.944417+0000","last_clean":"2024-09-19T11:23:42.944417+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:23:42.944417+0000","last_undegraded":"2024-09-19T11:23:42.944417+0000","last_fullsized":"2024-09-19T11:23:42.944417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:17.009054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.286788+0000","last_change":"2024-09-19T11:23:42.286200+0000","last_active":"2024-09-19T11:23:42.286788+0000","last_peered":"2024-09-19T11:23:42.286788+0000","last_clean":"2024-09-19T11:23:42.286788+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:23:42.286788+0000","last_undegraded":"2024-09-19T11:23:42.286788+0000","last_fullsized":"2024-09-19T11:23:42.286788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.397616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946537+0000","last_change":"2024-09-19T11:23:09.257194+0000","last_active":"2024-09-19T11:23:42.946537+0000","last_peered":"2024-09-19T11:23:42.946537+0000","last_clean":"2024-09-19T11:23:42.946537+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:23:42.946537+0000","last_undegraded":"2024-09-19T11:23:42.946537+0000","last_fullsized":"2024-09-19T11:23:42.946537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:10.690183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'282","reported_seq":418,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.372744+0000","last_change":"2024-09-19T11:23:11.268055+0000","last_active":"2024-09-19T11:26:12.372744+0000","last_peered":"2024-09-19T11:26:12.372744+0000","last_clean":"2024-09-19T11:26:12.372744+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:26:12.372744+0000","last_undegraded":"2024-09-19T11:26:12.372744+0000","last_fullsized":"2024-09-19T11:26:12.372744+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:33.997140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943313+0000","last_change":"2024-09-19T11:23:13.272264+0000","last_active":"2024-09-19T11:23:42.943313+0000","last_peered":"2024-09-19T11:23:42.943313+0000","last_clean":"2024-09-19T11:23:42.943313+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:23:42.943313+0000","last_undegraded":"2024-09-19T11:23:42.943313+0000","last_fullsized":"2024-09-19T11:23:42.943313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:34.821219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123209+0000","last_change":"2024-09-19T11:23:05.235764+0000","last_active":"2024-09-19T11:26:12.123209+0000","last_peered":"2024-09-19T11:26:12.123209+0000","last_clean":"2024-09-19T11:26:12.123209+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:12.123209+0000","last_undegraded":"2024-09-19T11:26:12.123209+0000","last_fullsized":"2024-09-19T11:26:12.123209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:41.326878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282378+0000","last_change":"2024-09-19T11:23:03.245357+0000","last_active":"2024-09-19T11:23:42.282378+0000","last_peered":"2024-09-19T11:23:42.282378+0000","last_clean":"2024-09-19T11:23:42.282378+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:23:42.282378+0000","last_undegraded":"2024-09-19T11:23:42.282378+0000","last_fullsized":"2024-09-19T11:23:42.282378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:23.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281352+0000","last_change":"2024-09-19T11:23:07.247890+0000","last_active":"2024-09-19T11:23:42.281352+0000","last_peered":"2024-09-19T11:23:42.281352+0000","last_clean":"2024-09-19T11:23:42.281352+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:23:42.281352+0000","last_undegraded":"2024-09-19T11:23:42.281352+0000","last_fullsized":"2024-09-19T11:23:42.281352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.412033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'36","reported_seq":82,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.731864+0000","last_change":"2024-09-19T11:23:09.246755+0000","last_active":"2024-09-19T11:25:09.731864+0000","last_peered":"2024-09-19T11:25:09.731864+0000","last_clean":"2024-09-19T11:25:09.731864+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:25:09.731864+0000","last_undegraded":"2024-09-19T11:25:09.731864+0000","last_fullsized":"2024-09-19T11:25:09.731864+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.229678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278129+0000","last_change":"2024-09-19T11:23:11.266713+0000","last_active":"2024-09-19T11:23:42.278129+0000","last_peered":"2024-09-19T11:23:42.278129+0000","last_clean":"2024-09-19T11:23:42.278129+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:23:42.278129+0000","last_undegraded":"2024-09-19T11:23:42.278129+0000","last_fullsized":"2024-09-19T11:23:42.278129+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:38.162577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283516+0000","last_change":"2024-09-19T11:23:13.268771+0000","last_active":"2024-09-19T11:23:42.283516+0000","last_peered":"2024-09-19T11:23:42.283516+0000","last_clean":"2024-09-19T11:23:42.283516+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:23:42.283516+0000","last_undegraded":"2024-09-19T11:23:42.283516+0000","last_fullsized":"2024-09-19T11:23:42.283516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:19.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266078+0000","last_change":"2024-09-19T11:23:05.236610+0000","last_active":"2024-09-19T11:26:12.266078+0000","last_peered":"2024-09-19T11:26:12.266078+0000","last_clean":"2024-09-19T11:26:12.266078+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:12.266078+0000","last_undegraded":"2024-09-19T11:26:12.266078+0000","last_fullsized":"2024-09-19T11:26:12.266078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:44.218259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946253+0000","last_change":"2024-09-19T11:23:03.258006+0000","last_active":"2024-09-19T11:23:42.946253+0000","last_peered":"2024-09-19T11:23:42.946253+0000","last_clean":"2024-09-19T11:23:42.946253+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:23:42.946253+0000","last_undegraded":"2024-09-19T11:23:42.946253+0000","last_fullsized":"2024-09-19T11:23:42.946253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:55.934419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278151+0000","last_change":"2024-09-19T11:23:07.247875+0000","last_active":"2024-09-19T11:23:42.278151+0000","last_peered":"2024-09-19T11:23:42.278151+0000","last_clean":"2024-09-19T11:23:42.278151+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:23:42.278151+0000","last_undegraded":"2024-09-19T11:23:42.278151+0000","last_fullsized":"2024-09-19T11:23:42.278151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:20.420374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'31","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.732852+0000","last_change":"2024-09-19T11:23:09.257850+0000","last_active":"2024-09-19T11:25:09.732852+0000","last_peered":"2024-09-19T11:25:09.732852+0000","last_clean":"2024-09-19T11:25:09.732852+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:25:09.732852+0000","last_undegraded":"2024-09-19T11:25:09.732852+0000","last_fullsized":"2024-09-19T11:25:09.732852+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:48.134834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943735+0000","last_change":"2024-09-19T11:23:11.271462+0000","last_active":"2024-09-19T11:23:42.943735+0000","last_peered":"2024-09-19T11:23:42.943735+0000","last_clean":"2024-09-19T11:23:42.943735+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:23:42.943735+0000","last_undegraded":"2024-09-19T11:23:42.943735+0000","last_fullsized":"2024-09-19T11:23:42.943735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.194246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368608+0000","last_change":"2024-09-19T11:23:42.282153+0000","last_active":"2024-09-19T11:26:12.368608+0000","last_peered":"2024-09-19T11:26:12.368608+0000","last_clean":"2024-09-19T11:26:12.368608+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:12.368608+0000","last_undegraded":"2024-09-19T11:26:12.368608+0000","last_fullsized":"2024-09-19T11:26:12.368608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.828536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.999397+0000","last_change":"2024-09-19T11:23:03.237858+0000","last_active":"2024-09-19T11:23:44.999397+0000","last_peered":"2024-09-19T11:23:44.999397+0000","last_clean":"2024-09-19T11:23:44.999397+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:23:44.999397+0000","last_undegraded":"2024-09-19T11:23:44.999397+0000","last_fullsized":"2024-09-19T11:23:44.999397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:38.094142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280358+0000","last_change":"2024-09-19T11:23:07.245080+0000","last_active":"2024-09-19T11:23:42.280358+0000","last_peered":"2024-09-19T11:23:42.280358+0000","last_clean":"2024-09-19T11:23:42.280358+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:23:42.280358+0000","last_undegraded":"2024-09-19T11:23:42.280358+0000","last_fullsized":"2024-09-19T11:23:42.280358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:41.815514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'26","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.756320+0000","last_change":"2024-09-19T11:23:09.258685+0000","last_active":"2024-09-19T11:25:09.756320+0000","last_peered":"2024-09-19T11:25:09.756320+0000","last_clean":"2024-09-19T11:25:09.756320+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:25:09.756320+0000","last_undegraded":"2024-09-19T11:25:09.756320+0000","last_fullsized":"2024-09-19T11:25:09.756320+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:33.508526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944317+0000","last_change":"2024-09-19T11:23:11.267904+0000","last_active":"2024-09-19T11:23:42.944317+0000","last_peered":"2024-09-19T11:23:42.944317+0000","last_clean":"2024-09-19T11:23:42.944317+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:23:42.944317+0000","last_undegraded":"2024-09-19T11:23:42.944317+0000","last_fullsized":"2024-09-19T11:23:42.944317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:16.874318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032486+0000","last_change":"2024-09-19T11:23:05.235573+0000","last_active":"2024-09-19T11:25:08.032486+0000","last_peered":"2024-09-19T11:25:08.032486+0000","last_clean":"2024-09-19T11:25:08.032486+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:25:08.032486+0000","last_undegraded":"2024-09-19T11:25:08.032486+0000","last_fullsized":"2024-09-19T11:25:08.032486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:07.062938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":46,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.996626+0000","last_change":"2024-09-19T11:23:03.249699+0000","last_active":"2024-09-19T11:23:44.996626+0000","last_peered":"2024-09-19T11:23:44.996626+0000","last_clean":"2024-09-19T11:23:44.996626+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:23:44.996626+0000","last_undegraded":"2024-09-19T11:23:44.996626+0000","last_fullsized":"2024-09-19T11:23:44.996626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:36.434981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'2","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.365932+0000","last_change":"2024-09-19T11:23:07.247476+0000","last_active":"2024-09-19T11:25:08.365932+0000","last_peered":"2024-09-19T11:25:08.365932+0000","last_clean":"2024-09-19T11:25:08.365932+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:25:08.365932+0000","last_undegraded":"2024-09-19T11:25:08.365932+0000","last_fullsized":"2024-09-19T11:25:08.365932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:03.848364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'34","reported_seq":79,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.561308+0000","last_change":"2024-09-19T11:23:09.255745+0000","last_active":"2024-09-19T11:26:11.561308+0000","last_peered":"2024-09-19T11:26:11.561308+0000","last_clean":"2024-09-19T11:26:11.561308+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:26:11.561308+0000","last_undegraded":"2024-09-19T11:26:11.561308+0000","last_fullsized":"2024-09-19T11:26:11.561308+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:35.538204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279071+0000","last_change":"2024-09-19T11:23:11.266117+0000","last_active":"2024-09-19T11:23:42.279071+0000","last_peered":"2024-09-19T11:23:42.279071+0000","last_clean":"2024-09-19T11:23:42.279071+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:23:42.279071+0000","last_undegraded":"2024-09-19T11:23:42.279071+0000","last_fullsized":"2024-09-19T11:23:42.279071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:22.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550414+0000","last_change":"2024-09-19T11:23:05.252243+0000","last_active":"2024-09-19T11:24:27.550414+0000","last_peered":"2024-09-19T11:24:27.550414+0000","last_clean":"2024-09-19T11:24:27.550414+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:24:27.550414+0000","last_undegraded":"2024-09-19T11:24:27.550414+0000","last_fullsized":"2024-09-19T11:24:27.550414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:18.179391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276311+0000","last_change":"2024-09-19T11:23:03.257178+0000","last_active":"2024-09-19T11:23:42.276311+0000","last_peered":"2024-09-19T11:23:42.276311+0000","last_clean":"2024-09-19T11:23:42.276311+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:23:42.276311+0000","last_undegraded":"2024-09-19T11:23:42.276311+0000","last_fullsized":"2024-09-19T11:23:42.276311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:13.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.024977+0000","last_change":"2024-09-19T11:23:05.238242+0000","last_active":"2024-09-19T11:26:12.024977+0000","last_peered":"2024-09-19T11:26:12.024977+0000","last_clean":"2024-09-19T11:26:12.024977+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:12.024977+0000","last_undegraded":"2024-09-19T11:26:12.024977+0000","last_fullsized":"2024-09-19T11:26:12.024977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278230+0000","last_change":"2024-09-19T11:23:03.238163+0000","last_active":"2024-09-19T11:23:42.278230+0000","last_peered":"2024-09-19T11:23:42.278230+0000","last_clean":"2024-09-19T11:23:42.278230+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:23:42.278230+0000","last_undegraded":"2024-09-19T11:23:42.278230+0000","last_fullsized":"2024-09-19T11:23:42.278230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:49:17.258919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342000+0000","last_change":"2024-09-19T11:23:05.237372+0000","last_active":"2024-09-19T11:24:27.342000+0000","last_peered":"2024-09-19T11:24:27.342000+0000","last_clean":"2024-09-19T11:24:27.342000+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:24:27.342000+0000","last_undegraded":"2024-09-19T11:24:27.342000+0000","last_fullsized":"2024-09-19T11:24:27.342000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:27.975988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940912+0000","last_change":"2024-09-19T11:23:03.256664+0000","last_active":"2024-09-19T11:23:43.940912+0000","last_peered":"2024-09-19T11:23:43.940912+0000","last_clean":"2024-09-19T11:23:43.940912+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:23:43.940912+0000","last_undegraded":"2024-09-19T11:23:43.940912+0000","last_fullsized":"2024-09-19T11:23:43.940912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:24.307449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.011182+0000","last_change":"2024-09-19T11:23:05.237761+0000","last_active":"2024-09-19T11:23:45.011182+0000","last_peered":"2024-09-19T11:23:45.011182+0000","last_clean":"2024-09-19T11:23:45.011182+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:23:45.011182+0000","last_undegraded":"2024-09-19T11:23:45.011182+0000","last_fullsized":"2024-09-19T11:23:45.011182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:29:24.776293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281499+0000","last_change":"2024-09-19T11:23:03.243433+0000","last_active":"2024-09-19T11:23:42.281499+0000","last_peered":"2024-09-19T11:23:42.281499+0000","last_clean":"2024-09-19T11:23:42.281499+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:23:42.281499+0000","last_undegraded":"2024-09-19T11:23:42.281499+0000","last_fullsized":"2024-09-19T11:23:42.281499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:45.982872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338525+0000","last_change":"2024-09-19T11:23:05.240645+0000","last_active":"2024-09-19T11:26:12.338525+0000","last_peered":"2024-09-19T11:26:12.338525+0000","last_clean":"2024-09-19T11:26:12.338525+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:12.338525+0000","last_undegraded":"2024-09-19T11:26:12.338525+0000","last_fullsized":"2024-09-19T11:26:12.338525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.997259+0000","last_change":"2024-09-19T11:23:03.234311+0000","last_active":"2024-09-19T11:26:11.997259+0000","last_peered":"2024-09-19T11:26:11.997259+0000","last_clean":"2024-09-19T11:26:11.997259+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:11.997259+0000","last_undegraded":"2024-09-19T11:26:11.997259+0000","last_fullsized":"2024-09-19T11:26:11.997259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:10.195241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342109+0000","last_change":"2024-09-19T11:23:05.251796+0000","last_active":"2024-09-19T11:24:27.342109+0000","last_peered":"2024-09-19T11:24:27.342109+0000","last_clean":"2024-09-19T11:24:27.342109+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:24:27.342109+0000","last_undegraded":"2024-09-19T11:24:27.342109+0000","last_fullsized":"2024-09-19T11:24:27.342109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:53.953637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281786+0000","last_change":"2024-09-19T11:23:03.248461+0000","last_active":"2024-09-19T11:23:42.281786+0000","last_peered":"2024-09-19T11:23:42.281786+0000","last_clean":"2024-09-19T11:23:42.281786+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:23:42.281786+0000","last_undegraded":"2024-09-19T11:23:42.281786+0000","last_fullsized":"2024-09-19T11:23:42.281786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:40.877834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279051+0000","last_change":"2024-09-19T11:23:05.248747+0000","last_active":"2024-09-19T11:23:42.279051+0000","last_peered":"2024-09-19T11:23:42.279051+0000","last_clean":"2024-09-19T11:23:42.279051+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:23:42.279051+0000","last_undegraded":"2024-09-19T11:23:42.279051+0000","last_fullsized":"2024-09-19T11:23:42.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.258514+0000","last_change":"2024-09-19T11:23:03.257927+0000","last_active":"2024-09-19T11:26:12.258514+0000","last_peered":"2024-09-19T11:26:12.258514+0000","last_clean":"2024-09-19T11:26:12.258514+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:12.258514+0000","last_undegraded":"2024-09-19T11:26:12.258514+0000","last_fullsized":"2024-09-19T11:26:12.258514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374459+0000","last_change":"2024-09-19T11:23:05.248818+0000","last_active":"2024-09-19T11:26:12.374459+0000","last_peered":"2024-09-19T11:26:12.374459+0000","last_clean":"2024-09-19T11:26:12.374459+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:12.374459+0000","last_undegraded":"2024-09-19T11:26:12.374459+0000","last_fullsized":"2024-09-19T11:26:12.374459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:41.343464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280250+0000","last_change":"2024-09-19T11:23:03.238591+0000","last_active":"2024-09-19T11:23:42.280250+0000","last_peered":"2024-09-19T11:23:42.280250+0000","last_clean":"2024-09-19T11:23:42.280250+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:23:42.280250+0000","last_undegraded":"2024-09-19T11:23:42.280250+0000","last_fullsized":"2024-09-19T11:23:42.280250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:42.869904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342282+0000","last_change":"2024-09-19T11:23:05.248696+0000","last_active":"2024-09-19T11:24:27.342282+0000","last_peered":"2024-09-19T11:24:27.342282+0000","last_clean":"2024-09-19T11:24:27.342282+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:24:27.342282+0000","last_undegraded":"2024-09-19T11:24:27.342282+0000","last_fullsized":"2024-09-19T11:24:27.342282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:37:55.133352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944801+0000","last_change":"2024-09-19T11:23:03.255783+0000","last_active":"2024-09-19T11:23:42.944801+0000","last_peered":"2024-09-19T11:23:42.944801+0000","last_clean":"2024-09-19T11:23:42.944801+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:23:42.944801+0000","last_undegraded":"2024-09-19T11:23:42.944801+0000","last_fullsized":"2024-09-19T11:23:42.944801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:24.578577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368100+0000","last_change":"2024-09-19T11:23:05.243797+0000","last_active":"2024-09-19T11:26:12.368100+0000","last_peered":"2024-09-19T11:26:12.368100+0000","last_clean":"2024-09-19T11:26:12.368100+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:26:12.368100+0000","last_undegraded":"2024-09-19T11:26:12.368100+0000","last_fullsized":"2024-09-19T11:26:12.368100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:33.524342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278358+0000","last_change":"2024-09-19T11:23:03.243321+0000","last_active":"2024-09-19T11:23:42.278358+0000","last_peered":"2024-09-19T11:23:42.278358+0000","last_clean":"2024-09-19T11:23:42.278358+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:23:42.278358+0000","last_undegraded":"2024-09-19T11:23:42.278358+0000","last_fullsized":"2024-09-19T11:23:42.278358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342015+0000","last_change":"2024-09-19T11:23:05.241366+0000","last_active":"2024-09-19T11:24:27.342015+0000","last_peered":"2024-09-19T11:24:27.342015+0000","last_clean":"2024-09-19T11:24:27.342015+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:24:27.342015+0000","last_undegraded":"2024-09-19T11:24:27.342015+0000","last_fullsized":"2024-09-19T11:24:27.342015+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.002195+0000","last_change":"2024-09-19T11:23:03.249920+0000","last_active":"2024-09-19T11:23:45.002195+0000","last_peered":"2024-09-19T11:23:45.002195+0000","last_clean":"2024-09-19T11:23:45.002195+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:23:45.002195+0000","last_undegraded":"2024-09-19T11:23:45.002195+0000","last_fullsized":"2024-09-19T11:23:45.002195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:45.629786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.482864+0000","last_change":"2024-09-19T11:23:05.246852+0000","last_active":"2024-09-19T11:23:45.482864+0000","last_peered":"2024-09-19T11:23:45.482864+0000","last_clean":"2024-09-19T11:23:45.482864+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:23:45.482864+0000","last_undegraded":"2024-09-19T11:23:45.482864+0000","last_fullsized":"2024-09-19T11:23:45.482864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.057981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278908+0000","last_change":"2024-09-19T11:23:03.243727+0000","last_active":"2024-09-19T11:23:42.278908+0000","last_peered":"2024-09-19T11:23:42.278908+0000","last_clean":"2024-09-19T11:23:42.278908+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:23:42.278908+0000","last_undegraded":"2024-09-19T11:23:42.278908+0000","last_fullsized":"2024-09-19T11:23:42.278908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:52:42.605160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433129+0000","last_change":"2024-09-19T11:23:05.245754+0000","last_active":"2024-09-19T11:23:44.433129+0000","last_peered":"2024-09-19T11:23:44.433129+0000","last_clean":"2024-09-19T11:23:44.433129+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:23:44.433129+0000","last_undegraded":"2024-09-19T11:23:44.433129+0000","last_fullsized":"2024-09-19T11:23:44.433129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:22.898396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:03.241382+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:01.554767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266030+0000","last_change":"2024-09-19T11:23:05.248307+0000","last_active":"2024-09-19T11:26:12.266030+0000","last_peered":"2024-09-19T11:26:12.266030+0000","last_clean":"2024-09-19T11:26:12.266030+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:12.266030+0000","last_undegraded":"2024-09-19T11:26:12.266030+0000","last_fullsized":"2024-09-19T11:26:12.266030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:44.091824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946371+0000","last_change":"2024-09-19T11:23:03.256127+0000","last_active":"2024-09-19T11:23:42.946371+0000","last_peered":"2024-09-19T11:23:42.946371+0000","last_clean":"2024-09-19T11:23:42.946371+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:23:42.946371+0000","last_undegraded":"2024-09-19T11:23:42.946371+0000","last_fullsized":"2024-09-19T11:23:42.946371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:03.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123790+0000","last_change":"2024-09-19T11:23:05.250503+0000","last_active":"2024-09-19T11:26:12.123790+0000","last_peered":"2024-09-19T11:26:12.123790+0000","last_clean":"2024-09-19T11:26:12.123790+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:26:12.123790+0000","last_undegraded":"2024-09-19T11:26:12.123790+0000","last_fullsized":"2024-09-19T11:26:12.123790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:46.547197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.993579+0000","last_change":"2024-09-19T11:23:03.259032+0000","last_active":"2024-09-19T11:23:44.993579+0000","last_peered":"2024-09-19T11:23:44.993579+0000","last_clean":"2024-09-19T11:23:44.993579+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:23:44.993579+0000","last_undegraded":"2024-09-19T11:23:44.993579+0000","last_fullsized":"2024-09-19T11:23:44.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:06.246922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368312+0000","last_change":"2024-09-19T11:23:05.248701+0000","last_active":"2024-09-19T11:26:12.368312+0000","last_peered":"2024-09-19T11:26:12.368312+0000","last_clean":"2024-09-19T11:26:12.368312+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:12.368312+0000","last_undegraded":"2024-09-19T11:26:12.368312+0000","last_fullsized":"2024-09-19T11:26:12.368312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:20.161230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947585+0000","last_change":"2024-09-19T11:23:03.252640+0000","last_active":"2024-09-19T11:23:42.947585+0000","last_peered":"2024-09-19T11:23:42.947585+0000","last_clean":"2024-09-19T11:23:42.947585+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:23:42.947585+0000","last_undegraded":"2024-09-19T11:23:42.947585+0000","last_fullsized":"2024-09-19T11:23:42.947585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:25.619002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'1","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:26.811813+0000","last_change":"2024-09-19T11:23:03.256515+0000","last_active":"2024-09-19T11:25:26.811813+0000","last_peered":"2024-09-19T11:25:26.811813+0000","last_clean":"2024-09-19T11:25:26.811813+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:25:26.811813+0000","last_undegraded":"2024-09-19T11:25:26.811813+0000","last_fullsized":"2024-09-19T11:25:26.811813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:08.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123873+0000","last_change":"2024-09-19T11:23:05.248691+0000","last_active":"2024-09-19T11:26:12.123873+0000","last_peered":"2024-09-19T11:26:12.123873+0000","last_clean":"2024-09-19T11:26:12.123873+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:12.123873+0000","last_undegraded":"2024-09-19T11:26:12.123873+0000","last_fullsized":"2024-09-19T11:26:12.123873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:20.035436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278475+0000","last_change":"2024-09-19T11:23:03.243676+0000","last_active":"2024-09-19T11:23:42.278475+0000","last_peered":"2024-09-19T11:23:42.278475+0000","last_clean":"2024-09-19T11:23:42.278475+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:23:42.278475+0000","last_undegraded":"2024-09-19T11:23:42.278475+0000","last_fullsized":"2024-09-19T11:23:42.278475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:37.230042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510844+0000","last_change":"2024-09-19T11:23:05.245906+0000","last_active":"2024-09-19T11:25:11.510844+0000","last_peered":"2024-09-19T11:25:11.510844+0000","last_clean":"2024-09-19T11:25:11.510844+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:25:11.510844+0000","last_undegraded":"2024-09-19T11:25:11.510844+0000","last_fullsized":"2024-09-19T11:25:11.510844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:19.236446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":182,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":38719488,"data_stored":38719488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673005,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40096,"kb_used_data":13072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331744,"statfs":{"total":96636764160,"available":96595705856,"internally_reserved":0,"allocated":13385728,"data_stored":13057068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673005,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":5408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":5537792,"data_stored":5255453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705709,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40300,"kb_used_data":13276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331540,"statfs":{"total":96636764160,"available":96595496960,"internally_reserved":0,"allocated":13594624,"data_stored":13323694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40128,"kb_used_data":8944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331712,"statfs":{"total":96636764160,"available":96595673088,"internally_reserved":0,"allocated":9158656,"data_stored":8862463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8712192,"data_stored":8712192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4517888,"data_stored":4517888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12906496,"data_stored":12906496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:26:20.321 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T11:26:20.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f91476d6640 1 Processor -- start 2024-09-19T11:26:20.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f91476d6640 1 -- start start 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f91476d6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f914006eaa0 0x7f91401326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f91476d6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91400729b0 con 0x7f914006dfb0 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f91476d6640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f91400714b0 con 0x7f914006eaa0 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f9145472640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f914006dfb0 0x7f914006e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41966/0 (socket says 172.21.15.22:41966) 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f9145472640 1 -- 172.21.15.22:0/2480029469 learned_addr learned my addr 172.21.15.22:0/2480029469 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.451+0000 7f9144c71640 1 --2- 172.21.15.22:0/2480029469 >> v2:172.21.15.22:3300/0 conn(0x7f914006eaa0 0x7f91401326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.452+0000 7f9137fff640 1 -- 172.21.15.22:0/2480029469 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960940305 0 0) 0x7f913c001110 con 0x7f914006dfb0 2024-09-19T11:26:20.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.452+0000 7f9137fff640 1 -- 172.21.15.22:0/2480029469 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f912c003520 con 0x7f914006dfb0 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.452+0000 7f9144c71640 1 -- 172.21.15.22:0/2480029469 >> v1:172.21.15.22:6789/0 conn(0x7f914006dfb0 legacy=0x7f914006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.452+0000 7f9144c71640 1 -- 172.21.15.22:0/2480029469 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91400725f0 con 0x7f914006eaa0 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.452+0000 7f9144c71640 1 --2- 172.21.15.22:0/2480029469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006eaa0 0x7f91401326a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9130009750 tx=0x7f913002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a35d25a3e58366 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f9137fff640 1 -- 172.21.15.22:0/2480029469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9130037070 con 0x7f914006eaa0 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f9137fff640 1 -- 172.21.15.22:0/2480029469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913002ae20 con 0x7f914006eaa0 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f9137fff640 1 -- 172.21.15.22:0/2480029469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9130032570 con 0x7f914006eaa0 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f91476d6640 1 -- 172.21.15.22:0/2480029469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006eaa0 msgr2=0x7f91401326a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f91476d6640 1 --2- 172.21.15.22:0/2480029469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006eaa0 0x7f91401326a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9130009750 tx=0x7f913002a1e0 comp rx=0 tx=0).stop 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f91476d6640 1 -- 172.21.15.22:0/2480029469 shutdown_connections 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f91476d6640 1 --2- 172.21.15.22:0/2480029469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006eaa0 0x7f91401326a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:20.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f91476d6640 1 -- 172.21.15.22:0/2480029469 >> 172.21.15.22:0/2480029469 conn(0x7f914009abc0 msgr2=0x7f9140098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:20.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f91476d6640 1 -- 172.21.15.22:0/2480029469 shutdown_connections 2024-09-19T11:26:20.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.453+0000 7f91476d6640 1 -- 172.21.15.22:0/2480029469 wait complete. 2024-09-19T11:26:20.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.454+0000 7f91476d6640 1 Processor -- start 2024-09-19T11:26:20.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.454+0000 7f91476d6640 1 -- start start 2024-09-19T11:26:20.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f91476d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006dfb0 0x7f91401c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:20.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f91476d6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9140071d70 con 0x7f914006dfb0 2024-09-19T11:26:20.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9145472640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006dfb0 0x7f91401c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:20.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9145472640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006dfb0 0x7f91401c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35134/0 (socket says 172.21.15.22:35134) 2024-09-19T11:26:20.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9145472640 1 -- 172.21.15.22:0/3094862474 learned_addr learned my addr 172.21.15.22:0/3094862474 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:20.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9145472640 1 -- 172.21.15.22:0/3094862474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914008c510 con 0x7f914006dfb0 2024-09-19T11:26:20.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9145472640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006dfb0 0x7f91401c3fd0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f913c00c680 tx=0x7f913c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:20.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9135ffb640 1 -- 172.21.15.22:0/3094862474 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f913c0058b0 con 0x7f914006dfb0 2024-09-19T11:26:20.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9135ffb640 1 -- 172.21.15.22:0/3094862474 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913c005a50 con 0x7f914006dfb0 2024-09-19T11:26:20.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914012a670 con 0x7f914006dfb0 2024-09-19T11:26:20.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f9135ffb640 1 -- 172.21.15.22:0/3094862474 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f913c005db0 con 0x7f914006dfb0 2024-09-19T11:26:20.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.455+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401c4510 con 0x7f914006dfb0 2024-09-19T11:26:20.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.456+0000 7f9135ffb640 1 -- 172.21.15.22:0/3094862474 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f913c007070 con 0x7f914006dfb0 2024-09-19T11:26:20.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.456+0000 7f9135ffb640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f912403e6c0 0x7f9124040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:20.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.456+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9140070fe0 con 0x7f914006dfb0 2024-09-19T11:26:20.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.456+0000 7f9135ffb640 1 -- 172.21.15.22:0/3094862474 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f913c0534c0 con 0x7f914006dfb0 2024-09-19T11:26:20.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.457+0000 7f9144c71640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f912403e6c0 0x7f9124040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:20.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.457+0000 7f9144c71640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f912403e6c0 0x7f9124040b80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9130034000 tx=0x7f913002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:20.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.461+0000 7f9135ffb640 1 -- 172.21.15.22:0/3094862474 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f91400729b0 con 0x7f914006dfb0 2024-09-19T11:26:20.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.652+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f914006e390 con 0x7f914006dfb0 2024-09-19T11:26:20.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.654+0000 7f9135ffb640 1 -- 172.21.15.22:0/3094862474 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+21142 (secure 0 0 0) 0x7f913c00aae0 con 0x7f914006dfb0 2024-09-19T11:26:20.655 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:26:20.655 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":33,"fsid":"200d5f3b-d410-4b92-82cf-1cbe85431f50","created":"2024-09-19T11:22:36.212681+0000","modified":"2024-09-19T11:23:42.273118+0000","last_up_change":"2024-09-19T11:22:45.784999+0000","last_in_change":"2024-09-19T11:22:37.307013+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T11:22:44.794902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-19T11:22:51.830554+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-09-19T11:23:01.216319+0000","flags":1,"flags_names":"hashpspool","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"client.0","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-09-19T11:23:03.567889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-09-19T11:23:05.539098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-09-19T11:23:07.313776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-09-19T11:23:09.266615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-09-19T11:23:11.332011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0cbb56a-4180-4b0e-948a-cb474e8486a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6801","nonce":3144434255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6803","nonce":3144434255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6807","nonce":3144434255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3144434255},{"type":"v1","addr":"172.21.15.22:6805","nonce":3144434255}]},"public_addr":"172.21.15.22:6801/3144434255","cluster_addr":"172.21.15.22:6803/3144434255","heartbeat_back_addr":"172.21.15.22:6807/3144434255","heartbeat_front_addr":"172.21.15.22:6805/3144434255","state":["exists","up"]},{"osd":1,"uuid":"f8538703-1f8e-48d3-8d2e-d423b3fb007a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6817","nonce":656119574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6819","nonce":656119574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6823","nonce":656119574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":656119574},{"type":"v1","addr":"172.21.15.28:6821","nonce":656119574}]},"public_addr":"172.21.15.28:6817/656119574","cluster_addr":"172.21.15.28:6819/656119574","heartbeat_back_addr":"172.21.15.28:6823/656119574","heartbeat_front_addr":"172.21.15.28:6821/656119574","state":["exists","up"]},{"osd":2,"uuid":"e738a240-46c1-4d91-98ec-1e2d64c3f437","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6801","nonce":2071226075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6803","nonce":2071226075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6807","nonce":2071226075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2071226075},{"type":"v1","addr":"172.21.15.28:6805","nonce":2071226075}]},"public_addr":"172.21.15.28:6801/2071226075","cluster_addr":"172.21.15.28:6803/2071226075","heartbeat_back_addr":"172.21.15.28:6807/2071226075","heartbeat_front_addr":"172.21.15.28:6805/2071226075","state":["exists","up"]},{"osd":3,"uuid":"a884f1f1-c119-4e37-b10b-9038bb154d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6809","nonce":3027134945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6811","nonce":3027134945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6815","nonce":3027134945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":3027134945},{"type":"v1","addr":"172.21.15.28:6813","nonce":3027134945}]},"public_addr":"172.21.15.28:6809/3027134945","cluster_addr":"172.21.15.28:6811/3027134945","heartbeat_back_addr":"172.21.15.28:6815/3027134945","heartbeat_front_addr":"172.21.15.28:6813/3027134945","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.039577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:44.013759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:42.894612+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T11:22:43.051197+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.12","mappings":[{"from":3,"to":2}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":2}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]},{"pgid":"5.17","mappings":[{"from":0,"to":2}]},{"pgid":"5.1d","mappings":[{"from":0,"to":2}]},{"pgid":"5.1e","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"client.0":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T11:26:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f912403e6c0 msgr2=0x7f9124040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f912403e6c0 0x7f9124040b80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9130034000 tx=0x7f913002e000 comp rx=0 tx=0).stop 2024-09-19T11:26:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006dfb0 msgr2=0x7f91401c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006dfb0 0x7f91401c3fd0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f913c00c680 tx=0x7f913c00cb40 comp rx=0 tx=0).stop 2024-09-19T11:26:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 shutdown_connections 2024-09-19T11:26:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f912403e6c0 0x7f9124040b80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 --2- 172.21.15.22:0/3094862474 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f914006dfb0 0x7f91401c3fd0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 >> 172.21.15.22:0/3094862474 conn(0x7f914009abc0 msgr2=0x7f914009b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 shutdown_connections 2024-09-19T11:26:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:20.656+0000 7f91476d6640 1 -- 172.21.15.22:0/3094862474 wait complete. 2024-09-19T11:26:21.615 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-09-19T11:26:21.670 INFO:tasks.ceph:Scrubbing osd.0 2024-09-19T11:26:21.670 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-19T11:26:21.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.787+0000 7f70a9524640 1 Processor -- start 2024-09-19T11:26:21.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.787+0000 7f70a9524640 1 -- start start 2024-09-19T11:26:21.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a9524640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70a412bec0 0x7f70a412c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:21.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a9524640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a4070c00 con 0x7f70a412c7e0 2024-09-19T11:26:21.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a9524640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f70a4070e70 con 0x7f70a412bec0 2024-09-19T11:26:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a2ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f70a412bec0 0x7f70a412c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a27fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f70a412c7e0 0x7f70a406dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41974/0 (socket says 172.21.15.22:41974) 2024-09-19T11:26:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a27fc640 1 -- 172.21.15.22:0/2367983254 learned_addr learned my addr 172.21.15.22:0/2367983254 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a2ffd640 1 -- 172.21.15.22:0/2367983254 >> v1:172.21.15.22:6789/0 conn(0x7f70a412c7e0 legacy=0x7f70a406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a2ffd640 1 -- 172.21.15.22:0/2367983254 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a406f930 con 0x7f70a412bec0 2024-09-19T11:26:21.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.788+0000 7f70a1ffb640 1 -- 172.21.15.22:0/2367983254 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1492248275 0 0) 0x7f70a4070c00 con 0x7f70a412c7e0 2024-09-19T11:26:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.789+0000 7f70a2ffd640 1 --2- 172.21.15.22:0/2367983254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a412c2a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f709400aea0 tx=0x7f709402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1541fbe9e8fbe719 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.789+0000 7f70a1ffb640 1 -- 172.21.15.22:0/2367983254 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7094037070 con 0x7f70a412bec0 2024-09-19T11:26:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.789+0000 7f70a1ffb640 1 -- 172.21.15.22:0/2367983254 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7094008440 con 0x7f70a412bec0 2024-09-19T11:26:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.789+0000 7f70a1ffb640 1 -- 172.21.15.22:0/2367983254 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7094008740 con 0x7f70a412bec0 2024-09-19T11:26:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.790+0000 7f70a9524640 1 -- 172.21.15.22:0/2367983254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 msgr2=0x7f70a412c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.790+0000 7f70a9524640 1 --2- 172.21.15.22:0/2367983254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a412c2a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f709400aea0 tx=0x7f709402b7b0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.790+0000 7f70a9524640 1 -- 172.21.15.22:0/2367983254 shutdown_connections 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.790+0000 7f70a9524640 1 --2- 172.21.15.22:0/2367983254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a412c2a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.790+0000 7f70a9524640 1 -- 172.21.15.22:0/2367983254 >> 172.21.15.22:0/2367983254 conn(0x7f70a409a9f0 msgr2=0x7f70a409ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.790+0000 7f70a9524640 1 -- 172.21.15.22:0/2367983254 shutdown_connections 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.790+0000 7f70a9524640 1 -- 172.21.15.22:0/2367983254 wait complete. 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.791+0000 7f70a9524640 1 Processor -- start 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.791+0000 7f70a9524640 1 -- start start 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.791+0000 7f70a9524640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a408f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.791+0000 7f70a9524640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4099930 con 0x7f70a412bec0 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.791+0000 7f70a2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a408f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.791+0000 7f70a2ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a408f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35144/0 (socket says 172.21.15.22:35144) 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.791+0000 7f70a2ffd640 1 -- 172.21.15.22:0/696423020 learned_addr learned my addr 172.21.15.22:0/696423020 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:21.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.792+0000 7f70a2ffd640 1 -- 172.21.15.22:0/696423020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a4090ad0 con 0x7f70a412bec0 2024-09-19T11:26:21.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.792+0000 7f70a2ffd640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a408f460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7094000f80 tx=0x7f7094004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:21.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.792+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7094037050 con 0x7f70a412bec0 2024-09-19T11:26:21.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.792+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7094038040 con 0x7f70a412bec0 2024-09-19T11:26:21.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.792+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a4091780 con 0x7f70a412bec0 2024-09-19T11:26:21.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.792+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7094032990 con 0x7f70a412bec0 2024-09-19T11:26:21.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.792+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a4091db0 con 0x7f70a412bec0 2024-09-19T11:26:21.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.793+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70a4099e70 con 0x7f70a412bec0 2024-09-19T11:26:21.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.793+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7094045020 con 0x7f70a412bec0 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.793+0000 7f709b7fe640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f708403e710 0x7f7084040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.793+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f7094043020 con 0x7f70a412bec0 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.794+0000 7f709b7fe640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f708404ab70 0x7f708404d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.794+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f709403fe50 con 0x7f708404ab70 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.794+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f709407b050 con 0x7f70a412bec0 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.794+0000 7f70a37fe640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f708404ab70 0x7f708404d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.794+0000 7f70a27fc640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f708403e710 0x7f7084040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.794+0000 7f70a37fe640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f708404ab70 0x7f708404d010 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.794+0000 7f70a27fc640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f708403e710 0x7f7084040bd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f708c000c00 tx=0x7f708c0078d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:21.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.795+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== osd.0 v2:172.21.15.22:6800/3144434255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f709403fe50 con 0x7f708404ab70 2024-09-19T11:26:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.807+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 --> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f70a4091b60 con 0x7f708404ab70 2024-09-19T11:26:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f709b7fe640 1 -- 172.21.15.22:0/696423020 <== osd.0 v2:172.21.15.22:6800/3144434255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f709c0097a0 con 0x7f708404ab70 2024-09-19T11:26:21.810 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T11:26:21.810 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T11:26:21.810 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T11:26:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f708404ab70 msgr2=0x7f708404d010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f708404ab70 0x7f708404d010 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f708403e710 msgr2=0x7f7084040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f708403e710 0x7f7084040bd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f708c000c00 tx=0x7f708c0078d0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 msgr2=0x7f70a408f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:21.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a408f460 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7094000f80 tx=0x7f7094004a40 comp rx=0 tx=0).stop 2024-09-19T11:26:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 shutdown_connections 2024-09-19T11:26:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:6800/3144434255,v1:172.21.15.22:6801/3144434255] conn(0x7f708404ab70 0x7f708404d010 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f708403e710 0x7f7084040bd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 --2- 172.21.15.22:0/696423020 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70a412bec0 0x7f70a408f460 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 >> 172.21.15.22:0/696423020 conn(0x7f70a409a9f0 msgr2=0x7f70a40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 shutdown_connections 2024-09-19T11:26:21.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.808+0000 7f70a9524640 1 -- 172.21.15.22:0/696423020 wait complete. 2024-09-19T11:26:21.820 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-19T11:26:21.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.921+0000 7f447d30e640 1 Processor -- start 2024-09-19T11:26:21.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.921+0000 7f447d30e640 1 -- start start 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f447d30e640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f447812cba0 0x7f44781315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f447d30e640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4478099e20 con 0x7f447812c0b0 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f447d30e640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f44780715a0 con 0x7f447812cba0 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f44767fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f447812cba0 0x7f44781315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f4476ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f447812c0b0 0x7f447812c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:41990/0 (socket says 172.21.15.22:41990) 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f4476ffd640 1 -- 172.21.15.22:0/3655153877 learned_addr learned my addr 172.21.15.22:0/3655153877 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f44767fc640 1 -- 172.21.15.22:0/3655153877 >> v1:172.21.15.22:6789/0 conn(0x7f447812c0b0 legacy=0x7f447812c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:21.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.922+0000 7f44767fc640 1 -- 172.21.15.22:0/3655153877 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44780705b0 con 0x7f447812cba0 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f44767fc640 1 --2- 172.21.15.22:0/3655153877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812cba0 0x7f44781315e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4460009750 tx=0x7f446002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c77b1c55acf8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f4475ffb640 1 -- 172.21.15.22:0/3655153877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4460037070 con 0x7f447812cba0 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f4475ffb640 1 -- 172.21.15.22:0/3655153877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f446002ad80 con 0x7f447812cba0 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f4475ffb640 1 -- 172.21.15.22:0/3655153877 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4460032590 con 0x7f447812cba0 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f447d30e640 1 -- 172.21.15.22:0/3655153877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812cba0 msgr2=0x7f44781315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f447d30e640 1 --2- 172.21.15.22:0/3655153877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812cba0 0x7f44781315e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f4460009750 tx=0x7f446002a140 comp rx=0 tx=0).stop 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f447d30e640 1 -- 172.21.15.22:0/3655153877 shutdown_connections 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f447d30e640 1 --2- 172.21.15.22:0/3655153877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812cba0 0x7f44781315e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f447d30e640 1 -- 172.21.15.22:0/3655153877 >> 172.21.15.22:0/3655153877 conn(0x7f447809abc0 msgr2=0x7f4478098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f447d30e640 1 -- 172.21.15.22:0/3655153877 shutdown_connections 2024-09-19T11:26:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.923+0000 7f447d30e640 1 -- 172.21.15.22:0/3655153877 wait complete. 2024-09-19T11:26:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.924+0000 7f447d30e640 1 Processor -- start 2024-09-19T11:26:21.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.924+0000 7f447d30e640 1 -- start start 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f447d30e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812c0b0 0x7f4478128c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f447d30e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4478131b20 con 0x7f447812c0b0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812c0b0 0x7f4478128c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f4476ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812c0b0 0x7f4478128c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35156/0 (socket says 172.21.15.22:35156) 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f4476ffd640 1 -- 172.21.15.22:0/3513699624 learned_addr learned my addr 172.21.15.22:0/3513699624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f4476ffd640 1 -- 172.21.15.22:0/3513699624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4478074090 con 0x7f447812c0b0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f4476ffd640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812c0b0 0x7f4478128c10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f447000cbd0 tx=0x7f447000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f446f7fe640 1 -- 172.21.15.22:0/3513699624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4470017070 con 0x7f447812c0b0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f446f7fe640 1 -- 172.21.15.22:0/3513699624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f447000abc0 con 0x7f447812c0b0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44781287d0 con 0x7f447812c0b0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f446f7fe640 1 -- 172.21.15.22:0/3513699624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4470012400 con 0x7f447812c0b0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.925+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4478126e20 con 0x7f447812c0b0 2024-09-19T11:26:21.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.926+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447812c490 con 0x7f447812c0b0 2024-09-19T11:26:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.926+0000 7f446f7fe640 1 -- 172.21.15.22:0/3513699624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f447001a070 con 0x7f447812c0b0 2024-09-19T11:26:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.927+0000 7f446f7fe640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f445803e710 0x7f4458040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.927+0000 7f446f7fe640 1 -- 172.21.15.22:0/3513699624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f4470053780 con 0x7f447812c0b0 2024-09-19T11:26:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.927+0000 7f44767fc640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f445803e710 0x7f4458040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:21.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.927+0000 7f44767fc640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f445803e710 0x7f4458040bd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4460009750 tx=0x7f446002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:21.931+0000 7f446f7fe640 1 -- 172.21.15.22:0/3513699624 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f4478074090 con 0x7f447812c0b0 2024-09-19T11:26:22.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.114+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f44780008d0 con 0x7f445803e710 2024-09-19T11:26:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.116+0000 7f446f7fe640 1 -- 172.21.15.22:0/3513699624 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f44780008d0 con 0x7f445803e710 2024-09-19T11:26:22.117 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 0 to deep-scrub 2024-09-19T11:26:22.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f445803e710 msgr2=0x7f4458040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f445803e710 0x7f4458040bd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4460009750 tx=0x7f446002e000 comp rx=0 tx=0).stop 2024-09-19T11:26:22.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812c0b0 msgr2=0x7f4478128c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812c0b0 0x7f4478128c10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f447000cbd0 tx=0x7f447000a430 comp rx=0 tx=0).stop 2024-09-19T11:26:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 shutdown_connections 2024-09-19T11:26:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f445803e710 0x7f4458040bd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 --2- 172.21.15.22:0/3513699624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f447812c0b0 0x7f4478128c10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 >> 172.21.15.22:0/3513699624 conn(0x7f447809abc0 msgr2=0x7f447806dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 shutdown_connections 2024-09-19T11:26:22.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.118+0000 7f447d30e640 1 -- 172.21.15.22:0/3513699624 wait complete. 2024-09-19T11:26:22.129 INFO:tasks.ceph:Scrubbing osd.1 2024-09-19T11:26:22.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-19T11:26:22.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.233+0000 7fda524b6640 1 Processor -- start 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.233+0000 7fda524b6640 1 -- start start 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.233+0000 7fda524b6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fda4c0974e0 0x7fda4c0978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.233+0000 7fda524b6640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda4c071e80 con 0x7fda4c097e00 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.233+0000 7fda524b6640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fda4c0715f0 con 0x7fda4c0974e0 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.233+0000 7fda4bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fda4c0974e0 0x7fda4c0978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.234+0000 7fda4b7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fda4c097e00 0x7fda4c06dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42002/0 (socket says 172.21.15.22:42002) 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.234+0000 7fda4b7fe640 1 -- 172.21.15.22:0/2414117196 learned_addr learned my addr 172.21.15.22:0/2414117196 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.234+0000 7fda4affd640 1 -- 172.21.15.22:0/2414117196 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682755035 0 0) 0x7fda4c071e80 con 0x7fda4c097e00 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.234+0000 7fda4affd640 1 -- 172.21.15.22:0/2414117196 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda34003520 con 0x7fda4c097e00 2024-09-19T11:26:22.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.234+0000 7fda4bfff640 1 -- 172.21.15.22:0/2414117196 >> v1:172.21.15.22:6789/0 conn(0x7fda4c097e00 legacy=0x7fda4c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.234+0000 7fda4bfff640 1 -- 172.21.15.22:0/2414117196 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c099ba0 con 0x7fda4c0974e0 2024-09-19T11:26:22.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda4bfff640 1 --2- 172.21.15.22:0/2414117196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c0978c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009720 tx=0x7fda3c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d4322090ffe48b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda4affd640 1 -- 172.21.15.22:0/2414117196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda3c037070 con 0x7fda4c0974e0 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda4affd640 1 -- 172.21.15.22:0/2414117196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3c02ad90 con 0x7fda4c0974e0 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda4affd640 1 -- 172.21.15.22:0/2414117196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda3c032590 con 0x7fda4c0974e0 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda524b6640 1 -- 172.21.15.22:0/2414117196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 msgr2=0x7fda4c0978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda524b6640 1 --2- 172.21.15.22:0/2414117196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c0978c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009720 tx=0x7fda3c02a110 comp rx=0 tx=0).stop 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda524b6640 1 -- 172.21.15.22:0/2414117196 shutdown_connections 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda524b6640 1 --2- 172.21.15.22:0/2414117196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c0978c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda524b6640 1 -- 172.21.15.22:0/2414117196 >> 172.21.15.22:0/2414117196 conn(0x7fda4c09a9f0 msgr2=0x7fda4c09ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda524b6640 1 -- 172.21.15.22:0/2414117196 shutdown_connections 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.235+0000 7fda524b6640 1 -- 172.21.15.22:0/2414117196 wait complete. 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.236+0000 7fda524b6640 1 Processor -- start 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.236+0000 7fda524b6640 1 -- start start 2024-09-19T11:26:22.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda524b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c1c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda524b6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c06e6c0 con 0x7fda4c0974e0 2024-09-19T11:26:22.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c1c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda4bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c1c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35174/0 (socket says 172.21.15.22:35174) 2024-09-19T11:26:22.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda4bfff640 1 -- 172.21.15.22:0/2814542956 learned_addr learned my addr 172.21.15.22:0/2814542956 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:22.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda4bfff640 1 -- 172.21.15.22:0/2814542956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c08d700 con 0x7fda4c0974e0 2024-09-19T11:26:22.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda4bfff640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c1c2260 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009720 tx=0x7fda3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda3c041070 con 0x7fda4c0974e0 2024-09-19T11:26:22.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda3c03c460 con 0x7fda4c0974e0 2024-09-19T11:26:22.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda3c037040 con 0x7fda4c0974e0 2024-09-19T11:26:22.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c073fd0 con 0x7fda4c0974e0 2024-09-19T11:26:22.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.237+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c1c27a0 con 0x7fda4c0974e0 2024-09-19T11:26:22.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.238+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fda4c06eba0 con 0x7fda4c0974e0 2024-09-19T11:26:22.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.238+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fda3c044070 con 0x7fda4c0974e0 2024-09-19T11:26:22.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda48ff9640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda3803e710 0x7fda38040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fda3c073ed0 con 0x7fda4c0974e0 2024-09-19T11:26:22.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda48ff9640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fda3804ab70 0x7fda3804d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fda3c007e80 con 0x7fda3804ab70 2024-09-19T11:26:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fda3c076580 con 0x7fda4c0974e0 2024-09-19T11:26:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda4b7fe640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda3803e710 0x7fda38040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda50a53640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fda3804ab70 0x7fda3804d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda4b7fe640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda3803e710 0x7fda38040bd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fda4c09b9d0 tx=0x7fda400078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.239+0000 7fda50a53640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fda3804ab70 0x7fda3804d010 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.240+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== osd.1 v2:172.21.15.28:6816/656119574 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fda3c007e80 con 0x7fda3804ab70 2024-09-19T11:26:22.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.253+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 --> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fda4c097e00 con 0x7fda3804ab70 2024-09-19T11:26:22.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.253+0000 7fda48ff9640 1 -- 172.21.15.22:0/2814542956 <== osd.1 v2:172.21.15.28:6816/656119574 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fda4c097e00 con 0x7fda3804ab70 2024-09-19T11:26:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T11:26:22.255 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T11:26:22.255 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T11:26:22.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.253+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fda3804ab70 msgr2=0x7fda3804d010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.253+0000 7fda524b6640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fda3804ab70 0x7fda3804d010 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.253+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda3803e710 msgr2=0x7fda38040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.253+0000 7fda524b6640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda3803e710 0x7fda38040bd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fda4c09b9d0 tx=0x7fda400078f0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 msgr2=0x7fda4c1c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c1c2260 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009720 tx=0x7fda3c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 shutdown_connections 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fda3803e710 0x7fda38040bd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.28:6816/656119574,v1:172.21.15.28:6817/656119574] conn(0x7fda3804ab70 0x7fda3804d010 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 --2- 172.21.15.22:0/2814542956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda4c0974e0 0x7fda4c1c2260 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 >> 172.21.15.22:0/2814542956 conn(0x7fda4c09a9f0 msgr2=0x7fda4c09ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 shutdown_connections 2024-09-19T11:26:22.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.254+0000 7fda524b6640 1 -- 172.21.15.22:0/2814542956 wait complete. 2024-09-19T11:26:22.265 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-19T11:26:22.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4c52fe640 1 Processor -- start 2024-09-19T11:26:22.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4c52fe640 1 -- start start 2024-09-19T11:26:22.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4c52fe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe4c012c0b0 0x7fe4c012c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4c52fe640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c009a290 con 0x7fe4c012c9d0 2024-09-19T11:26:22.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4c52fe640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c0071860 con 0x7fe4c012c0b0 2024-09-19T11:26:22.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4beffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe4c012c0b0 0x7fe4c012c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4b7fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe4c012c9d0 0x7fe4c0097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42018/0 (socket says 172.21.15.22:42018) 2024-09-19T11:26:22.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.367+0000 7fe4b7fff640 1 -- 172.21.15.22:0/2444975165 learned_addr learned my addr 172.21.15.22:0/2444975165 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:22.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.368+0000 7fe4beffd640 1 -- 172.21.15.22:0/2444975165 >> v1:172.21.15.22:6789/0 conn(0x7fe4c012c9d0 legacy=0x7fe4c0097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.368+0000 7fe4beffd640 1 -- 172.21.15.22:0/2444975165 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0099e20 con 0x7fe4c012c0b0 2024-09-19T11:26:22.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.368+0000 7fe4beffd640 1 --2- 172.21.15.22:0/2444975165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c012c490 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000b9f0 tx=0x7fe4b002c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ef9c6e2c52d937 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.368+0000 7fe4be7fc640 1 -- 172.21.15.22:0/2444975165 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b003a070 con 0x7fe4c012c0b0 2024-09-19T11:26:22.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.368+0000 7fe4be7fc640 1 -- 172.21.15.22:0/2444975165 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b0031070 con 0x7fe4c012c0b0 2024-09-19T11:26:22.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.368+0000 7fe4be7fc640 1 -- 172.21.15.22:0/2444975165 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b0032620 con 0x7fe4c012c0b0 2024-09-19T11:26:22.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.369+0000 7fe4c52fe640 1 -- 172.21.15.22:0/2444975165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 msgr2=0x7fe4c012c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.369+0000 7fe4c52fe640 1 --2- 172.21.15.22:0/2444975165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c012c490 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000b9f0 tx=0x7fe4b002c810 comp rx=0 tx=0).stop 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.369+0000 7fe4c52fe640 1 -- 172.21.15.22:0/2444975165 shutdown_connections 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.369+0000 7fe4c52fe640 1 --2- 172.21.15.22:0/2444975165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c012c490 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.369+0000 7fe4c52fe640 1 -- 172.21.15.22:0/2444975165 >> 172.21.15.22:0/2444975165 conn(0x7fe4c009abc0 msgr2=0x7fe4c0098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.369+0000 7fe4c52fe640 1 -- 172.21.15.22:0/2444975165 shutdown_connections 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.369+0000 7fe4c52fe640 1 -- 172.21.15.22:0/2444975165 wait complete. 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.370+0000 7fe4c52fe640 1 Processor -- start 2024-09-19T11:26:22.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.370+0000 7fe4c52fe640 1 -- start start 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.370+0000 7fe4c52fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c0127480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.370+0000 7fe4c52fe640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c012ce00 con 0x7fe4c012c0b0 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.370+0000 7fe4beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c0127480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.370+0000 7fe4beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c0127480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35202/0 (socket says 172.21.15.22:35202) 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.370+0000 7fe4beffd640 1 -- 172.21.15.22:0/89901313 learned_addr learned my addr 172.21.15.22:0/89901313 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.371+0000 7fe4beffd640 1 -- 172.21.15.22:0/89901313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0128b70 con 0x7fe4c012c0b0 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.371+0000 7fe4beffd640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c0127480 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000b9f0 tx=0x7fe4b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.371+0000 7fe4b77fe640 1 -- 172.21.15.22:0/89901313 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b003a040 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.371+0000 7fe4b77fe640 1 -- 172.21.15.22:0/89901313 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b0045070 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.371+0000 7fe4b77fe640 1 -- 172.21.15.22:0/89901313 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4b0035820 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.371+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c00904d0 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.371+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c0127ba0 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.372+0000 7fe4b77fe640 1 -- 172.21.15.22:0/89901313 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe4b0033070 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.372+0000 7fe4b77fe640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe4a003e760 0x7fe4a0040c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.372+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c012c490 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.372+0000 7fe4b77fe640 1 -- 172.21.15.22:0/89901313 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fe4b0074400 con 0x7fe4c012c0b0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.372+0000 7fe4b7fff640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe4a003e760 0x7fe4a0040c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.373+0000 7fe4b7fff640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe4a003e760 0x7fe4a0040c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8007fd0 tx=0x7fe4a8007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.377+0000 7fe4b77fe640 1 -- 172.21.15.22:0/89901313 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fe4b00493c0 con 0x7fe4c012c0b0 2024-09-19T11:26:22.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.565+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe4c00008d0 con 0x7fe4a003e760 2024-09-19T11:26:22.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.566+0000 7fe4b77fe640 1 -- 172.21.15.22:0/89901313 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe4c00008d0 con 0x7fe4a003e760 2024-09-19T11:26:22.567 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 1 to deep-scrub 2024-09-19T11:26:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe4a003e760 msgr2=0x7fe4a0040c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe4a003e760 0x7fe4a0040c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8007fd0 tx=0x7fe4a8007aa0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 msgr2=0x7fe4c0127480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c0127480 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000b9f0 tx=0x7fe4b0004730 comp rx=0 tx=0).stop 2024-09-19T11:26:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 shutdown_connections 2024-09-19T11:26:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe4a003e760 0x7fe4a0040c20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 --2- 172.21.15.22:0/89901313 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4c012c0b0 0x7fe4c0127480 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 >> 172.21.15.22:0/89901313 conn(0x7fe4c009abc0 msgr2=0x7fe4c0097550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 shutdown_connections 2024-09-19T11:26:22.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.569+0000 7fe4c52fe640 1 -- 172.21.15.22:0/89901313 wait complete. 2024-09-19T11:26:22.580 INFO:tasks.ceph:Scrubbing osd.2 2024-09-19T11:26:22.581 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-19T11:26:22.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc97722a640 1 Processor -- start 2024-09-19T11:26:22.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc97722a640 1 -- start start 2024-09-19T11:26:22.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc97722a640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc97006f4f0 0x7fc97006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc97722a640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc97006f0c0 con 0x7fc970070b40 2024-09-19T11:26:22.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc97722a640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fc970099ba0 con 0x7fc97006f4f0 2024-09-19T11:26:22.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc974fc6640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fc97006f4f0 0x7fc97006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc96ffff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fc970070b40 0x7fc97012bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42034/0 (socket says 172.21.15.22:42034) 2024-09-19T11:26:22.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.685+0000 7fc96ffff640 1 -- 172.21.15.22:0/2880475287 learned_addr learned my addr 172.21.15.22:0/2880475287 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:22.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.686+0000 7fc974fc6640 1 -- 172.21.15.22:0/2880475287 >> v1:172.21.15.22:6789/0 conn(0x7fc970070b40 legacy=0x7fc97012bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.686+0000 7fc96f7fe640 1 -- 172.21.15.22:0/2880475287 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 558796785 0 0) 0x7fc97006f0c0 con 0x7fc970070b40 2024-09-19T11:26:22.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.686+0000 7fc974fc6640 1 -- 172.21.15.22:0/2880475287 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9700712b0 con 0x7fc97006f4f0 2024-09-19T11:26:22.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.686+0000 7fc974fc6640 1 --2- 172.21.15.22:0/2880475287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc97006f8d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc964009720 tx=0x7fc96402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20dbe9a0084aa58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc96f7fe640 1 -- 172.21.15.22:0/2880475287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc964037070 con 0x7fc97006f4f0 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc96f7fe640 1 -- 172.21.15.22:0/2880475287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96402ad90 con 0x7fc97006f4f0 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc96f7fe640 1 -- 172.21.15.22:0/2880475287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc964032590 con 0x7fc97006f4f0 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc97722a640 1 -- 172.21.15.22:0/2880475287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 msgr2=0x7fc97006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc97722a640 1 --2- 172.21.15.22:0/2880475287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc97006f8d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc964009720 tx=0x7fc96402a110 comp rx=0 tx=0).stop 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc97722a640 1 -- 172.21.15.22:0/2880475287 shutdown_connections 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc97722a640 1 --2- 172.21.15.22:0/2880475287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc97006f8d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc97722a640 1 -- 172.21.15.22:0/2880475287 >> 172.21.15.22:0/2880475287 conn(0x7fc97009a9f0 msgr2=0x7fc97009ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc97722a640 1 -- 172.21.15.22:0/2880475287 shutdown_connections 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.687+0000 7fc97722a640 1 -- 172.21.15.22:0/2880475287 wait complete. 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.688+0000 7fc97722a640 1 Processor -- start 2024-09-19T11:26:22.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.688+0000 7fc97722a640 1 -- start start 2024-09-19T11:26:22.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc97722a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc9701c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc97722a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9700701a0 con 0x7fc97006f4f0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc974fc6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc9701c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc974fc6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc9701c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35214/0 (socket says 172.21.15.22:35214) 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc974fc6640 1 -- 172.21.15.22:0/1663943084 learned_addr learned my addr 172.21.15.22:0/1663943084 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc974fc6640 1 -- 172.21.15.22:0/1663943084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97008ec50 con 0x7fc97006f4f0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc974fc6640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc9701c20a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc964009720 tx=0x7fc9640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc964041070 con 0x7fc97006f4f0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc96403c460 con 0x7fc97006f4f0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97008f6a0 con 0x7fc97006f4f0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc964037040 con 0x7fc97006f4f0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.689+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9701c27c0 con 0x7fc97006f4f0 2024-09-19T11:26:22.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.690+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc97008f990 con 0x7fc97006f4f0 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.690+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fc964044070 con 0x7fc97006f4f0 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.690+0000 7fc96d7fa640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc95403e740 0x7fc954040c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.690+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fc9640742c0 con 0x7fc97006f4f0 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.691+0000 7fc96d7fa640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc95404ab40 0x7fc95404cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.691+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc96402fec0 con 0x7fc95404ab40 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.691+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc96407b050 con 0x7fc97006f4f0 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.691+0000 7fc96ffff640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc95403e740 0x7fc954040c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.691+0000 7fc9757c7640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc95404ab40 0x7fc95404cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.691+0000 7fc96ffff640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc95403e740 0x7fc954040c00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc95c005880 tx=0x7fc95c007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.691+0000 7fc9757c7640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc95404ab40 0x7fc95404cfe0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.692+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== osd.2 v2:172.21.15.28:6800/2071226075 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc96402fec0 con 0x7fc95404ab40 2024-09-19T11:26:22.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.705+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 --> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc970073e60 con 0x7fc95404ab40 2024-09-19T11:26:22.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.705+0000 7fc96d7fa640 1 -- 172.21.15.22:0/1663943084 <== osd.2 v2:172.21.15.28:6800/2071226075 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc96800ad10 con 0x7fc95404ab40 2024-09-19T11:26:22.707 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T11:26:22.707 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T11:26:22.707 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T11:26:22.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc95404ab40 msgr2=0x7fc95404cfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc95404ab40 0x7fc95404cfe0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc95403e740 msgr2=0x7fc954040c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc95403e740 0x7fc954040c00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc95c005880 tx=0x7fc95c007970 comp rx=0 tx=0).stop 2024-09-19T11:26:22.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 msgr2=0x7fc9701c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc9701c20a0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc964009720 tx=0x7fc9640046c0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 shutdown_connections 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fc95403e740 0x7fc954040c00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.28:6800/2071226075,v1:172.21.15.28:6801/2071226075] conn(0x7fc95404ab40 0x7fc95404cfe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 --2- 172.21.15.22:0/1663943084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc97006f4f0 0x7fc9701c20a0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 >> 172.21.15.22:0/1663943084 conn(0x7fc97009a9f0 msgr2=0x7fc970071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 shutdown_connections 2024-09-19T11:26:22.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.706+0000 7fc97722a640 1 -- 172.21.15.22:0/1663943084 wait complete. 2024-09-19T11:26:22.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-19T11:26:22.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.817+0000 7fdef0cef640 1 Processor -- start 2024-09-19T11:26:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.817+0000 7fdef0cef640 1 -- start start 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdef0cef640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdeec099bb0 0x7fdeec06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdef0cef640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeec09a290 con 0x7fdeec06e4f0 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdef0cef640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fdeec070490 con 0x7fdeec099bb0 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdeea59c640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fdeec099bb0 0x7fdeec06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdee9d9b640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fdeec06e4f0 0x7fdeec133c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42048/0 (socket says 172.21.15.22:42048) 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdee9d9b640 1 -- 172.21.15.22:0/1496542651 learned_addr learned my addr 172.21.15.22:0/1496542651 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdeea59c640 1 -- 172.21.15.22:0/1496542651 >> v1:172.21.15.22:6789/0 conn(0x7fdeec06e4f0 legacy=0x7fdeec133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.818+0000 7fdeea59c640 1 -- 172.21.15.22:0/1496542651 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec073e50 con 0x7fdeec099bb0 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdeea59c640 1 --2- 172.21.15.22:0/1496542651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec099bb0 0x7fdeec06dfb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009d90 tx=0x7fdedc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c601cf11a97f2b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdee959a640 1 -- 172.21.15.22:0/1496542651 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdedc007670 con 0x7fdeec099bb0 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdee959a640 1 -- 172.21.15.22:0/1496542651 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdedc007810 con 0x7fdeec099bb0 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdee959a640 1 -- 172.21.15.22:0/1496542651 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdedc007b10 con 0x7fdeec099bb0 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdef0cef640 1 -- 172.21.15.22:0/1496542651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec099bb0 msgr2=0x7fdeec06dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdef0cef640 1 --2- 172.21.15.22:0/1496542651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec099bb0 0x7fdeec06dfb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009d90 tx=0x7fdedc02a660 comp rx=0 tx=0).stop 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdef0cef640 1 -- 172.21.15.22:0/1496542651 shutdown_connections 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdef0cef640 1 --2- 172.21.15.22:0/1496542651 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec099bb0 0x7fdeec06dfb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.819+0000 7fdef0cef640 1 -- 172.21.15.22:0/1496542651 >> 172.21.15.22:0/1496542651 conn(0x7fdeec09abc0 msgr2=0x7fdeec098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.820+0000 7fdef0cef640 1 -- 172.21.15.22:0/1496542651 shutdown_connections 2024-09-19T11:26:22.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.820+0000 7fdef0cef640 1 -- 172.21.15.22:0/1496542651 wait complete. 2024-09-19T11:26:22.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.820+0000 7fdef0cef640 1 Processor -- start 2024-09-19T11:26:22.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdef0cef640 1 -- start start 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdef0cef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec06e4f0 0x7fdeec1a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdef0cef640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdeec06eac0 con 0x7fdeec06e4f0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdeea59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec06e4f0 0x7fdeec1a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdeea59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec06e4f0 0x7fdeec1a7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35244/0 (socket says 172.21.15.22:35244) 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdeea59c640 1 -- 172.21.15.22:0/974597298 learned_addr learned my addr 172.21.15.22:0/974597298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdeea59c640 1 -- 172.21.15.22:0/974597298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeec1aaef0 con 0x7fdeec06e4f0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.821+0000 7fdeea59c640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec06e4f0 0x7fdeec1a7f10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009d90 tx=0x7fdedc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.822+0000 7fdee2ffd640 1 -- 172.21.15.22:0/974597298 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdedc040070 con 0x7fdeec06e4f0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.822+0000 7fdee2ffd640 1 -- 172.21.15.22:0/974597298 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdedc031b10 con 0x7fdeec06e4f0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.822+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeec1a94b0 con 0x7fdeec06e4f0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.822+0000 7fdee2ffd640 1 -- 172.21.15.22:0/974597298 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdedc031df0 con 0x7fdeec06e4f0 2024-09-19T11:26:22.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.822+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeec1aa820 con 0x7fdeec06e4f0 2024-09-19T11:26:22.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.823+0000 7fdee2ffd640 1 -- 172.21.15.22:0/974597298 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fdedc039040 con 0x7fdeec06e4f0 2024-09-19T11:26:22.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.822+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeec06dfb0 con 0x7fdeec06e4f0 2024-09-19T11:26:22.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.823+0000 7fdee2ffd640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdecc03e710 0x7fdecc040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:22.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.823+0000 7fdee2ffd640 1 -- 172.21.15.22:0/974597298 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fdedc02ae00 con 0x7fdeec06e4f0 2024-09-19T11:26:22.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.823+0000 7fdee9d9b640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdecc03e710 0x7fdecc040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:22.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.824+0000 7fdee9d9b640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdecc03e710 0x7fdecc040bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fded4007fd0 tx=0x7fded4007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:22.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:22.827+0000 7fdee2ffd640 1 -- 172.21.15.22:0/974597298 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fdeec1aaef0 con 0x7fdeec06e4f0 2024-09-19T11:26:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.014+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fdeec050ee0 con 0x7fdecc03e710 2024-09-19T11:26:23.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.016+0000 7fdee2ffd640 1 -- 172.21.15.22:0/974597298 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdeec050ee0 con 0x7fdecc03e710 2024-09-19T11:26:23.017 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 2 to deep-scrub 2024-09-19T11:26:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdecc03e710 msgr2=0x7fdecc040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdecc03e710 0x7fdecc040bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fded4007fd0 tx=0x7fded4007aa0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec06e4f0 msgr2=0x7fdeec1a7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec06e4f0 0x7fdeec1a7f10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009d90 tx=0x7fdedc004a40 comp rx=0 tx=0).stop 2024-09-19T11:26:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 shutdown_connections 2024-09-19T11:26:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fdecc03e710 0x7fdecc040bd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 --2- 172.21.15.22:0/974597298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdeec06e4f0 0x7fdeec1a7f10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 >> 172.21.15.22:0/974597298 conn(0x7fdeec09abc0 msgr2=0x7fdeec1b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 shutdown_connections 2024-09-19T11:26:23.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.017+0000 7fdef0cef640 1 -- 172.21.15.22:0/974597298 wait complete. 2024-09-19T11:26:23.029 INFO:tasks.ceph:Scrubbing osd.3 2024-09-19T11:26:23.029 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.133+0000 7fe949447640 1 Processor -- start 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.133+0000 7fe949447640 1 -- start start 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe949447640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe944130470 0x7fe944132860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe949447640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe944097fd0 con 0x7fe9440974e0 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe949447640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fe9440715f0 con 0x7fe944130470 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe9427fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe944130470 0x7fe944132860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe9427fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fe944130470 0x7fe944132860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35256/0 (socket says 172.21.15.22:35256) 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe942ffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fe9440974e0 0x7fe9440978c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42064/0 (socket says 172.21.15.22:42064) 2024-09-19T11:26:23.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe9427fc640 1 -- 172.21.15.22:0/3633217183 learned_addr learned my addr 172.21.15.22:0/3633217183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe9427fc640 1 -- 172.21.15.22:0/3633217183 >> v1:172.21.15.22:6789/0 conn(0x7fe9440974e0 legacy=0x7fe9440978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.134+0000 7fe9427fc640 1 -- 172.21.15.22:0/3633217183 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe944071e80 con 0x7fe944130470 2024-09-19T11:26:23.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.135+0000 7fe9427fc640 1 --2- 172.21.15.22:0/3633217183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe944130470 0x7fe944132860 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0097f0 tx=0x7fe92c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea2b1a17bee5471 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.135+0000 7fe941ffb640 1 -- 172.21.15.22:0/3633217183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92c037070 con 0x7fe944130470 2024-09-19T11:26:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.135+0000 7fe941ffb640 1 -- 172.21.15.22:0/3633217183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c032440 con 0x7fe944130470 2024-09-19T11:26:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.135+0000 7fe941ffb640 1 -- 172.21.15.22:0/3633217183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92c02ae60 con 0x7fe944130470 2024-09-19T11:26:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.136+0000 7fe949447640 1 -- 172.21.15.22:0/3633217183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe944130470 msgr2=0x7fe944132860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.136+0000 7fe949447640 1 --2- 172.21.15.22:0/3633217183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe944130470 0x7fe944132860 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0097f0 tx=0x7fe92c02a1e0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.136+0000 7fe949447640 1 -- 172.21.15.22:0/3633217183 shutdown_connections 2024-09-19T11:26:23.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.136+0000 7fe949447640 1 --2- 172.21.15.22:0/3633217183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe944130470 0x7fe944132860 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.136+0000 7fe949447640 1 -- 172.21.15.22:0/3633217183 >> 172.21.15.22:0/3633217183 conn(0x7fe94409a9f0 msgr2=0x7fe94409ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.136+0000 7fe949447640 1 -- 172.21.15.22:0/3633217183 shutdown_connections 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.136+0000 7fe949447640 1 -- 172.21.15.22:0/3633217183 wait complete. 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe949447640 1 Processor -- start 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe949447640 1 -- start start 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe949447640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9440974e0 0x7fe944128f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe949447640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe9440723b0 con 0x7fe9440974e0 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe942ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9440974e0 0x7fe944128f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe942ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9440974e0 0x7fe944128f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35264/0 (socket says 172.21.15.22:35264) 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe942ffd640 1 -- 172.21.15.22:0/3980548760 learned_addr learned my addr 172.21.15.22:0/3980548760 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:23.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.137+0000 7fe942ffd640 1 -- 172.21.15.22:0/3980548760 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9440709b0 con 0x7fe9440974e0 2024-09-19T11:26:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.138+0000 7fe942ffd640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9440974e0 0x7fe944128f40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe93400c7c0 tx=0x7fe93400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.138+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe934017070 con 0x7fe9440974e0 2024-09-19T11:26:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.138+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe934005da0 con 0x7fe9440974e0 2024-09-19T11:26:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.138+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94406f8f0 con 0x7fe9440974e0 2024-09-19T11:26:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.138+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe93400a760 con 0x7fe9440974e0 2024-09-19T11:26:23.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.138+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe944129660 con 0x7fe9440974e0 2024-09-19T11:26:23.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9440978c0 con 0x7fe9440974e0 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fe93401a070 con 0x7fe9440974e0 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe93b7fe640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe92403e6c0 0x7fe924040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fe934052ed0 con 0x7fe9440974e0 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe93b7fe640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fe92404aaa0 0x7fe92404cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe934016ec0 con 0x7fe92404aaa0 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe934055600 con 0x7fe9440974e0 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe9427fc640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe92403e6c0 0x7fe924040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.139+0000 7fe9437fe640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fe92404aaa0 0x7fe92404cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.140+0000 7fe9437fe640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fe92404aaa0 0x7fe92404cf40 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.140+0000 7fe9427fc640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe92403e6c0 0x7fe924040b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe92c034000 tx=0x7fe92c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.141+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== osd.3 v2:172.21.15.28:6808/3027134945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe934016ec0 con 0x7fe92404aaa0 2024-09-19T11:26:23.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.153+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 --> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe94406fb50 con 0x7fe92404aaa0 2024-09-19T11:26:23.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.153+0000 7fe93b7fe640 1 -- 172.21.15.22:0/3980548760 <== osd.3 v2:172.21.15.28:6808/3027134945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe94406fb50 con 0x7fe92404aaa0 2024-09-19T11:26:23.155 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-19T11:26:23.155 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T11:26:23.155 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fe92404aaa0 msgr2=0x7fe92404cf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fe92404aaa0 0x7fe92404cf40 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe92403e6c0 msgr2=0x7fe924040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe92403e6c0 0x7fe924040b80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe92c034000 tx=0x7fe92c02e000 comp rx=0 tx=0).stop 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9440974e0 msgr2=0x7fe944128f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9440974e0 0x7fe944128f40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe93400c7c0 tx=0x7fe93400cc80 comp rx=0 tx=0).stop 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 shutdown_connections 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6808/3027134945,v1:172.21.15.28:6809/3027134945] conn(0x7fe92404aaa0 0x7fe92404cf40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fe92403e6c0 0x7fe924040b80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 --2- 172.21.15.22:0/3980548760 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9440974e0 0x7fe944128f40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 >> 172.21.15.22:0/3980548760 conn(0x7fe94409a9f0 msgr2=0x7fe94412b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 shutdown_connections 2024-09-19T11:26:23.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.154+0000 7fe949447640 1 -- 172.21.15.22:0/3980548760 wait complete. 2024-09-19T11:26:23.166 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-19T11:26:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f212ec65640 1 Processor -- start 2024-09-19T11:26:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f212ec65640 1 -- start start 2024-09-19T11:26:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f212ec65640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f212812c0b0 0x7f212812c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f212ec65640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f212809a290 con 0x7f212812c9d0 2024-09-19T11:26:23.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f212ec65640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f2128071860 con 0x7f212812c0b0 2024-09-19T11:26:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f2127fff640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f212812c9d0 0x7f2128097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42078/0 (socket says 172.21.15.22:42078) 2024-09-19T11:26:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f2127fff640 1 -- 172.21.15.22:0/4001952282 learned_addr learned my addr 172.21.15.22:0/4001952282 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f212ca01640 1 --2- 172.21.15.22:0/4001952282 >> v2:172.21.15.22:3300/0 conn(0x7f212812c0b0 0x7f212812c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f21277fe640 1 -- 172.21.15.22:0/4001952282 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519672239 0 0) 0x7f212809a290 con 0x7f212812c9d0 2024-09-19T11:26:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.268+0000 7f21277fe640 1 -- 172.21.15.22:0/4001952282 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2118003520 con 0x7f212812c9d0 2024-09-19T11:26:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.269+0000 7f212ca01640 1 -- 172.21.15.22:0/4001952282 >> v1:172.21.15.22:6789/0 conn(0x7f212812c9d0 legacy=0x7f2128097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.269+0000 7f212ca01640 1 -- 172.21.15.22:0/4001952282 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2128099e20 con 0x7f212812c0b0 2024-09-19T11:26:23.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.269+0000 7f212ca01640 1 --2- 172.21.15.22:0/4001952282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f212812c490 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f211c009750 tx=0x7f211c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf6d2d1e33cd23a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.269+0000 7f21277fe640 1 -- 172.21.15.22:0/4001952282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f211c037070 con 0x7f212812c0b0 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.269+0000 7f21277fe640 1 -- 172.21.15.22:0/4001952282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c02ad80 con 0x7f212812c0b0 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.269+0000 7f21277fe640 1 -- 172.21.15.22:0/4001952282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f211c032570 con 0x7f212812c0b0 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.270+0000 7f212ec65640 1 -- 172.21.15.22:0/4001952282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 msgr2=0x7f212812c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.270+0000 7f212ec65640 1 --2- 172.21.15.22:0/4001952282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f212812c490 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f211c009750 tx=0x7f211c02a140 comp rx=0 tx=0).stop 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.270+0000 7f212ec65640 1 -- 172.21.15.22:0/4001952282 shutdown_connections 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.270+0000 7f212ec65640 1 --2- 172.21.15.22:0/4001952282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f212812c490 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.270+0000 7f212ec65640 1 -- 172.21.15.22:0/4001952282 >> 172.21.15.22:0/4001952282 conn(0x7f212809abc0 msgr2=0x7f2128098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.270+0000 7f212ec65640 1 -- 172.21.15.22:0/4001952282 shutdown_connections 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.270+0000 7f212ec65640 1 -- 172.21.15.22:0/4001952282 wait complete. 2024-09-19T11:26:23.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.271+0000 7f212ec65640 1 Processor -- start 2024-09-19T11:26:23.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.271+0000 7f212ec65640 1 -- start start 2024-09-19T11:26:23.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ec65640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f2128125fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ec65640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f212812ce00 con 0x7f212812c0b0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ca01640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f2128125fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ca01640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f2128125fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35280/0 (socket says 172.21.15.22:35280) 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ca01640 1 -- 172.21.15.22:0/2492200346 learned_addr learned my addr 172.21.15.22:0/2492200346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ca01640 1 -- 172.21.15.22:0/2492200346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2128074090 con 0x7f212812c0b0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ca01640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f2128125fc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f211c009750 tx=0x7f211c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f21257fa640 1 -- 172.21.15.22:0/2492200346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f211c041020 con 0x7f212812c0b0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f21257fa640 1 -- 172.21.15.22:0/2492200346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c0044c0 con 0x7f212812c0b0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f21257fa640 1 -- 172.21.15.22:0/2492200346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f211c037040 con 0x7f212812c0b0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212806dfb0 con 0x7f212812c0b0 2024-09-19T11:26:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.272+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2128127aa0 con 0x7f212812c0b0 2024-09-19T11:26:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.273+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212812c490 con 0x7f212812c0b0 2024-09-19T11:26:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.274+0000 7f21257fa640 1 -- 172.21.15.22:0/2492200346 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f211c044020 con 0x7f212812c0b0 2024-09-19T11:26:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.274+0000 7f21257fa640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f210c03e710 0x7f210c040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.274+0000 7f21257fa640 1 -- 172.21.15.22:0/2492200346 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f211c073df0 con 0x7f212812c0b0 2024-09-19T11:26:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.274+0000 7f2127fff640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f210c03e710 0x7f210c040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.274+0000 7f2127fff640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f210c03e710 0x7f210c040bd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2114001610 tx=0x7f21140078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.278+0000 7f21257fa640 1 -- 172.21.15.22:0/2492200346 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f211c04b030 con 0x7f212812c0b0 2024-09-19T11:26:23.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.465+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f212806e2f0 con 0x7f210c03e710 2024-09-19T11:26:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.466+0000 7f21257fa640 1 -- 172.21.15.22:0/2492200346 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f212806e2f0 con 0x7f210c03e710 2024-09-19T11:26:23.469 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 3 to deep-scrub 2024-09-19T11:26:23.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f210c03e710 msgr2=0x7f210c040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f210c03e710 0x7f210c040bd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2114001610 tx=0x7f21140078f0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 msgr2=0x7f2128125fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f2128125fc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f211c009750 tx=0x7f211c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 shutdown_connections 2024-09-19T11:26:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f210c03e710 0x7f210c040bd0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 --2- 172.21.15.22:0/2492200346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f212812c0b0 0x7f2128125fc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.468+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 >> 172.21.15.22:0/2492200346 conn(0x7f212809abc0 msgr2=0x7f2128071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.469+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 shutdown_connections 2024-09-19T11:26:23.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.469+0000 7f212ec65640 1 -- 172.21.15.22:0/2492200346 wait complete. 2024-09-19T11:26:23.481 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:26:23.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.581+0000 7f277262b640 1 Processor -- start 2024-09-19T11:26:23.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.581+0000 7f277262b640 1 -- start start 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.581+0000 7f277262b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f276c06dfb0 0x7f276c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.581+0000 7f277262b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f276c071660 con 0x7f276c06e8d0 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.581+0000 7f277262b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f276c072550 con 0x7f276c06dfb0 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.582+0000 7f276bfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f276c06dfb0 0x7f276c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.582+0000 7f276b7fe640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f276c06e8d0 0x7f276c097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:42094/0 (socket says 172.21.15.22:42094) 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.582+0000 7f276b7fe640 1 -- 172.21.15.22:0/3640722206 learned_addr learned my addr 172.21.15.22:0/3640722206 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.582+0000 7f276bfff640 1 -- 172.21.15.22:0/3640722206 >> v1:172.21.15.22:6789/0 conn(0x7f276c06e8d0 legacy=0x7f276c097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.582+0000 7f276affd640 1 -- 172.21.15.22:0/3640722206 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112282298 0 0) 0x7f276c071660 con 0x7f276c06e8d0 2024-09-19T11:26:23.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.582+0000 7f276bfff640 1 -- 172.21.15.22:0/3640722206 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c099e20 con 0x7f276c06dfb0 2024-09-19T11:26:23.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f276bfff640 1 --2- 172.21.15.22:0/3640722206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c06e390 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f275c009720 tx=0x7f275c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6b321c85a475f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f276affd640 1 -- 172.21.15.22:0/3640722206 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f275c037070 con 0x7f276c06dfb0 2024-09-19T11:26:23.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f276affd640 1 -- 172.21.15.22:0/3640722206 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c02ae20 con 0x7f276c06dfb0 2024-09-19T11:26:23.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f276affd640 1 -- 172.21.15.22:0/3640722206 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f275c032590 con 0x7f276c06dfb0 2024-09-19T11:26:23.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f277262b640 1 -- 172.21.15.22:0/3640722206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 msgr2=0x7f276c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f277262b640 1 --2- 172.21.15.22:0/3640722206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c06e390 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f275c009720 tx=0x7f275c02a1a0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f277262b640 1 -- 172.21.15.22:0/3640722206 shutdown_connections 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f277262b640 1 --2- 172.21.15.22:0/3640722206 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c06e390 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f277262b640 1 -- 172.21.15.22:0/3640722206 >> 172.21.15.22:0/3640722206 conn(0x7f276c09abc0 msgr2=0x7f276c098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f277262b640 1 -- 172.21.15.22:0/3640722206 shutdown_connections 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.583+0000 7f277262b640 1 -- 172.21.15.22:0/3640722206 wait complete. 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.584+0000 7f277262b640 1 Processor -- start 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.584+0000 7f277262b640 1 -- start start 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f277262b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c1292a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f277262b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f276c06ece0 con 0x7f276c06dfb0 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f276bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c1292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f276bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c1292a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35292/0 (socket says 172.21.15.22:35292) 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f276bfff640 1 -- 172.21.15.22:0/4243495320 learned_addr learned my addr 172.21.15.22:0/4243495320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:23.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f276bfff640 1 -- 172.21.15.22:0/4243495320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c08c580 con 0x7f276c06dfb0 2024-09-19T11:26:23.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f276bfff640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c1292a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f275c035040 tx=0x7f275c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f2768ff9640 1 -- 172.21.15.22:0/4243495320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f275c037040 con 0x7f276c06dfb0 2024-09-19T11:26:23.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f2768ff9640 1 -- 172.21.15.22:0/4243495320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c032e30 con 0x7f276c06dfb0 2024-09-19T11:26:23.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c08cbc0 con 0x7f276c06dfb0 2024-09-19T11:26:23.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.585+0000 7f2768ff9640 1 -- 172.21.15.22:0/4243495320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f275c004160 con 0x7f276c06dfb0 2024-09-19T11:26:23.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.586+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c08d2b0 con 0x7f276c06dfb0 2024-09-19T11:26:23.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.586+0000 7f2768ff9640 1 -- 172.21.15.22:0/4243495320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f275c03a040 con 0x7f276c06dfb0 2024-09-19T11:26:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.586+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c08c120 con 0x7f276c06dfb0 2024-09-19T11:26:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.587+0000 7f2768ff9640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f274c03e710 0x7f274c040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.587+0000 7f2768ff9640 1 -- 172.21.15.22:0/4243495320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f275c073ff0 con 0x7f276c06dfb0 2024-09-19T11:26:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.587+0000 7f276b7fe640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f274c03e710 0x7f274c040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:23.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.588+0000 7f276b7fe640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f274c03e710 0x7f274c040bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2754000c00 tx=0x7f27540078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:23.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.591+0000 7f2768ff9640 1 -- 172.21.15.22:0/4243495320 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f276c08c120 con 0x7f276c06dfb0 2024-09-19T11:26:23.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.774+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f276c08c340 con 0x7f274c03e710 2024-09-19T11:26:23.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.785+0000 7f2768ff9640 1 -- 172.21.15.22:0/4243495320 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665680 (secure 0 0 0) 0x7f276c08c340 con 0x7f274c03e710 2024-09-19T11:26:23.794 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:26:23.800 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:26:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.789+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f274c03e710 msgr2=0x7f274c040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.789+0000 7f277262b640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f274c03e710 0x7f274c040bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2754000c00 tx=0x7f27540078f0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.789+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 msgr2=0x7f276c1292a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.789+0000 7f277262b640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c1292a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f275c035040 tx=0x7f275c0046c0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.790+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 shutdown_connections 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.790+0000 7f277262b640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f274c03e710 0x7f274c040bd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.790+0000 7f277262b640 1 --2- 172.21.15.22:0/4243495320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f276c06dfb0 0x7f276c1292a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.790+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 >> 172.21.15.22:0/4243495320 conn(0x7f276c09abc0 msgr2=0x7f276c12d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.790+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 shutdown_connections 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:23.790+0000 7f277262b640 1 -- 172.21.15.22:0/4243495320 wait complete. 2024-09-19T11:26:23.803 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-19T11:26:22.842879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605410,"num_objects":222,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":0,"num_read":5189,"num_read_kb":4731,"num_write":2121,"num_write_kb":60587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4564,"ondisk_log_size":4564,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":127432,"kb_used_data":15172,"kb_used_omap":6,"kb_used_meta":112185,"kb_avail":377359928,"statfs":{"total":386547056640,"available":386416566272,"internally_reserved":0,"allocated":15536128,"data_stored":14360391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":114878230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4281,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":750,"num_read_kb":697,"num_write":457,"num_write_kb":29372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004321"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940271+0000","last_change":"2024-09-19T11:23:03.259044+0000","last_active":"2024-09-19T11:23:43.940271+0000","last_peered":"2024-09-19T11:23:43.940271+0000","last_clean":"2024-09-19T11:23:43.940271+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:23:43.940271+0000","last_undegraded":"2024-09-19T11:23:43.940271+0000","last_fullsized":"2024-09-19T11:23:43.940271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:41.435159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333097+0000","last_change":"2024-09-19T11:23:05.248495+0000","last_active":"2024-09-19T11:26:12.333097+0000","last_peered":"2024-09-19T11:26:12.333097+0000","last_clean":"2024-09-19T11:26:12.333097+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:12.333097+0000","last_undegraded":"2024-09-19T11:26:12.333097+0000","last_fullsized":"2024-09-19T11:26:12.333097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:27.929797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.237576+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:20:44.729040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333061+0000","last_change":"2024-09-19T11:23:05.242656+0000","last_active":"2024-09-19T11:26:12.333061+0000","last_peered":"2024-09-19T11:26:12.333061+0000","last_clean":"2024-09-19T11:26:12.333061+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:26:12.333061+0000","last_undegraded":"2024-09-19T11:26:12.333061+0000","last_fullsized":"2024-09-19T11:26:12.333061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.791517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.022905+0000","last_change":"2024-09-19T11:23:03.252002+0000","last_active":"2024-09-19T11:25:08.022905+0000","last_peered":"2024-09-19T11:25:08.022905+0000","last_clean":"2024-09-19T11:25:08.022905+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:25:08.022905+0000","last_undegraded":"2024-09-19T11:25:08.022905+0000","last_fullsized":"2024-09-19T11:25:08.022905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:04:47.977517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'11","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470023+0000","last_change":"2024-09-19T11:23:42.285084+0000","last_active":"2024-09-19T11:26:12.470023+0000","last_peered":"2024-09-19T11:26:12.470023+0000","last_clean":"2024-09-19T11:26:12.470023+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:12.470023+0000","last_undegraded":"2024-09-19T11:26:12.470023+0000","last_fullsized":"2024-09-19T11:26:12.470023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:31.524562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.939807+0000","last_change":"2024-09-19T11:23:03.257377+0000","last_active":"2024-09-19T11:23:43.939807+0000","last_peered":"2024-09-19T11:23:43.939807+0000","last_clean":"2024-09-19T11:23:43.939807+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:23:43.939807+0000","last_undegraded":"2024-09-19T11:23:43.939807+0000","last_fullsized":"2024-09-19T11:23:43.939807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:25:06.788652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944750+0000","last_change":"2024-09-19T11:23:05.233162+0000","last_active":"2024-09-19T11:23:42.944750+0000","last_peered":"2024-09-19T11:23:42.944750+0000","last_clean":"2024-09-19T11:23:42.944750+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:23:42.944750+0000","last_undegraded":"2024-09-19T11:23:42.944750+0000","last_fullsized":"2024-09-19T11:23:42.944750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:49.026520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.891678+0000","last_change":"2024-09-19T11:23:03.254504+0000","last_active":"2024-09-19T11:26:11.891678+0000","last_peered":"2024-09-19T11:26:11.891678+0000","last_clean":"2024-09-19T11:26:11.891678+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:26:11.891678+0000","last_undegraded":"2024-09-19T11:26:11.891678+0000","last_fullsized":"2024-09-19T11:26:11.891678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:05.223629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'11","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.128818+0000","last_change":"2024-09-19T11:23:05.247147+0000","last_active":"2024-09-19T11:26:12.128818+0000","last_peered":"2024-09-19T11:26:12.128818+0000","last_clean":"2024-09-19T11:26:12.128818+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:12.128818+0000","last_undegraded":"2024-09-19T11:26:12.128818+0000","last_fullsized":"2024-09-19T11:26:12.128818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:25.162001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276791+0000","last_change":"2024-09-19T11:23:03.256689+0000","last_active":"2024-09-19T11:23:42.276791+0000","last_peered":"2024-09-19T11:23:42.276791+0000","last_clean":"2024-09-19T11:23:42.276791+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:23:42.276791+0000","last_undegraded":"2024-09-19T11:23:42.276791+0000","last_fullsized":"2024-09-19T11:23:42.276791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:09.931084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374918+0000","last_change":"2024-09-19T11:23:05.254262+0000","last_active":"2024-09-19T11:26:12.374918+0000","last_peered":"2024-09-19T11:26:12.374918+0000","last_clean":"2024-09-19T11:26:12.374918+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:26:12.374918+0000","last_undegraded":"2024-09-19T11:26:12.374918+0000","last_fullsized":"2024-09-19T11:26:12.374918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:58.352519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.252828+0000","last_change":"2024-09-19T11:23:03.239664+0000","last_active":"2024-09-19T11:26:12.252828+0000","last_peered":"2024-09-19T11:26:12.252828+0000","last_clean":"2024-09-19T11:26:12.252828+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:12.252828+0000","last_undegraded":"2024-09-19T11:26:12.252828+0000","last_fullsized":"2024-09-19T11:26:12.252828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:43.733405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333282+0000","last_change":"2024-09-19T11:23:05.250554+0000","last_active":"2024-09-19T11:26:12.333282+0000","last_peered":"2024-09-19T11:26:12.333282+0000","last_clean":"2024-09-19T11:26:12.333282+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:12.333282+0000","last_undegraded":"2024-09-19T11:26:12.333282+0000","last_fullsized":"2024-09-19T11:26:12.333282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278484+0000","last_change":"2024-09-19T11:23:03.237565+0000","last_active":"2024-09-19T11:23:42.278484+0000","last_peered":"2024-09-19T11:23:42.278484+0000","last_clean":"2024-09-19T11:23:42.278484+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:23:42.278484+0000","last_undegraded":"2024-09-19T11:23:42.278484+0000","last_fullsized":"2024-09-19T11:23:42.278484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:24:40.441642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.012417+0000","last_change":"2024-09-19T11:23:05.246773+0000","last_active":"2024-09-19T11:23:45.012417+0000","last_peered":"2024-09-19T11:23:45.012417+0000","last_clean":"2024-09-19T11:23:45.012417+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:23:45.012417+0000","last_undegraded":"2024-09-19T11:23:45.012417+0000","last_fullsized":"2024-09-19T11:23:45.012417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:05:02.041530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:05.241849+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:04.673024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283475+0000","last_change":"2024-09-19T11:23:03.250062+0000","last_active":"2024-09-19T11:23:42.283475+0000","last_peered":"2024-09-19T11:23:42.283475+0000","last_clean":"2024-09-19T11:23:42.283475+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:23:42.283475+0000","last_undegraded":"2024-09-19T11:23:42.283475+0000","last_fullsized":"2024-09-19T11:23:42.283475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:32.934189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.483130+0000","last_change":"2024-09-19T11:23:05.246402+0000","last_active":"2024-09-19T11:23:45.483130+0000","last_peered":"2024-09-19T11:23:45.483130+0000","last_clean":"2024-09-19T11:23:45.483130+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:23:45.483130+0000","last_undegraded":"2024-09-19T11:23:45.483130+0000","last_fullsized":"2024-09-19T11:23:45.483130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:44.280015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281849+0000","last_change":"2024-09-19T11:23:03.259838+0000","last_active":"2024-09-19T11:23:42.281849+0000","last_peered":"2024-09-19T11:23:42.281849+0000","last_clean":"2024-09-19T11:23:42.281849+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:23:42.281849+0000","last_undegraded":"2024-09-19T11:23:42.281849+0000","last_fullsized":"2024-09-19T11:23:42.281849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342502+0000","last_change":"2024-09-19T11:23:05.248326+0000","last_active":"2024-09-19T11:24:27.342502+0000","last_peered":"2024-09-19T11:24:27.342502+0000","last_clean":"2024-09-19T11:24:27.342502+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:24:27.342502+0000","last_undegraded":"2024-09-19T11:24:27.342502+0000","last_fullsized":"2024-09-19T11:24:27.342502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:28.400630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.969817+0000","last_change":"2024-09-19T11:23:03.250122+0000","last_active":"2024-09-19T11:26:11.969817+0000","last_peered":"2024-09-19T11:26:11.969817+0000","last_clean":"2024-09-19T11:26:11.969817+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:11.969817+0000","last_undegraded":"2024-09-19T11:26:11.969817+0000","last_fullsized":"2024-09-19T11:26:11.969817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:13.455767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123608+0000","last_change":"2024-09-19T11:23:05.240918+0000","last_active":"2024-09-19T11:26:12.123608+0000","last_peered":"2024-09-19T11:26:12.123608+0000","last_clean":"2024-09-19T11:26:12.123608+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:12.123608+0000","last_undegraded":"2024-09-19T11:26:12.123608+0000","last_fullsized":"2024-09-19T11:26:12.123608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:19.590735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282416+0000","last_change":"2024-09-19T11:23:03.245456+0000","last_active":"2024-09-19T11:23:42.282416+0000","last_peered":"2024-09-19T11:23:42.282416+0000","last_clean":"2024-09-19T11:23:42.282416+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:23:42.282416+0000","last_undegraded":"2024-09-19T11:23:42.282416+0000","last_fullsized":"2024-09-19T11:23:42.282416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:14.954640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266363+0000","last_change":"2024-09-19T11:23:05.245486+0000","last_active":"2024-09-19T11:26:12.266363+0000","last_peered":"2024-09-19T11:26:12.266363+0000","last_clean":"2024-09-19T11:26:12.266363+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:12.266363+0000","last_undegraded":"2024-09-19T11:26:12.266363+0000","last_fullsized":"2024-09-19T11:26:12.266363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.454541+0000","last_change":"2024-09-19T11:23:03.243483+0000","last_active":"2024-09-19T11:25:11.454541+0000","last_peered":"2024-09-19T11:25:11.454541+0000","last_clean":"2024-09-19T11:25:11.454541+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:25:11.454541+0000","last_undegraded":"2024-09-19T11:25:11.454541+0000","last_fullsized":"2024-09-19T11:25:11.454541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:10.061252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266347+0000","last_change":"2024-09-19T11:23:05.248431+0000","last_active":"2024-09-19T11:26:12.266347+0000","last_peered":"2024-09-19T11:26:12.266347+0000","last_clean":"2024-09-19T11:26:12.266347+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:12.266347+0000","last_undegraded":"2024-09-19T11:26:12.266347+0000","last_fullsized":"2024-09-19T11:26:12.266347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:17.337349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945193+0000","last_change":"2024-09-19T11:23:03.240547+0000","last_active":"2024-09-19T11:23:42.945193+0000","last_peered":"2024-09-19T11:23:42.945193+0000","last_clean":"2024-09-19T11:23:42.945193+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:23:42.945193+0000","last_undegraded":"2024-09-19T11:23:42.945193+0000","last_fullsized":"2024-09-19T11:23:42.945193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:55.218589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'37","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.305388+0000","last_change":"2024-09-19T11:23:13.275524+0000","last_active":"2024-09-19T11:26:12.305388+0000","last_peered":"2024-09-19T11:26:12.305388+0000","last_clean":"2024-09-19T11:26:12.305388+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:26:12.305388+0000","last_undegraded":"2024-09-19T11:26:12.305388+0000","last_fullsized":"2024-09-19T11:26:12.305388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.670701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.018412+0000","last_change":"2024-09-19T11:23:42.283032+0000","last_active":"2024-09-19T11:26:12.018412+0000","last_peered":"2024-09-19T11:26:12.018412+0000","last_clean":"2024-09-19T11:26:12.018412+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:12.018412+0000","last_undegraded":"2024-09-19T11:26:12.018412+0000","last_fullsized":"2024-09-19T11:26:12.018412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.760502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.929057+0000","last_change":"2024-09-19T11:23:03.258428+0000","last_active":"2024-09-19T11:26:11.929057+0000","last_peered":"2024-09-19T11:26:11.929057+0000","last_clean":"2024-09-19T11:26:11.929057+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:26:11.929057+0000","last_undegraded":"2024-09-19T11:26:11.929057+0000","last_fullsized":"2024-09-19T11:26:11.929057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:51.654373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945113+0000","last_change":"2024-09-19T11:23:07.245040+0000","last_active":"2024-09-19T11:23:42.945113+0000","last_peered":"2024-09-19T11:23:42.945113+0000","last_clean":"2024-09-19T11:23:42.945113+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:23:42.945113+0000","last_undegraded":"2024-09-19T11:23:42.945113+0000","last_fullsized":"2024-09-19T11:23:42.945113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.056714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.204815+0000","last_change":"2024-09-19T11:23:09.256765+0000","last_active":"2024-09-19T11:26:19.204815+0000","last_peered":"2024-09-19T11:26:19.204815+0000","last_clean":"2024-09-19T11:26:19.204815+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:26:19.204815+0000","last_undegraded":"2024-09-19T11:26:19.204815+0000","last_fullsized":"2024-09-19T11:26:19.204815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:39:32.451376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283356+0000","last_change":"2024-09-19T11:23:11.267021+0000","last_active":"2024-09-19T11:23:42.283356+0000","last_peered":"2024-09-19T11:23:42.283356+0000","last_clean":"2024-09-19T11:23:42.283356+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:23:42.283356+0000","last_undegraded":"2024-09-19T11:23:42.283356+0000","last_fullsized":"2024-09-19T11:23:42.283356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.806510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.707113+0000","last_change":"2024-09-19T11:23:13.274325+0000","last_active":"2024-09-19T11:26:11.707113+0000","last_peered":"2024-09-19T11:26:11.707113+0000","last_clean":"2024-09-19T11:26:11.707113+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:26:11.707113+0000","last_undegraded":"2024-09-19T11:26:11.707113+0000","last_fullsized":"2024-09-19T11:26:11.707113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:53:22.517816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.332597+0000","last_change":"2024-09-19T11:23:05.237978+0000","last_active":"2024-09-19T11:26:12.332597+0000","last_peered":"2024-09-19T11:26:12.332597+0000","last_clean":"2024-09-19T11:26:12.332597+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:12.332597+0000","last_undegraded":"2024-09-19T11:26:12.332597+0000","last_fullsized":"2024-09-19T11:26:12.332597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:02.964073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.319875+0000","last_change":"2024-09-19T11:23:03.256489+0000","last_active":"2024-09-19T11:25:11.319875+0000","last_peered":"2024-09-19T11:25:11.319875+0000","last_clean":"2024-09-19T11:25:11.319875+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:25:11.319875+0000","last_undegraded":"2024-09-19T11:25:11.319875+0000","last_fullsized":"2024-09-19T11:25:11.319875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:54:34.312130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279716+0000","last_change":"2024-09-19T11:23:07.245213+0000","last_active":"2024-09-19T11:23:42.279716+0000","last_peered":"2024-09-19T11:23:42.279716+0000","last_clean":"2024-09-19T11:23:42.279716+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:23:42.279716+0000","last_undegraded":"2024-09-19T11:23:42.279716+0000","last_fullsized":"2024-09-19T11:23:42.279716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:56.312564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'7","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.551668+0000","last_change":"2024-09-19T11:23:09.248257+0000","last_active":"2024-09-19T11:24:27.551668+0000","last_peered":"2024-09-19T11:24:27.551668+0000","last_clean":"2024-09-19T11:24:27.551668+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:24:27.551668+0000","last_undegraded":"2024-09-19T11:24:27.551668+0000","last_fullsized":"2024-09-19T11:24:27.551668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:18.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281824+0000","last_change":"2024-09-19T11:23:11.263544+0000","last_active":"2024-09-19T11:23:42.281824+0000","last_peered":"2024-09-19T11:23:42.281824+0000","last_clean":"2024-09-19T11:23:42.281824+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:23:42.281824+0000","last_undegraded":"2024-09-19T11:23:42.281824+0000","last_fullsized":"2024-09-19T11:23:42.281824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:44.652949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278313+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:23:42.278313+0000","last_peered":"2024-09-19T11:23:42.278313+0000","last_clean":"2024-09-19T11:23:42.278313+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:23:42.278313+0000","last_undegraded":"2024-09-19T11:23:42.278313+0000","last_fullsized":"2024-09-19T11:23:42.278313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:33:33.618068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510054+0000","last_change":"2024-09-19T11:23:05.236597+0000","last_active":"2024-09-19T11:25:11.510054+0000","last_peered":"2024-09-19T11:25:11.510054+0000","last_clean":"2024-09-19T11:25:11.510054+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:25:11.510054+0000","last_undegraded":"2024-09-19T11:25:11.510054+0000","last_fullsized":"2024-09-19T11:25:11.510054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:11.702021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.486679+0000","last_change":"2024-09-19T11:23:03.257697+0000","last_active":"2024-09-19T11:25:11.486679+0000","last_peered":"2024-09-19T11:25:11.486679+0000","last_clean":"2024-09-19T11:25:11.486679+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:25:11.486679+0000","last_undegraded":"2024-09-19T11:25:11.486679+0000","last_fullsized":"2024-09-19T11:25:11.486679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:55.338804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796210+0000","last_change":"2024-09-19T11:23:42.284049+0000","last_active":"2024-09-19T11:23:44.796210+0000","last_peered":"2024-09-19T11:23:44.796210+0000","last_clean":"2024-09-19T11:23:44.796210+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:23:44.796210+0000","last_undegraded":"2024-09-19T11:23:44.796210+0000","last_fullsized":"2024-09-19T11:23:44.796210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279160+0000","last_change":"2024-09-19T11:23:09.253591+0000","last_active":"2024-09-19T11:23:42.279160+0000","last_peered":"2024-09-19T11:23:42.279160+0000","last_clean":"2024-09-19T11:23:42.279160+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:23:42.279160+0000","last_undegraded":"2024-09-19T11:23:42.279160+0000","last_fullsized":"2024-09-19T11:23:42.279160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:52.970909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'285","reported_seq":389,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.073009+0000","last_change":"2024-09-19T11:23:11.271056+0000","last_active":"2024-09-19T11:26:19.073009+0000","last_peered":"2024-09-19T11:26:19.073009+0000","last_clean":"2024-09-19T11:26:19.073009+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:26:19.073009+0000","last_undegraded":"2024-09-19T11:26:19.073009+0000","last_fullsized":"2024-09-19T11:26:19.073009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:07.822761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.312802+0000","last_change":"2024-09-19T11:23:13.270157+0000","last_active":"2024-09-19T11:26:12.312802+0000","last_peered":"2024-09-19T11:26:12.312802+0000","last_clean":"2024-09-19T11:26:12.312802+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:26:12.312802+0000","last_undegraded":"2024-09-19T11:26:12.312802+0000","last_fullsized":"2024-09-19T11:26:12.312802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:32.371537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.781965+0000","last_change":"2024-09-19T11:23:05.248712+0000","last_active":"2024-09-19T11:23:44.781965+0000","last_peered":"2024-09-19T11:23:44.781965+0000","last_clean":"2024-09-19T11:23:44.781965+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:23:44.781965+0000","last_undegraded":"2024-09-19T11:23:44.781965+0000","last_fullsized":"2024-09-19T11:23:44.781965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:01:31.288073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279237+0000","last_change":"2024-09-19T11:23:03.250291+0000","last_active":"2024-09-19T11:23:42.279237+0000","last_peered":"2024-09-19T11:23:42.279237+0000","last_clean":"2024-09-19T11:23:42.279237+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:23:42.279237+0000","last_undegraded":"2024-09-19T11:23:42.279237+0000","last_fullsized":"2024-09-19T11:23:42.279237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:27.615110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947691+0000","last_change":"2024-09-19T11:23:07.248024+0000","last_active":"2024-09-19T11:23:42.947691+0000","last_peered":"2024-09-19T11:23:42.947691+0000","last_clean":"2024-09-19T11:23:42.947691+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:23:42.947691+0000","last_undegraded":"2024-09-19T11:23:42.947691+0000","last_fullsized":"2024-09-19T11:23:42.947691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:02.812050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.189031+0000","last_change":"2024-09-19T11:23:09.260866+0000","last_active":"2024-09-19T11:26:19.189031+0000","last_peered":"2024-09-19T11:26:19.189031+0000","last_clean":"2024-09-19T11:26:19.189031+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:26:19.189031+0000","last_undegraded":"2024-09-19T11:26:19.189031+0000","last_fullsized":"2024-09-19T11:26:19.189031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:22.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'285","reported_seq":405,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.078340+0000","last_change":"2024-09-19T11:23:11.260474+0000","last_active":"2024-09-19T11:26:19.078340+0000","last_peered":"2024-09-19T11:26:19.078340+0000","last_clean":"2024-09-19T11:26:19.078340+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:26:19.078340+0000","last_undegraded":"2024-09-19T11:26:19.078340+0000","last_fullsized":"2024-09-19T11:26:19.078340+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:32.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032873+0000","last_change":"2024-09-19T11:23:05.253843+0000","last_active":"2024-09-19T11:25:08.032873+0000","last_peered":"2024-09-19T11:25:08.032873+0000","last_clean":"2024-09-19T11:25:08.032873+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:25:08.032873+0000","last_undegraded":"2024-09-19T11:25:08.032873+0000","last_fullsized":"2024-09-19T11:25:08.032873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:22.232407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279601+0000","last_change":"2024-09-19T11:23:03.256377+0000","last_active":"2024-09-19T11:23:42.279601+0000","last_peered":"2024-09-19T11:23:42.279601+0000","last_clean":"2024-09-19T11:23:42.279601+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:23:42.279601+0000","last_undegraded":"2024-09-19T11:23:42.279601+0000","last_fullsized":"2024-09-19T11:23:42.279601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:29.305136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281170+0000","last_change":"2024-09-19T11:23:07.244371+0000","last_active":"2024-09-19T11:23:42.281170+0000","last_peered":"2024-09-19T11:23:42.281170+0000","last_clean":"2024-09-19T11:23:42.281170+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:23:42.281170+0000","last_undegraded":"2024-09-19T11:23:42.281170+0000","last_fullsized":"2024-09-19T11:23:42.281170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:53.267571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'43","reported_seq":95,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.217387+0000","last_change":"2024-09-19T11:23:09.249075+0000","last_active":"2024-09-19T11:26:19.217387+0000","last_peered":"2024-09-19T11:26:19.217387+0000","last_clean":"2024-09-19T11:26:19.217387+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:26:19.217387+0000","last_undegraded":"2024-09-19T11:26:19.217387+0000","last_fullsized":"2024-09-19T11:26:19.217387+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:03.176195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281316+0000","last_change":"2024-09-19T11:23:11.258275+0000","last_active":"2024-09-19T11:23:42.281316+0000","last_peered":"2024-09-19T11:23:42.281316+0000","last_clean":"2024-09-19T11:23:42.281316+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:23:42.281316+0000","last_undegraded":"2024-09-19T11:23:42.281316+0000","last_fullsized":"2024-09-19T11:23:42.281316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:14.017912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.349482+0000","last_change":"2024-09-19T11:23:13.274689+0000","last_active":"2024-09-19T11:24:27.349482+0000","last_peered":"2024-09-19T11:24:27.349482+0000","last_clean":"2024-09-19T11:24:27.349482+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:24:27.349482+0000","last_undegraded":"2024-09-19T11:24:27.349482+0000","last_fullsized":"2024-09-19T11:24:27.349482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:05.210803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281462+0000","last_change":"2024-09-19T11:23:05.245370+0000","last_active":"2024-09-19T11:23:42.281462+0000","last_peered":"2024-09-19T11:23:42.281462+0000","last_clean":"2024-09-19T11:23:42.281462+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:23:42.281462+0000","last_undegraded":"2024-09-19T11:23:42.281462+0000","last_fullsized":"2024-09-19T11:23:42.281462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:55:45.570918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280712+0000","last_change":"2024-09-19T11:23:03.257588+0000","last_active":"2024-09-19T11:23:42.280712+0000","last_peered":"2024-09-19T11:23:42.280712+0000","last_clean":"2024-09-19T11:23:42.280712+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:23:42.280712+0000","last_undegraded":"2024-09-19T11:23:42.280712+0000","last_fullsized":"2024-09-19T11:23:42.280712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.928791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278703+0000","last_change":"2024-09-19T11:23:07.247386+0000","last_active":"2024-09-19T11:23:42.278703+0000","last_peered":"2024-09-19T11:23:42.278703+0000","last_clean":"2024-09-19T11:23:42.278703+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:23:42.278703+0000","last_undegraded":"2024-09-19T11:23:42.278703+0000","last_fullsized":"2024-09-19T11:23:42.278703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:53.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'33","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.158685+0000","last_change":"2024-09-19T11:23:09.262668+0000","last_active":"2024-09-19T11:26:19.158685+0000","last_peered":"2024-09-19T11:26:19.158685+0000","last_clean":"2024-09-19T11:26:19.158685+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:26:19.158685+0000","last_undegraded":"2024-09-19T11:26:19.158685+0000","last_fullsized":"2024-09-19T11:26:19.158685+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:42.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278485+0000","last_change":"2024-09-19T11:23:11.265649+0000","last_active":"2024-09-19T11:23:42.278485+0000","last_peered":"2024-09-19T11:23:42.278485+0000","last_clean":"2024-09-19T11:23:42.278485+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:23:42.278485+0000","last_undegraded":"2024-09-19T11:23:42.278485+0000","last_fullsized":"2024-09-19T11:23:42.278485+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:22.946739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'5","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.124970+0000","last_change":"2024-09-19T11:23:13.268845+0000","last_active":"2024-09-19T11:26:12.124970+0000","last_peered":"2024-09-19T11:26:12.124970+0000","last_clean":"2024-09-19T11:26:12.124970+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:26:12.124970+0000","last_undegraded":"2024-09-19T11:26:12.124970+0000","last_fullsized":"2024-09-19T11:26:12.124970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:30.688522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342674+0000","last_change":"2024-09-19T11:23:05.251302+0000","last_active":"2024-09-19T11:24:27.342674+0000","last_peered":"2024-09-19T11:24:27.342674+0000","last_clean":"2024-09-19T11:24:27.342674+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:24:27.342674+0000","last_undegraded":"2024-09-19T11:24:27.342674+0000","last_fullsized":"2024-09-19T11:24:27.342674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:55.904446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.252317+0000","last_change":"2024-09-19T11:23:03.242612+0000","last_active":"2024-09-19T11:26:12.252317+0000","last_peered":"2024-09-19T11:26:12.252317+0000","last_clean":"2024-09-19T11:26:12.252317+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:12.252317+0000","last_undegraded":"2024-09-19T11:26:12.252317+0000","last_fullsized":"2024-09-19T11:26:12.252317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:48.666399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281855+0000","last_change":"2024-09-19T11:23:07.247594+0000","last_active":"2024-09-19T11:23:42.281855+0000","last_peered":"2024-09-19T11:23:42.281855+0000","last_clean":"2024-09-19T11:23:42.281855+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:23:42.281855+0000","last_undegraded":"2024-09-19T11:23:42.281855+0000","last_fullsized":"2024-09-19T11:23:42.281855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:41.108832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.222983+0000","last_change":"2024-09-19T11:23:09.259476+0000","last_active":"2024-09-19T11:26:19.222983+0000","last_peered":"2024-09-19T11:26:19.222983+0000","last_clean":"2024-09-19T11:26:19.222983+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:26:19.222983+0000","last_undegraded":"2024-09-19T11:26:19.222983+0000","last_fullsized":"2024-09-19T11:26:19.222983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:21.364894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'285","reported_seq":352,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.079494+0000","last_change":"2024-09-19T11:23:11.267050+0000","last_active":"2024-09-19T11:26:19.079494+0000","last_peered":"2024-09-19T11:26:19.079494+0000","last_clean":"2024-09-19T11:26:19.079494+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:26:19.079494+0000","last_undegraded":"2024-09-19T11:26:19.079494+0000","last_fullsized":"2024-09-19T11:26:19.079494+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.330204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.334604+0000","last_change":"2024-09-19T11:23:13.269663+0000","last_active":"2024-09-19T11:26:12.334604+0000","last_peered":"2024-09-19T11:26:12.334604+0000","last_clean":"2024-09-19T11:26:12.334604+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:26:12.334604+0000","last_undegraded":"2024-09-19T11:26:12.334604+0000","last_fullsized":"2024-09-19T11:26:12.334604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:57:11.742213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368223+0000","last_change":"2024-09-19T11:23:05.235632+0000","last_active":"2024-09-19T11:26:12.368223+0000","last_peered":"2024-09-19T11:26:12.368223+0000","last_clean":"2024-09-19T11:26:12.368223+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:12.368223+0000","last_undegraded":"2024-09-19T11:26:12.368223+0000","last_fullsized":"2024-09-19T11:26:12.368223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945253+0000","last_change":"2024-09-19T11:23:03.257570+0000","last_active":"2024-09-19T11:23:42.945253+0000","last_peered":"2024-09-19T11:23:42.945253+0000","last_clean":"2024-09-19T11:23:42.945253+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:23:42.945253+0000","last_undegraded":"2024-09-19T11:23:42.945253+0000","last_fullsized":"2024-09-19T11:23:42.945253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:41:29.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279784+0000","last_change":"2024-09-19T11:23:07.247239+0000","last_active":"2024-09-19T11:23:42.279784+0000","last_peered":"2024-09-19T11:23:42.279784+0000","last_clean":"2024-09-19T11:23:42.279784+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:23:42.279784+0000","last_undegraded":"2024-09-19T11:23:42.279784+0000","last_fullsized":"2024-09-19T11:23:42.279784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:21.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'61","reported_seq":126,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.167477+0000","last_change":"2024-09-19T11:23:09.259576+0000","last_active":"2024-09-19T11:26:19.167477+0000","last_peered":"2024-09-19T11:26:19.167477+0000","last_clean":"2024-09-19T11:26:19.167477+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:26:19.167477+0000","last_undegraded":"2024-09-19T11:26:19.167477+0000","last_fullsized":"2024-09-19T11:26:19.167477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.556001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278323+0000","last_change":"2024-09-19T11:23:11.260403+0000","last_active":"2024-09-19T11:23:42.278323+0000","last_peered":"2024-09-19T11:23:42.278323+0000","last_clean":"2024-09-19T11:23:42.278323+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:23:42.278323+0000","last_undegraded":"2024-09-19T11:23:42.278323+0000","last_fullsized":"2024-09-19T11:23:42.278323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:11.335945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'18","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585256+0000","last_change":"2024-09-19T11:23:13.273720+0000","last_active":"2024-09-19T11:26:12.585256+0000","last_peered":"2024-09-19T11:26:12.585256+0000","last_clean":"2024-09-19T11:26:12.585256+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:26:12.585256+0000","last_undegraded":"2024-09-19T11:26:12.585256+0000","last_fullsized":"2024-09-19T11:26:12.585256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:46.547393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:21.810844+0000","last_change":"2024-09-19T11:23:05.247032+0000","last_active":"2024-09-19T11:25:21.810844+0000","last_peered":"2024-09-19T11:25:21.810844+0000","last_clean":"2024-09-19T11:25:21.810844+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:25:21.810844+0000","last_undegraded":"2024-09-19T11:25:21.810844+0000","last_fullsized":"2024-09-19T11:25:21.810844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:24.496941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.255786+0000","last_change":"2024-09-19T11:23:03.234223+0000","last_active":"2024-09-19T11:26:12.255786+0000","last_peered":"2024-09-19T11:26:12.255786+0000","last_clean":"2024-09-19T11:26:12.255786+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:12.255786+0000","last_undegraded":"2024-09-19T11:26:12.255786+0000","last_fullsized":"2024-09-19T11:26:12.255786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278469+0000","last_change":"2024-09-19T11:23:07.244885+0000","last_active":"2024-09-19T11:23:42.278469+0000","last_peered":"2024-09-19T11:23:42.278469+0000","last_clean":"2024-09-19T11:23:42.278469+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:23:42.278469+0000","last_undegraded":"2024-09-19T11:23:42.278469+0000","last_fullsized":"2024-09-19T11:23:42.278469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:45:02.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'36","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.170832+0000","last_change":"2024-09-19T11:23:09.248190+0000","last_active":"2024-09-19T11:26:19.170832+0000","last_peered":"2024-09-19T11:26:19.170832+0000","last_clean":"2024-09-19T11:26:19.170832+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:26:19.170832+0000","last_undegraded":"2024-09-19T11:26:19.170832+0000","last_fullsized":"2024-09-19T11:26:19.170832+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:38.180379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946607+0000","last_change":"2024-09-19T11:23:11.270709+0000","last_active":"2024-09-19T11:23:42.946607+0000","last_peered":"2024-09-19T11:23:42.946607+0000","last_clean":"2024-09-19T11:23:42.946607+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:23:42.946607+0000","last_undegraded":"2024-09-19T11:23:42.946607+0000","last_fullsized":"2024-09-19T11:23:42.946607+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:39.157590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809883+0000","last_change":"2024-09-19T11:23:13.272542+0000","last_active":"2024-09-19T11:25:16.809883+0000","last_peered":"2024-09-19T11:25:16.809883+0000","last_clean":"2024-09-19T11:25:16.809883+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:25:16.809883+0000","last_undegraded":"2024-09-19T11:25:16.809883+0000","last_fullsized":"2024-09-19T11:25:16.809883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:52.793325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433382+0000","last_change":"2024-09-19T11:23:05.254593+0000","last_active":"2024-09-19T11:23:44.433382+0000","last_peered":"2024-09-19T11:23:44.433382+0000","last_clean":"2024-09-19T11:23:44.433382+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:23:44.433382+0000","last_undegraded":"2024-09-19T11:23:44.433382+0000","last_fullsized":"2024-09-19T11:23:44.433382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:27.816142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945690+0000","last_change":"2024-09-19T11:23:03.241691+0000","last_active":"2024-09-19T11:23:42.945690+0000","last_peered":"2024-09-19T11:23:42.945690+0000","last_clean":"2024-09-19T11:23:42.945690+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:23:42.945690+0000","last_undegraded":"2024-09-19T11:23:42.945690+0000","last_fullsized":"2024-09-19T11:23:42.945690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:21.286478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279479+0000","last_change":"2024-09-19T11:23:07.247411+0000","last_active":"2024-09-19T11:23:42.279479+0000","last_peered":"2024-09-19T11:23:42.279479+0000","last_clean":"2024-09-19T11:23:42.279479+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:23:42.279479+0000","last_undegraded":"2024-09-19T11:23:42.279479+0000","last_fullsized":"2024-09-19T11:23:42.279479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:20:41.691053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.148513+0000","last_change":"2024-09-19T11:23:09.246515+0000","last_active":"2024-09-19T11:26:19.148513+0000","last_peered":"2024-09-19T11:26:19.148513+0000","last_clean":"2024-09-19T11:26:19.148513+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:26:19.148513+0000","last_undegraded":"2024-09-19T11:26:19.148513+0000","last_fullsized":"2024-09-19T11:26:19.148513+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:54.969396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'285","reported_seq":407,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.074087+0000","last_change":"2024-09-19T11:23:11.263693+0000","last_active":"2024-09-19T11:26:19.074087+0000","last_peered":"2024-09-19T11:26:19.074087+0000","last_clean":"2024-09-19T11:26:19.074087+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:26:19.074087+0000","last_undegraded":"2024-09-19T11:26:19.074087+0000","last_fullsized":"2024-09-19T11:26:19.074087+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:29.980859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'14","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.475120+0000","last_change":"2024-09-19T11:23:13.265074+0000","last_active":"2024-09-19T11:26:12.475120+0000","last_peered":"2024-09-19T11:26:12.475120+0000","last_clean":"2024-09-19T11:26:12.475120+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:26:12.475120+0000","last_undegraded":"2024-09-19T11:26:12.475120+0000","last_fullsized":"2024-09-19T11:26:12.475120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:47.118156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368115+0000","last_change":"2024-09-19T11:23:05.250216+0000","last_active":"2024-09-19T11:26:12.368115+0000","last_peered":"2024-09-19T11:26:12.368115+0000","last_clean":"2024-09-19T11:26:12.368115+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:12.368115+0000","last_undegraded":"2024-09-19T11:26:12.368115+0000","last_fullsized":"2024-09-19T11:26:12.368115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.941399+0000","last_change":"2024-09-19T11:23:03.255564+0000","last_active":"2024-09-19T11:23:43.941399+0000","last_peered":"2024-09-19T11:23:43.941399+0000","last_clean":"2024-09-19T11:23:43.941399+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:23:43.941399+0000","last_undegraded":"2024-09-19T11:23:43.941399+0000","last_fullsized":"2024-09-19T11:23:43.941399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:10:03.443220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'3","reported_seq":418,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.054439+0000","last_change":"2024-09-19T11:23:07.242151+0000","last_active":"2024-09-19T11:26:19.054439+0000","last_peered":"2024-09-19T11:26:19.054439+0000","last_clean":"2024-09-19T11:26:19.054439+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:26:19.054439+0000","last_undegraded":"2024-09-19T11:26:19.054439+0000","last_fullsized":"2024-09-19T11:26:19.054439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:15.211742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946255+0000","last_change":"2024-09-19T11:23:09.255928+0000","last_active":"2024-09-19T11:23:42.946255+0000","last_peered":"2024-09-19T11:23:42.946255+0000","last_clean":"2024-09-19T11:23:42.946255+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:23:42.946255+0000","last_undegraded":"2024-09-19T11:23:42.946255+0000","last_fullsized":"2024-09-19T11:23:42.946255+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:04.689951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'285","reported_seq":469,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.075845+0000","last_change":"2024-09-19T11:23:11.255933+0000","last_active":"2024-09-19T11:26:19.075845+0000","last_peered":"2024-09-19T11:26:19.075845+0000","last_clean":"2024-09-19T11:26:19.075845+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:26:19.075845+0000","last_undegraded":"2024-09-19T11:26:19.075845+0000","last_fullsized":"2024-09-19T11:26:19.075845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:01.160503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.141728+0000","last_change":"2024-09-19T11:23:13.266957+0000","last_active":"2024-09-19T11:25:11.141728+0000","last_peered":"2024-09-19T11:25:11.141728+0000","last_clean":"2024-09-19T11:25:11.141728+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:25:11.141728+0000","last_undegraded":"2024-09-19T11:25:11.141728+0000","last_fullsized":"2024-09-19T11:25:11.141728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:59.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":101,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338578+0000","last_change":"2024-09-19T11:23:05.255344+0000","last_active":"2024-09-19T11:26:12.338578+0000","last_peered":"2024-09-19T11:26:12.338578+0000","last_clean":"2024-09-19T11:26:12.338578+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:12.338578+0000","last_undegraded":"2024-09-19T11:26:12.338578+0000","last_fullsized":"2024-09-19T11:26:12.338578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:22.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944743+0000","last_change":"2024-09-19T11:23:03.254989+0000","last_active":"2024-09-19T11:23:42.944743+0000","last_peered":"2024-09-19T11:23:42.944743+0000","last_clean":"2024-09-19T11:23:42.944743+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:23:42.944743+0000","last_undegraded":"2024-09-19T11:23:42.944743+0000","last_fullsized":"2024-09-19T11:23:42.944743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:39.840111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":420,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.055152+0000","last_change":"2024-09-19T11:23:07.250416+0000","last_active":"2024-09-19T11:26:19.055152+0000","last_peered":"2024-09-19T11:26:19.055152+0000","last_clean":"2024-09-19T11:26:19.055152+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:26:19.055152+0000","last_undegraded":"2024-09-19T11:26:19.055152+0000","last_fullsized":"2024-09-19T11:26:19.055152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":586,"num_read_kb":586,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'41","reported_seq":92,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.208185+0000","last_change":"2024-09-19T11:23:09.247733+0000","last_active":"2024-09-19T11:26:19.208185+0000","last_peered":"2024-09-19T11:26:19.208185+0000","last_clean":"2024-09-19T11:26:19.208185+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:26:19.208185+0000","last_undegraded":"2024-09-19T11:26:19.208185+0000","last_fullsized":"2024-09-19T11:26:19.208185+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:38.921761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279060+0000","last_change":"2024-09-19T11:23:11.266008+0000","last_active":"2024-09-19T11:23:42.279060+0000","last_peered":"2024-09-19T11:23:42.279060+0000","last_clean":"2024-09-19T11:23:42.279060+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:23:42.279060+0000","last_undegraded":"2024-09-19T11:23:42.279060+0000","last_fullsized":"2024-09-19T11:23:42.279060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:38:43.871781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.426809+0000","last_change":"2024-09-19T11:23:13.263896+0000","last_active":"2024-09-19T11:26:12.426809+0000","last_peered":"2024-09-19T11:26:12.426809+0000","last_clean":"2024-09-19T11:26:12.426809+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:26:12.426809+0000","last_undegraded":"2024-09-19T11:26:12.426809+0000","last_fullsized":"2024-09-19T11:26:12.426809+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:35.364893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'7","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.273438+0000","last_change":"2024-09-19T11:23:05.254167+0000","last_active":"2024-09-19T11:26:12.273438+0000","last_peered":"2024-09-19T11:26:12.273438+0000","last_clean":"2024-09-19T11:26:12.273438+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:12.273438+0000","last_undegraded":"2024-09-19T11:26:12.273438+0000","last_fullsized":"2024-09-19T11:26:12.273438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:44.425539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":146,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279138+0000","last_change":"2024-09-19T11:22:46.832286+0000","last_active":"2024-09-19T11:23:42.279138+0000","last_peered":"2024-09-19T11:23:42.279138+0000","last_clean":"2024-09-19T11:23:42.279138+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:23:42.279138+0000","last_undegraded":"2024-09-19T11:23:42.279138+0000","last_fullsized":"2024-09-19T11:23:42.279138+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:45.784999+0000","last_clean_scrub_stamp":"2024-09-19T11:22:45.784999+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:57:24.575130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944159+0000","last_change":"2024-09-19T11:22:55.817718+0000","last_active":"2024-09-19T11:23:42.944159+0000","last_peered":"2024-09-19T11:23:42.944159+0000","last_clean":"2024-09-19T11:23:42.944159+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:23:42.944159+0000","last_undegraded":"2024-09-19T11:23:42.944159+0000","last_fullsized":"2024-09-19T11:23:42.944159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:33.890847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944557+0000","last_change":"2024-09-19T11:23:03.258130+0000","last_active":"2024-09-19T11:23:42.944557+0000","last_peered":"2024-09-19T11:23:42.944557+0000","last_clean":"2024-09-19T11:23:42.944557+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:23:42.944557+0000","last_undegraded":"2024-09-19T11:23:42.944557+0000","last_fullsized":"2024-09-19T11:23:42.944557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:11.381818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281740+0000","last_change":"2024-09-19T11:23:07.238893+0000","last_active":"2024-09-19T11:23:42.281740+0000","last_peered":"2024-09-19T11:23:42.281740+0000","last_clean":"2024-09-19T11:23:42.281740+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:23:42.281740+0000","last_undegraded":"2024-09-19T11:23:42.281740+0000","last_fullsized":"2024-09-19T11:23:42.281740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:35.034936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'85","reported_seq":167,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.214097+0000","last_change":"2024-09-19T11:23:09.260843+0000","last_active":"2024-09-19T11:26:19.214097+0000","last_peered":"2024-09-19T11:26:19.214097+0000","last_clean":"2024-09-19T11:26:19.214097+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:26:19.214097+0000","last_undegraded":"2024-09-19T11:26:19.214097+0000","last_fullsized":"2024-09-19T11:26:19.214097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:17.846685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280169+0000","last_change":"2024-09-19T11:23:11.266314+0000","last_active":"2024-09-19T11:23:42.280169+0000","last_peered":"2024-09-19T11:23:42.280169+0000","last_clean":"2024-09-19T11:23:42.280169+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:23:42.280169+0000","last_undegraded":"2024-09-19T11:23:42.280169+0000","last_fullsized":"2024-09-19T11:23:42.280169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:28.832170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":163,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.377387+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:26:12.377387+0000","last_peered":"2024-09-19T11:26:12.377387+0000","last_clean":"2024-09-19T11:26:12.377387+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:26:12.377387+0000","last_undegraded":"2024-09-19T11:26:12.377387+0000","last_fullsized":"2024-09-19T11:26:12.377387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:51.720881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'5","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.810212+0000","last_change":"2024-09-19T11:23:05.233784+0000","last_active":"2024-09-19T11:25:16.810212+0000","last_peered":"2024-09-19T11:25:16.810212+0000","last_clean":"2024-09-19T11:25:16.810212+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:25:16.810212+0000","last_undegraded":"2024-09-19T11:25:16.810212+0000","last_fullsized":"2024-09-19T11:25:16.810212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943897+0000","last_change":"2024-09-19T11:22:55.817444+0000","last_active":"2024-09-19T11:23:42.943897+0000","last_peered":"2024-09-19T11:23:42.943897+0000","last_clean":"2024-09-19T11:23:42.943897+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:23:42.943897+0000","last_undegraded":"2024-09-19T11:23:42.943897+0000","last_fullsized":"2024-09-19T11:23:42.943897+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:39.661543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.920820+0000","last_change":"2024-09-19T11:23:03.236740+0000","last_active":"2024-09-19T11:26:11.920820+0000","last_peered":"2024-09-19T11:26:11.920820+0000","last_clean":"2024-09-19T11:26:11.920820+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:26:11.920820+0000","last_undegraded":"2024-09-19T11:26:11.920820+0000","last_fullsized":"2024-09-19T11:26:11.920820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:40.283716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948430+0000","last_change":"2024-09-19T11:23:07.246124+0000","last_active":"2024-09-19T11:23:42.948430+0000","last_peered":"2024-09-19T11:23:42.948430+0000","last_clean":"2024-09-19T11:23:42.948430+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:23:42.948430+0000","last_undegraded":"2024-09-19T11:23:42.948430+0000","last_fullsized":"2024-09-19T11:23:42.948430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'284","reported_seq":501,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.212063+0000","last_change":"2024-09-19T11:23:09.257240+0000","last_active":"2024-09-19T11:26:19.212063+0000","last_peered":"2024-09-19T11:26:19.212063+0000","last_clean":"2024-09-19T11:26:19.212063+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:26:19.212063+0000","last_undegraded":"2024-09-19T11:26:19.212063+0000","last_fullsized":"2024-09-19T11:26:19.212063+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:14.514960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944824+0000","last_change":"2024-09-19T11:23:11.270807+0000","last_active":"2024-09-19T11:23:42.944824+0000","last_peered":"2024-09-19T11:23:42.944824+0000","last_clean":"2024-09-19T11:23:42.944824+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:23:42.944824+0000","last_undegraded":"2024-09-19T11:23:42.944824+0000","last_fullsized":"2024-09-19T11:23:42.944824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:23.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.346118+0000","last_change":"2024-09-19T11:23:13.269927+0000","last_active":"2024-09-19T11:24:27.346118+0000","last_peered":"2024-09-19T11:24:27.346118+0000","last_clean":"2024-09-19T11:24:27.346118+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:24:27.346118+0000","last_undegraded":"2024-09-19T11:24:27.346118+0000","last_fullsized":"2024-09-19T11:24:27.346118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:59.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123437+0000","last_change":"2024-09-19T11:23:05.248320+0000","last_active":"2024-09-19T11:26:12.123437+0000","last_peered":"2024-09-19T11:26:12.123437+0000","last_clean":"2024-09-19T11:26:12.123437+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:12.123437+0000","last_undegraded":"2024-09-19T11:26:12.123437+0000","last_fullsized":"2024-09-19T11:26:12.123437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:59.152903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276942+0000","last_change":"2024-09-19T11:22:55.886390+0000","last_active":"2024-09-19T11:23:42.276942+0000","last_peered":"2024-09-19T11:23:42.276942+0000","last_clean":"2024-09-19T11:23:42.276942+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:23:42.276942+0000","last_undegraded":"2024-09-19T11:23:42.276942+0000","last_fullsized":"2024-09-19T11:23:42.276942+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:42.873153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.275364+0000","last_change":"2024-09-19T11:23:03.244094+0000","last_active":"2024-09-19T11:25:11.275364+0000","last_peered":"2024-09-19T11:25:11.275364+0000","last_clean":"2024-09-19T11:25:11.275364+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:25:11.275364+0000","last_undegraded":"2024-09-19T11:25:11.275364+0000","last_fullsized":"2024-09-19T11:25:11.275364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:27.152551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280552+0000","last_change":"2024-09-19T11:23:07.245296+0000","last_active":"2024-09-19T11:23:42.280552+0000","last_peered":"2024-09-19T11:23:42.280552+0000","last_clean":"2024-09-19T11:23:42.280552+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:23:42.280552+0000","last_undegraded":"2024-09-19T11:23:42.280552+0000","last_fullsized":"2024-09-19T11:23:42.280552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:23:05.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'16","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327294+0000","last_change":"2024-09-19T11:23:09.249984+0000","last_active":"2024-09-19T11:26:12.327294+0000","last_peered":"2024-09-19T11:26:12.327294+0000","last_clean":"2024-09-19T11:26:12.327294+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:12.327294+0000","last_undegraded":"2024-09-19T11:26:12.327294+0000","last_fullsized":"2024-09-19T11:26:12.327294+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:47.233637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280391+0000","last_change":"2024-09-19T11:23:11.265633+0000","last_active":"2024-09-19T11:23:42.280391+0000","last_peered":"2024-09-19T11:23:42.280391+0000","last_clean":"2024-09-19T11:23:42.280391+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:23:42.280391+0000","last_undegraded":"2024-09-19T11:23:42.280391+0000","last_fullsized":"2024-09-19T11:23:42.280391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:16.181121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943722+0000","last_change":"2024-09-19T11:23:13.273373+0000","last_active":"2024-09-19T11:23:42.943722+0000","last_peered":"2024-09-19T11:23:42.943722+0000","last_clean":"2024-09-19T11:23:42.943722+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:23:42.943722+0000","last_undegraded":"2024-09-19T11:23:42.943722+0000","last_fullsized":"2024-09-19T11:23:42.943722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:11:13.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368226+0000","last_change":"2024-09-19T11:23:05.238227+0000","last_active":"2024-09-19T11:26:12.368226+0000","last_peered":"2024-09-19T11:26:12.368226+0000","last_clean":"2024-09-19T11:26:12.368226+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:12.368226+0000","last_undegraded":"2024-09-19T11:26:12.368226+0000","last_fullsized":"2024-09-19T11:26:12.368226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:05.469042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944111+0000","last_change":"2024-09-19T11:22:55.817947+0000","last_active":"2024-09-19T11:23:42.944111+0000","last_peered":"2024-09-19T11:23:42.944111+0000","last_clean":"2024-09-19T11:23:42.944111+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:23:42.944111+0000","last_undegraded":"2024-09-19T11:23:42.944111+0000","last_fullsized":"2024-09-19T11:23:42.944111+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:23:52.495235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.363564+0000","last_change":"2024-09-19T11:23:03.247494+0000","last_active":"2024-09-19T11:26:12.363564+0000","last_peered":"2024-09-19T11:26:12.363564+0000","last_clean":"2024-09-19T11:26:12.363564+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:12.363564+0000","last_undegraded":"2024-09-19T11:26:12.363564+0000","last_fullsized":"2024-09-19T11:26:12.363564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:58.764280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279722+0000","last_change":"2024-09-19T11:23:07.243946+0000","last_active":"2024-09-19T11:23:42.279722+0000","last_peered":"2024-09-19T11:23:42.279722+0000","last_clean":"2024-09-19T11:23:42.279722+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:23:42.279722+0000","last_undegraded":"2024-09-19T11:23:42.279722+0000","last_fullsized":"2024-09-19T11:23:42.279722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:02.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'13","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.530475+0000","last_change":"2024-09-19T11:23:09.254499+0000","last_active":"2024-09-19T11:26:11.530475+0000","last_peered":"2024-09-19T11:26:11.530475+0000","last_clean":"2024-09-19T11:26:11.530475+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:26:11.530475+0000","last_undegraded":"2024-09-19T11:26:11.530475+0000","last_fullsized":"2024-09-19T11:26:11.530475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:18.957107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947826+0000","last_change":"2024-09-19T11:23:11.271073+0000","last_active":"2024-09-19T11:23:42.947826+0000","last_peered":"2024-09-19T11:23:42.947826+0000","last_clean":"2024-09-19T11:23:42.947826+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:23:42.947826+0000","last_undegraded":"2024-09-19T11:23:42.947826+0000","last_fullsized":"2024-09-19T11:23:42.947826+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:52.079937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.088603+0000","last_change":"2024-09-19T11:23:13.273910+0000","last_active":"2024-09-19T11:25:11.088603+0000","last_peered":"2024-09-19T11:25:11.088603+0000","last_clean":"2024-09-19T11:25:11.088603+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:25:11.088603+0000","last_undegraded":"2024-09-19T11:25:11.088603+0000","last_fullsized":"2024-09-19T11:25:11.088603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:02.581618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470157+0000","last_change":"2024-09-19T11:23:05.247999+0000","last_active":"2024-09-19T11:26:12.470157+0000","last_peered":"2024-09-19T11:26:12.470157+0000","last_clean":"2024-09-19T11:26:12.470157+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:12.470157+0000","last_undegraded":"2024-09-19T11:26:12.470157+0000","last_fullsized":"2024-09-19T11:26:12.470157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.575444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277923+0000","last_change":"2024-09-19T11:23:03.251364+0000","last_active":"2024-09-19T11:23:42.277923+0000","last_peered":"2024-09-19T11:23:42.277923+0000","last_clean":"2024-09-19T11:23:42.277923+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:23:42.277923+0000","last_undegraded":"2024-09-19T11:23:42.277923+0000","last_fullsized":"2024-09-19T11:23:42.277923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279906+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.279906+0000","last_peered":"2024-09-19T11:23:42.279906+0000","last_clean":"2024-09-19T11:23:42.279906+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:23:42.279906+0000","last_undegraded":"2024-09-19T11:23:42.279906+0000","last_fullsized":"2024-09-19T11:23:42.279906+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:44:41.618944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'1","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.641108+0000","last_change":"2024-09-19T11:23:07.247913+0000","last_active":"2024-09-19T11:26:12.641108+0000","last_peered":"2024-09-19T11:26:12.641108+0000","last_clean":"2024-09-19T11:26:12.641108+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:26:12.641108+0000","last_undegraded":"2024-09-19T11:26:12.641108+0000","last_fullsized":"2024-09-19T11:26:12.641108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:03.208183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'68","reported_seq":136,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.192730+0000","last_change":"2024-09-19T11:23:09.257853+0000","last_active":"2024-09-19T11:26:19.192730+0000","last_peered":"2024-09-19T11:26:19.192730+0000","last_clean":"2024-09-19T11:26:19.192730+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:26:19.192730+0000","last_undegraded":"2024-09-19T11:26:19.192730+0000","last_fullsized":"2024-09-19T11:26:19.192730+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:02.366161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277438+0000","last_change":"2024-09-19T11:23:11.268600+0000","last_active":"2024-09-19T11:23:42.277438+0000","last_peered":"2024-09-19T11:23:42.277438+0000","last_clean":"2024-09-19T11:23:42.277438+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:23:42.277438+0000","last_undegraded":"2024-09-19T11:23:42.277438+0000","last_fullsized":"2024-09-19T11:23:42.277438+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:50.540232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.422953+0000","last_change":"2024-09-19T11:23:13.267518+0000","last_active":"2024-09-19T11:25:11.422953+0000","last_peered":"2024-09-19T11:25:11.422953+0000","last_clean":"2024-09-19T11:25:11.422953+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:25:11.422953+0000","last_undegraded":"2024-09-19T11:25:11.422953+0000","last_fullsized":"2024-09-19T11:25:11.422953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:35.030653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338573+0000","last_change":"2024-09-19T11:23:05.247369+0000","last_active":"2024-09-19T11:26:12.338573+0000","last_peered":"2024-09-19T11:26:12.338573+0000","last_clean":"2024-09-19T11:26:12.338573+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:12.338573+0000","last_undegraded":"2024-09-19T11:26:12.338573+0000","last_fullsized":"2024-09-19T11:26:12.338573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:05.530837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.954337+0000","last_change":"2024-09-19T11:23:03.236746+0000","last_active":"2024-09-19T11:26:11.954337+0000","last_peered":"2024-09-19T11:26:11.954337+0000","last_clean":"2024-09-19T11:26:11.954337+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:26:11.954337+0000","last_undegraded":"2024-09-19T11:26:11.954337+0000","last_fullsized":"2024-09-19T11:26:11.954337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:23.056208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945428+0000","last_change":"2024-09-19T11:22:55.818302+0000","last_active":"2024-09-19T11:23:42.945428+0000","last_peered":"2024-09-19T11:23:42.945428+0000","last_clean":"2024-09-19T11:23:42.945428+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:23:42.945428+0000","last_undegraded":"2024-09-19T11:23:42.945428+0000","last_fullsized":"2024-09-19T11:23:42.945428+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:51.911098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943947+0000","last_change":"2024-09-19T11:23:07.243184+0000","last_active":"2024-09-19T11:23:42.943947+0000","last_peered":"2024-09-19T11:23:42.943947+0000","last_clean":"2024-09-19T11:23:42.943947+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:23:42.943947+0000","last_undegraded":"2024-09-19T11:23:42.943947+0000","last_fullsized":"2024-09-19T11:23:42.943947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:32:59.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'15","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809866+0000","last_change":"2024-09-19T11:23:09.261039+0000","last_active":"2024-09-19T11:25:16.809866+0000","last_peered":"2024-09-19T11:25:16.809866+0000","last_clean":"2024-09-19T11:25:16.809866+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:25:16.809866+0000","last_undegraded":"2024-09-19T11:25:16.809866+0000","last_fullsized":"2024-09-19T11:25:16.809866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:42.183684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'285","reported_seq":359,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.071730+0000","last_change":"2024-09-19T11:23:11.263125+0000","last_active":"2024-09-19T11:26:19.071730+0000","last_peered":"2024-09-19T11:26:19.071730+0000","last_clean":"2024-09-19T11:26:19.071730+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:26:19.071730+0000","last_undegraded":"2024-09-19T11:26:19.071730+0000","last_fullsized":"2024-09-19T11:26:19.071730+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:10.328770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946621+0000","last_change":"2024-09-19T11:23:13.269759+0000","last_active":"2024-09-19T11:23:42.946621+0000","last_peered":"2024-09-19T11:23:42.946621+0000","last_clean":"2024-09-19T11:23:42.946621+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:23:42.946621+0000","last_undegraded":"2024-09-19T11:23:42.946621+0000","last_fullsized":"2024-09-19T11:23:42.946621+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:46.820239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'11","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.025445+0000","last_change":"2024-09-19T11:23:05.254850+0000","last_active":"2024-09-19T11:26:12.025445+0000","last_peered":"2024-09-19T11:26:12.025445+0000","last_clean":"2024-09-19T11:26:12.025445+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:12.025445+0000","last_undegraded":"2024-09-19T11:26:12.025445+0000","last_fullsized":"2024-09-19T11:26:12.025445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:44.121192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948269+0000","last_change":"2024-09-19T11:23:03.241499+0000","last_active":"2024-09-19T11:23:42.948269+0000","last_peered":"2024-09-19T11:23:42.948269+0000","last_clean":"2024-09-19T11:23:42.948269+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:23:42.948269+0000","last_undegraded":"2024-09-19T11:23:42.948269+0000","last_fullsized":"2024-09-19T11:23:42.948269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:40.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280554+0000","last_change":"2024-09-19T11:22:55.813485+0000","last_active":"2024-09-19T11:23:42.280554+0000","last_peered":"2024-09-19T11:23:42.280554+0000","last_clean":"2024-09-19T11:23:42.280554+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:23:42.280554+0000","last_undegraded":"2024-09-19T11:23:42.280554+0000","last_fullsized":"2024-09-19T11:23:42.280554+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:08:18.682837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946584+0000","last_change":"2024-09-19T11:23:07.245138+0000","last_active":"2024-09-19T11:23:42.946584+0000","last_peered":"2024-09-19T11:23:42.946584+0000","last_clean":"2024-09-19T11:23:42.946584+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:23:42.946584+0000","last_undegraded":"2024-09-19T11:23:42.946584+0000","last_fullsized":"2024-09-19T11:23:42.946584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:31.680535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'11","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945354+0000","last_change":"2024-09-19T11:23:09.256698+0000","last_active":"2024-09-19T11:23:42.945354+0000","last_peered":"2024-09-19T11:23:42.945354+0000","last_clean":"2024-09-19T11:23:42.945354+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:23:42.945354+0000","last_undegraded":"2024-09-19T11:23:42.945354+0000","last_fullsized":"2024-09-19T11:23:42.945354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:21.852554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'285","reported_seq":433,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.076916+0000","last_change":"2024-09-19T11:23:11.258279+0000","last_active":"2024-09-19T11:26:19.076916+0000","last_peered":"2024-09-19T11:26:19.076916+0000","last_clean":"2024-09-19T11:26:19.076916+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:26:19.076916+0000","last_undegraded":"2024-09-19T11:26:19.076916+0000","last_fullsized":"2024-09-19T11:26:19.076916+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:24.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281484+0000","last_change":"2024-09-19T11:23:13.274470+0000","last_active":"2024-09-19T11:23:42.281484+0000","last_peered":"2024-09-19T11:23:42.281484+0000","last_clean":"2024-09-19T11:23:42.281484+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:23:42.281484+0000","last_undegraded":"2024-09-19T11:23:42.281484+0000","last_fullsized":"2024-09-19T11:23:42.281484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:33.500152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550363+0000","last_change":"2024-09-19T11:23:05.250199+0000","last_active":"2024-09-19T11:24:27.550363+0000","last_peered":"2024-09-19T11:24:27.550363+0000","last_clean":"2024-09-19T11:24:27.550363+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:24:27.550363+0000","last_undegraded":"2024-09-19T11:24:27.550363+0000","last_fullsized":"2024-09-19T11:24:27.550363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:20.188966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282786+0000","last_change":"2024-09-19T11:23:03.243876+0000","last_active":"2024-09-19T11:23:42.282786+0000","last_peered":"2024-09-19T11:23:42.282786+0000","last_clean":"2024-09-19T11:23:42.282786+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:23:42.282786+0000","last_undegraded":"2024-09-19T11:23:42.282786+0000","last_fullsized":"2024-09-19T11:23:42.282786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:15.709376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946831+0000","last_change":"2024-09-19T11:22:55.818050+0000","last_active":"2024-09-19T11:23:42.946831+0000","last_peered":"2024-09-19T11:23:42.946831+0000","last_clean":"2024-09-19T11:23:42.946831+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:23:42.946831+0000","last_undegraded":"2024-09-19T11:23:42.946831+0000","last_fullsized":"2024-09-19T11:23:42.946831+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:22:52.801931+0000","last_clean_scrub_stamp":"2024-09-19T11:22:52.801931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.933095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945462+0000","last_change":"2024-09-19T11:23:07.248324+0000","last_active":"2024-09-19T11:23:42.945462+0000","last_peered":"2024-09-19T11:23:42.945462+0000","last_clean":"2024-09-19T11:23:42.945462+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:23:42.945462+0000","last_undegraded":"2024-09-19T11:23:42.945462+0000","last_fullsized":"2024-09-19T11:23:42.945462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.767188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'91","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.201893+0000","last_change":"2024-09-19T11:23:09.258292+0000","last_active":"2024-09-19T11:26:19.201893+0000","last_peered":"2024-09-19T11:26:19.201893+0000","last_clean":"2024-09-19T11:26:19.201893+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:26:19.201893+0000","last_undegraded":"2024-09-19T11:26:19.201893+0000","last_fullsized":"2024-09-19T11:26:19.201893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:51.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280109+0000","last_change":"2024-09-19T11:23:11.263614+0000","last_active":"2024-09-19T11:23:42.280109+0000","last_peered":"2024-09-19T11:23:42.280109+0000","last_clean":"2024-09-19T11:23:42.280109+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:23:42.280109+0000","last_undegraded":"2024-09-19T11:23:42.280109+0000","last_fullsized":"2024-09-19T11:23:42.280109+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.842570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278855+0000","last_change":"2024-09-19T11:23:13.265949+0000","last_active":"2024-09-19T11:23:42.278855+0000","last_peered":"2024-09-19T11:23:42.278855+0000","last_clean":"2024-09-19T11:23:42.278855+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:23:42.278855+0000","last_undegraded":"2024-09-19T11:23:42.278855+0000","last_fullsized":"2024-09-19T11:23:42.278855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:37.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278953+0000","last_change":"2024-09-19T11:23:13.271066+0000","last_active":"2024-09-19T11:23:42.278953+0000","last_peered":"2024-09-19T11:23:42.278953+0000","last_clean":"2024-09-19T11:23:42.278953+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:23:42.278953+0000","last_undegraded":"2024-09-19T11:23:42.278953+0000","last_fullsized":"2024-09-19T11:23:42.278953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:34.307629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277914+0000","last_change":"2024-09-19T11:23:05.245620+0000","last_active":"2024-09-19T11:23:42.277914+0000","last_peered":"2024-09-19T11:23:42.277914+0000","last_clean":"2024-09-19T11:23:42.277914+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:23:42.277914+0000","last_undegraded":"2024-09-19T11:23:42.277914+0000","last_fullsized":"2024-09-19T11:23:42.277914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.990710+0000","last_change":"2024-09-19T11:23:03.254634+0000","last_active":"2024-09-19T11:23:44.990710+0000","last_peered":"2024-09-19T11:23:44.990710+0000","last_clean":"2024-09-19T11:23:44.990710+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:23:44.990710+0000","last_undegraded":"2024-09-19T11:23:44.990710+0000","last_fullsized":"2024-09-19T11:23:44.990710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:06.416384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282381+0000","last_change":"2024-09-19T11:23:07.240696+0000","last_active":"2024-09-19T11:23:42.282381+0000","last_peered":"2024-09-19T11:23:42.282381+0000","last_clean":"2024-09-19T11:23:42.282381+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:23:42.282381+0000","last_undegraded":"2024-09-19T11:23:42.282381+0000","last_fullsized":"2024-09-19T11:23:42.282381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:02.659138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'35","reported_seq":83,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.220186+0000","last_change":"2024-09-19T11:23:09.259148+0000","last_active":"2024-09-19T11:26:19.220186+0000","last_peered":"2024-09-19T11:26:19.220186+0000","last_clean":"2024-09-19T11:26:19.220186+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:26:19.220186+0000","last_undegraded":"2024-09-19T11:26:19.220186+0000","last_fullsized":"2024-09-19T11:26:19.220186+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:29.000573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945264+0000","last_change":"2024-09-19T11:23:11.271128+0000","last_active":"2024-09-19T11:23:42.945264+0000","last_peered":"2024-09-19T11:23:42.945264+0000","last_clean":"2024-09-19T11:23:42.945264+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:23:42.945264+0000","last_undegraded":"2024-09-19T11:23:42.945264+0000","last_fullsized":"2024-09-19T11:23:42.945264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:35.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.427118+0000","last_change":"2024-09-19T11:23:13.273576+0000","last_active":"2024-09-19T11:25:09.427118+0000","last_peered":"2024-09-19T11:25:09.427118+0000","last_clean":"2024-09-19T11:25:09.427118+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:25:09.427118+0000","last_undegraded":"2024-09-19T11:25:09.427118+0000","last_fullsized":"2024-09-19T11:25:09.427118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:43.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.782138+0000","last_change":"2024-09-19T11:23:42.285785+0000","last_active":"2024-09-19T11:23:44.782138+0000","last_peered":"2024-09-19T11:23:44.782138+0000","last_clean":"2024-09-19T11:23:44.782138+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:23:44.782138+0000","last_undegraded":"2024-09-19T11:23:44.782138+0000","last_fullsized":"2024-09-19T11:23:44.782138+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:15.907367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277946+0000","last_change":"2024-09-19T11:23:03.255902+0000","last_active":"2024-09-19T11:23:42.277946+0000","last_peered":"2024-09-19T11:23:42.277946+0000","last_clean":"2024-09-19T11:23:42.277946+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:23:42.277946+0000","last_undegraded":"2024-09-19T11:23:42.277946+0000","last_fullsized":"2024-09-19T11:23:42.277946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:41.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945177+0000","last_change":"2024-09-19T11:23:07.248908+0000","last_active":"2024-09-19T11:23:42.945177+0000","last_peered":"2024-09-19T11:23:42.945177+0000","last_clean":"2024-09-19T11:23:42.945177+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:23:42.945177+0000","last_undegraded":"2024-09-19T11:23:42.945177+0000","last_fullsized":"2024-09-19T11:23:42.945177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:57.835222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'5","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.535761+0000","last_change":"2024-09-19T11:23:09.244959+0000","last_active":"2024-09-19T11:26:11.535761+0000","last_peered":"2024-09-19T11:26:11.535761+0000","last_clean":"2024-09-19T11:26:11.535761+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:26:11.535761+0000","last_undegraded":"2024-09-19T11:26:11.535761+0000","last_fullsized":"2024-09-19T11:26:11.535761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:13.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277093+0000","last_change":"2024-09-19T11:23:11.268038+0000","last_active":"2024-09-19T11:23:42.277093+0000","last_peered":"2024-09-19T11:23:42.277093+0000","last_clean":"2024-09-19T11:23:42.277093+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:23:42.277093+0000","last_undegraded":"2024-09-19T11:23:42.277093+0000","last_fullsized":"2024-09-19T11:23:42.277093+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:40.232592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279462+0000","last_change":"2024-09-19T11:23:13.267238+0000","last_active":"2024-09-19T11:23:42.279462+0000","last_peered":"2024-09-19T11:23:42.279462+0000","last_clean":"2024-09-19T11:23:42.279462+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:23:42.279462+0000","last_undegraded":"2024-09-19T11:23:42.279462+0000","last_fullsized":"2024-09-19T11:23:42.279462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:36.252981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123419+0000","last_change":"2024-09-19T11:23:05.240162+0000","last_active":"2024-09-19T11:26:12.123419+0000","last_peered":"2024-09-19T11:26:12.123419+0000","last_clean":"2024-09-19T11:26:12.123419+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:12.123419+0000","last_undegraded":"2024-09-19T11:26:12.123419+0000","last_fullsized":"2024-09-19T11:26:12.123419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:45.738004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.261555+0000","last_change":"2024-09-19T11:23:03.257719+0000","last_active":"2024-09-19T11:26:12.261555+0000","last_peered":"2024-09-19T11:26:12.261555+0000","last_clean":"2024-09-19T11:26:12.261555+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:12.261555+0000","last_undegraded":"2024-09-19T11:26:12.261555+0000","last_fullsized":"2024-09-19T11:26:12.261555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:46.873725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277308+0000","last_change":"2024-09-19T11:23:07.243467+0000","last_active":"2024-09-19T11:23:42.277308+0000","last_peered":"2024-09-19T11:23:42.277308+0000","last_clean":"2024-09-19T11:23:42.277308+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:23:42.277308+0000","last_undegraded":"2024-09-19T11:23:42.277308+0000","last_fullsized":"2024-09-19T11:23:42.277308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:48.792369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'43","reported_seq":107,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.186175+0000","last_change":"2024-09-19T11:23:09.248715+0000","last_active":"2024-09-19T11:26:19.186175+0000","last_peered":"2024-09-19T11:26:19.186175+0000","last_clean":"2024-09-19T11:26:19.186175+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:26:19.186175+0000","last_undegraded":"2024-09-19T11:26:19.186175+0000","last_fullsized":"2024-09-19T11:26:19.186175+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:51.723342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280357+0000","last_change":"2024-09-19T11:23:11.262690+0000","last_active":"2024-09-19T11:23:42.280357+0000","last_peered":"2024-09-19T11:23:42.280357+0000","last_clean":"2024-09-19T11:23:42.280357+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:23:42.280357+0000","last_undegraded":"2024-09-19T11:23:42.280357+0000","last_fullsized":"2024-09-19T11:23:42.280357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:55.778119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.511055+0000","last_change":"2024-09-19T11:23:05.253736+0000","last_active":"2024-09-19T11:25:11.511055+0000","last_peered":"2024-09-19T11:25:11.511055+0000","last_clean":"2024-09-19T11:25:11.511055+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:25:11.511055+0000","last_undegraded":"2024-09-19T11:25:11.511055+0000","last_fullsized":"2024-09-19T11:25:11.511055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:09.050323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283174+0000","last_change":"2024-09-19T11:23:03.248284+0000","last_active":"2024-09-19T11:23:42.283174+0000","last_peered":"2024-09-19T11:23:42.283174+0000","last_clean":"2024-09-19T11:23:42.283174+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:23:42.283174+0000","last_undegraded":"2024-09-19T11:23:42.283174+0000","last_fullsized":"2024-09-19T11:23:42.283174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:40.686948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944123+0000","last_change":"2024-09-19T11:23:07.245862+0000","last_active":"2024-09-19T11:23:42.944123+0000","last_peered":"2024-09-19T11:23:42.944123+0000","last_clean":"2024-09-19T11:23:42.944123+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:23:42.944123+0000","last_undegraded":"2024-09-19T11:23:42.944123+0000","last_fullsized":"2024-09-19T11:23:42.944123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:09.692396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.177030+0000","last_change":"2024-09-19T11:23:09.247041+0000","last_active":"2024-09-19T11:26:19.177030+0000","last_peered":"2024-09-19T11:26:19.177030+0000","last_clean":"2024-09-19T11:26:19.177030+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:26:19.177030+0000","last_undegraded":"2024-09-19T11:26:19.177030+0000","last_fullsized":"2024-09-19T11:26:19.177030+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:21.727511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277107+0000","last_change":"2024-09-19T11:23:11.253755+0000","last_active":"2024-09-19T11:23:42.277107+0000","last_peered":"2024-09-19T11:23:42.277107+0000","last_clean":"2024-09-19T11:23:42.277107+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:23:42.277107+0000","last_undegraded":"2024-09-19T11:23:42.277107+0000","last_fullsized":"2024-09-19T11:23:42.277107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:51.209331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510416+0000","last_change":"2024-09-19T11:23:05.236615+0000","last_active":"2024-09-19T11:25:11.510416+0000","last_peered":"2024-09-19T11:25:11.510416+0000","last_clean":"2024-09-19T11:25:11.510416+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:25:11.510416+0000","last_undegraded":"2024-09-19T11:25:11.510416+0000","last_fullsized":"2024-09-19T11:25:11.510416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:36.749643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'4","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.902758+0000","last_change":"2024-09-19T11:23:03.255452+0000","last_active":"2024-09-19T11:26:11.902758+0000","last_peered":"2024-09-19T11:26:11.902758+0000","last_clean":"2024-09-19T11:26:11.902758+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:26:11.902758+0000","last_undegraded":"2024-09-19T11:26:11.902758+0000","last_fullsized":"2024-09-19T11:26:11.902758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:34.861274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796174+0000","last_change":"2024-09-19T11:23:42.945035+0000","last_active":"2024-09-19T11:23:44.796174+0000","last_peered":"2024-09-19T11:23:44.796174+0000","last_clean":"2024-09-19T11:23:44.796174+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:23:44.796174+0000","last_undegraded":"2024-09-19T11:23:44.796174+0000","last_fullsized":"2024-09-19T11:23:44.796174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'62","reported_seq":127,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.174069+0000","last_change":"2024-09-19T11:23:09.246587+0000","last_active":"2024-09-19T11:26:19.174069+0000","last_peered":"2024-09-19T11:26:19.174069+0000","last_clean":"2024-09-19T11:26:19.174069+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:26:19.174069+0000","last_undegraded":"2024-09-19T11:26:19.174069+0000","last_fullsized":"2024-09-19T11:26:19.174069+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:53.040162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":65,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278167+0000","last_change":"2024-09-19T11:23:11.260449+0000","last_active":"2024-09-19T11:23:42.278167+0000","last_peered":"2024-09-19T11:23:42.278167+0000","last_clean":"2024-09-19T11:23:42.278167+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:23:42.278167+0000","last_undegraded":"2024-09-19T11:23:42.278167+0000","last_fullsized":"2024-09-19T11:23:42.278167+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:30.983027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.419187+0000","last_change":"2024-09-19T11:23:13.268105+0000","last_active":"2024-09-19T11:25:11.419187+0000","last_peered":"2024-09-19T11:25:11.419187+0000","last_clean":"2024-09-19T11:25:11.419187+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:25:11.419187+0000","last_undegraded":"2024-09-19T11:25:11.419187+0000","last_fullsized":"2024-09-19T11:25:11.419187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:21.113571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368061+0000","last_change":"2024-09-19T11:23:05.246352+0000","last_active":"2024-09-19T11:26:12.368061+0000","last_peered":"2024-09-19T11:26:12.368061+0000","last_clean":"2024-09-19T11:26:12.368061+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:12.368061+0000","last_undegraded":"2024-09-19T11:26:12.368061+0000","last_fullsized":"2024-09-19T11:26:12.368061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280887+0000","last_change":"2024-09-19T11:23:03.248495+0000","last_active":"2024-09-19T11:23:42.280887+0000","last_peered":"2024-09-19T11:23:42.280887+0000","last_clean":"2024-09-19T11:23:42.280887+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:23:42.280887+0000","last_undegraded":"2024-09-19T11:23:42.280887+0000","last_fullsized":"2024-09-19T11:23:42.280887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:28:20.959521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279359+0000","last_change":"2024-09-19T11:23:07.243165+0000","last_active":"2024-09-19T11:23:42.279359+0000","last_peered":"2024-09-19T11:23:42.279359+0000","last_clean":"2024-09-19T11:23:42.279359+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:23:42.279359+0000","last_undegraded":"2024-09-19T11:23:42.279359+0000","last_fullsized":"2024-09-19T11:23:42.279359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:21.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.179930+0000","last_change":"2024-09-19T11:23:09.256642+0000","last_active":"2024-09-19T11:26:19.179930+0000","last_peered":"2024-09-19T11:26:19.179930+0000","last_clean":"2024-09-19T11:26:19.179930+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:26:19.179930+0000","last_undegraded":"2024-09-19T11:26:19.179930+0000","last_fullsized":"2024-09-19T11:26:19.179930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:42.797325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282300+0000","last_change":"2024-09-19T11:23:11.266817+0000","last_active":"2024-09-19T11:23:42.282300+0000","last_peered":"2024-09-19T11:23:42.282300+0000","last_clean":"2024-09-19T11:23:42.282300+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:23:42.282300+0000","last_undegraded":"2024-09-19T11:23:42.282300+0000","last_fullsized":"2024-09-19T11:23:42.282300+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:41:28.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947653+0000","last_change":"2024-09-19T11:23:05.254589+0000","last_active":"2024-09-19T11:23:42.947653+0000","last_peered":"2024-09-19T11:23:42.947653+0000","last_clean":"2024-09-19T11:23:42.947653+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:23:42.947653+0000","last_undegraded":"2024-09-19T11:23:42.947653+0000","last_fullsized":"2024-09-19T11:23:42.947653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:11.078441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280548+0000","last_change":"2024-09-19T11:23:03.244404+0000","last_active":"2024-09-19T11:23:42.280548+0000","last_peered":"2024-09-19T11:23:42.280548+0000","last_clean":"2024-09-19T11:23:42.280548+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:23:42.280548+0000","last_undegraded":"2024-09-19T11:23:42.280548+0000","last_fullsized":"2024-09-19T11:23:42.280548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:07.948011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278448+0000","last_change":"2024-09-19T11:23:07.242009+0000","last_active":"2024-09-19T11:23:42.278448+0000","last_peered":"2024-09-19T11:23:42.278448+0000","last_clean":"2024-09-19T11:23:42.278448+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:23:42.278448+0000","last_undegraded":"2024-09-19T11:23:42.278448+0000","last_fullsized":"2024-09-19T11:23:42.278448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:37.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'37","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.126834+0000","last_change":"2024-09-19T11:23:09.258190+0000","last_active":"2024-09-19T11:26:19.126834+0000","last_peered":"2024-09-19T11:26:19.126834+0000","last_clean":"2024-09-19T11:26:19.126834+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:26:19.126834+0000","last_undegraded":"2024-09-19T11:26:19.126834+0000","last_fullsized":"2024-09-19T11:26:19.126834+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:31.989630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280329+0000","last_change":"2024-09-19T11:23:11.261698+0000","last_active":"2024-09-19T11:23:42.280329+0000","last_peered":"2024-09-19T11:23:42.280329+0000","last_clean":"2024-09-19T11:23:42.280329+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:23:42.280329+0000","last_undegraded":"2024-09-19T11:23:42.280329+0000","last_fullsized":"2024-09-19T11:23:42.280329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:48.537146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032904+0000","last_change":"2024-09-19T11:23:05.238234+0000","last_active":"2024-09-19T11:25:08.032904+0000","last_peered":"2024-09-19T11:25:08.032904+0000","last_clean":"2024-09-19T11:25:08.032904+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:25:08.032904+0000","last_undegraded":"2024-09-19T11:25:08.032904+0000","last_fullsized":"2024-09-19T11:25:08.032904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:03:26.967357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944417+0000","last_change":"2024-09-19T11:23:03.253512+0000","last_active":"2024-09-19T11:23:42.944417+0000","last_peered":"2024-09-19T11:23:42.944417+0000","last_clean":"2024-09-19T11:23:42.944417+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:23:42.944417+0000","last_undegraded":"2024-09-19T11:23:42.944417+0000","last_fullsized":"2024-09-19T11:23:42.944417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:17.009054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.286788+0000","last_change":"2024-09-19T11:23:42.286200+0000","last_active":"2024-09-19T11:23:42.286788+0000","last_peered":"2024-09-19T11:23:42.286788+0000","last_clean":"2024-09-19T11:23:42.286788+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:23:42.286788+0000","last_undegraded":"2024-09-19T11:23:42.286788+0000","last_fullsized":"2024-09-19T11:23:42.286788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.397616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946537+0000","last_change":"2024-09-19T11:23:09.257194+0000","last_active":"2024-09-19T11:23:42.946537+0000","last_peered":"2024-09-19T11:23:42.946537+0000","last_clean":"2024-09-19T11:23:42.946537+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:23:42.946537+0000","last_undegraded":"2024-09-19T11:23:42.946537+0000","last_fullsized":"2024-09-19T11:23:42.946537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:10.690183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'285","reported_seq":421,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.070038+0000","last_change":"2024-09-19T11:23:11.268055+0000","last_active":"2024-09-19T11:26:19.070038+0000","last_peered":"2024-09-19T11:26:19.070038+0000","last_clean":"2024-09-19T11:26:19.070038+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:26:19.070038+0000","last_undegraded":"2024-09-19T11:26:19.070038+0000","last_fullsized":"2024-09-19T11:26:19.070038+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:33.997140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943313+0000","last_change":"2024-09-19T11:23:13.272264+0000","last_active":"2024-09-19T11:23:42.943313+0000","last_peered":"2024-09-19T11:23:42.943313+0000","last_clean":"2024-09-19T11:23:42.943313+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:23:42.943313+0000","last_undegraded":"2024-09-19T11:23:42.943313+0000","last_fullsized":"2024-09-19T11:23:42.943313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:34.821219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123209+0000","last_change":"2024-09-19T11:23:05.235764+0000","last_active":"2024-09-19T11:26:12.123209+0000","last_peered":"2024-09-19T11:26:12.123209+0000","last_clean":"2024-09-19T11:26:12.123209+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:12.123209+0000","last_undegraded":"2024-09-19T11:26:12.123209+0000","last_fullsized":"2024-09-19T11:26:12.123209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:41.326878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282378+0000","last_change":"2024-09-19T11:23:03.245357+0000","last_active":"2024-09-19T11:23:42.282378+0000","last_peered":"2024-09-19T11:23:42.282378+0000","last_clean":"2024-09-19T11:23:42.282378+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:23:42.282378+0000","last_undegraded":"2024-09-19T11:23:42.282378+0000","last_fullsized":"2024-09-19T11:23:42.282378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:23.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281352+0000","last_change":"2024-09-19T11:23:07.247890+0000","last_active":"2024-09-19T11:23:42.281352+0000","last_peered":"2024-09-19T11:23:42.281352+0000","last_clean":"2024-09-19T11:23:42.281352+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:23:42.281352+0000","last_undegraded":"2024-09-19T11:23:42.281352+0000","last_fullsized":"2024-09-19T11:23:42.281352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.412033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.151902+0000","last_change":"2024-09-19T11:23:09.246755+0000","last_active":"2024-09-19T11:26:19.151902+0000","last_peered":"2024-09-19T11:26:19.151902+0000","last_clean":"2024-09-19T11:26:19.151902+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:26:19.151902+0000","last_undegraded":"2024-09-19T11:26:19.151902+0000","last_fullsized":"2024-09-19T11:26:19.151902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.229678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278129+0000","last_change":"2024-09-19T11:23:11.266713+0000","last_active":"2024-09-19T11:23:42.278129+0000","last_peered":"2024-09-19T11:23:42.278129+0000","last_clean":"2024-09-19T11:23:42.278129+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:23:42.278129+0000","last_undegraded":"2024-09-19T11:23:42.278129+0000","last_fullsized":"2024-09-19T11:23:42.278129+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:38.162577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283516+0000","last_change":"2024-09-19T11:23:13.268771+0000","last_active":"2024-09-19T11:23:42.283516+0000","last_peered":"2024-09-19T11:23:42.283516+0000","last_clean":"2024-09-19T11:23:42.283516+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:23:42.283516+0000","last_undegraded":"2024-09-19T11:23:42.283516+0000","last_fullsized":"2024-09-19T11:23:42.283516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:19.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266078+0000","last_change":"2024-09-19T11:23:05.236610+0000","last_active":"2024-09-19T11:26:12.266078+0000","last_peered":"2024-09-19T11:26:12.266078+0000","last_clean":"2024-09-19T11:26:12.266078+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:12.266078+0000","last_undegraded":"2024-09-19T11:26:12.266078+0000","last_fullsized":"2024-09-19T11:26:12.266078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:44.218259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946253+0000","last_change":"2024-09-19T11:23:03.258006+0000","last_active":"2024-09-19T11:23:42.946253+0000","last_peered":"2024-09-19T11:23:42.946253+0000","last_clean":"2024-09-19T11:23:42.946253+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:23:42.946253+0000","last_undegraded":"2024-09-19T11:23:42.946253+0000","last_fullsized":"2024-09-19T11:23:42.946253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:55.934419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278151+0000","last_change":"2024-09-19T11:23:07.247875+0000","last_active":"2024-09-19T11:23:42.278151+0000","last_peered":"2024-09-19T11:23:42.278151+0000","last_clean":"2024-09-19T11:23:42.278151+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:23:42.278151+0000","last_undegraded":"2024-09-19T11:23:42.278151+0000","last_fullsized":"2024-09-19T11:23:42.278151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:20.420374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.155681+0000","last_change":"2024-09-19T11:23:09.257850+0000","last_active":"2024-09-19T11:26:19.155681+0000","last_peered":"2024-09-19T11:26:19.155681+0000","last_clean":"2024-09-19T11:26:19.155681+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:26:19.155681+0000","last_undegraded":"2024-09-19T11:26:19.155681+0000","last_fullsized":"2024-09-19T11:26:19.155681+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:48.134834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943735+0000","last_change":"2024-09-19T11:23:11.271462+0000","last_active":"2024-09-19T11:23:42.943735+0000","last_peered":"2024-09-19T11:23:42.943735+0000","last_clean":"2024-09-19T11:23:42.943735+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:23:42.943735+0000","last_undegraded":"2024-09-19T11:23:42.943735+0000","last_fullsized":"2024-09-19T11:23:42.943735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.194246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368608+0000","last_change":"2024-09-19T11:23:42.282153+0000","last_active":"2024-09-19T11:26:12.368608+0000","last_peered":"2024-09-19T11:26:12.368608+0000","last_clean":"2024-09-19T11:26:12.368608+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:12.368608+0000","last_undegraded":"2024-09-19T11:26:12.368608+0000","last_fullsized":"2024-09-19T11:26:12.368608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.828536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.999397+0000","last_change":"2024-09-19T11:23:03.237858+0000","last_active":"2024-09-19T11:23:44.999397+0000","last_peered":"2024-09-19T11:23:44.999397+0000","last_clean":"2024-09-19T11:23:44.999397+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:23:44.999397+0000","last_undegraded":"2024-09-19T11:23:44.999397+0000","last_fullsized":"2024-09-19T11:23:44.999397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:38.094142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280358+0000","last_change":"2024-09-19T11:23:07.245080+0000","last_active":"2024-09-19T11:23:42.280358+0000","last_peered":"2024-09-19T11:23:42.280358+0000","last_clean":"2024-09-19T11:23:42.280358+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:23:42.280358+0000","last_undegraded":"2024-09-19T11:23:42.280358+0000","last_fullsized":"2024-09-19T11:23:42.280358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:41.815514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'29","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.120252+0000","last_change":"2024-09-19T11:23:09.258685+0000","last_active":"2024-09-19T11:26:19.120252+0000","last_peered":"2024-09-19T11:26:19.120252+0000","last_clean":"2024-09-19T11:26:19.120252+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:26:19.120252+0000","last_undegraded":"2024-09-19T11:26:19.120252+0000","last_fullsized":"2024-09-19T11:26:19.120252+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:33.508526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944317+0000","last_change":"2024-09-19T11:23:11.267904+0000","last_active":"2024-09-19T11:23:42.944317+0000","last_peered":"2024-09-19T11:23:42.944317+0000","last_clean":"2024-09-19T11:23:42.944317+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:23:42.944317+0000","last_undegraded":"2024-09-19T11:23:42.944317+0000","last_fullsized":"2024-09-19T11:23:42.944317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:16.874318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032486+0000","last_change":"2024-09-19T11:23:05.235573+0000","last_active":"2024-09-19T11:25:08.032486+0000","last_peered":"2024-09-19T11:25:08.032486+0000","last_clean":"2024-09-19T11:25:08.032486+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:25:08.032486+0000","last_undegraded":"2024-09-19T11:25:08.032486+0000","last_fullsized":"2024-09-19T11:25:08.032486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:07.062938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":46,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.996626+0000","last_change":"2024-09-19T11:23:03.249699+0000","last_active":"2024-09-19T11:23:44.996626+0000","last_peered":"2024-09-19T11:23:44.996626+0000","last_clean":"2024-09-19T11:23:44.996626+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:23:44.996626+0000","last_undegraded":"2024-09-19T11:23:44.996626+0000","last_fullsized":"2024-09-19T11:23:44.996626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:36.434981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'2","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.365932+0000","last_change":"2024-09-19T11:23:07.247476+0000","last_active":"2024-09-19T11:25:08.365932+0000","last_peered":"2024-09-19T11:25:08.365932+0000","last_clean":"2024-09-19T11:25:08.365932+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:25:08.365932+0000","last_undegraded":"2024-09-19T11:25:08.365932+0000","last_fullsized":"2024-09-19T11:25:08.365932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:03.848364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'37","reported_seq":86,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.129686+0000","last_change":"2024-09-19T11:23:09.255745+0000","last_active":"2024-09-19T11:26:19.129686+0000","last_peered":"2024-09-19T11:26:19.129686+0000","last_clean":"2024-09-19T11:26:19.129686+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:26:19.129686+0000","last_undegraded":"2024-09-19T11:26:19.129686+0000","last_fullsized":"2024-09-19T11:26:19.129686+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:35.538204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279071+0000","last_change":"2024-09-19T11:23:11.266117+0000","last_active":"2024-09-19T11:23:42.279071+0000","last_peered":"2024-09-19T11:23:42.279071+0000","last_clean":"2024-09-19T11:23:42.279071+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:23:42.279071+0000","last_undegraded":"2024-09-19T11:23:42.279071+0000","last_fullsized":"2024-09-19T11:23:42.279071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:22.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550414+0000","last_change":"2024-09-19T11:23:05.252243+0000","last_active":"2024-09-19T11:24:27.550414+0000","last_peered":"2024-09-19T11:24:27.550414+0000","last_clean":"2024-09-19T11:24:27.550414+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:24:27.550414+0000","last_undegraded":"2024-09-19T11:24:27.550414+0000","last_fullsized":"2024-09-19T11:24:27.550414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:18.179391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.276311+0000","last_change":"2024-09-19T11:23:03.257178+0000","last_active":"2024-09-19T11:23:42.276311+0000","last_peered":"2024-09-19T11:23:42.276311+0000","last_clean":"2024-09-19T11:23:42.276311+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:23:42.276311+0000","last_undegraded":"2024-09-19T11:23:42.276311+0000","last_fullsized":"2024-09-19T11:23:42.276311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:13.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.024977+0000","last_change":"2024-09-19T11:23:05.238242+0000","last_active":"2024-09-19T11:26:12.024977+0000","last_peered":"2024-09-19T11:26:12.024977+0000","last_clean":"2024-09-19T11:26:12.024977+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:12.024977+0000","last_undegraded":"2024-09-19T11:26:12.024977+0000","last_fullsized":"2024-09-19T11:26:12.024977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278230+0000","last_change":"2024-09-19T11:23:03.238163+0000","last_active":"2024-09-19T11:23:42.278230+0000","last_peered":"2024-09-19T11:23:42.278230+0000","last_clean":"2024-09-19T11:23:42.278230+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:23:42.278230+0000","last_undegraded":"2024-09-19T11:23:42.278230+0000","last_fullsized":"2024-09-19T11:23:42.278230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:49:17.258919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342000+0000","last_change":"2024-09-19T11:23:05.237372+0000","last_active":"2024-09-19T11:24:27.342000+0000","last_peered":"2024-09-19T11:24:27.342000+0000","last_clean":"2024-09-19T11:24:27.342000+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:24:27.342000+0000","last_undegraded":"2024-09-19T11:24:27.342000+0000","last_fullsized":"2024-09-19T11:24:27.342000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:27.975988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940912+0000","last_change":"2024-09-19T11:23:03.256664+0000","last_active":"2024-09-19T11:23:43.940912+0000","last_peered":"2024-09-19T11:23:43.940912+0000","last_clean":"2024-09-19T11:23:43.940912+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:23:43.940912+0000","last_undegraded":"2024-09-19T11:23:43.940912+0000","last_fullsized":"2024-09-19T11:23:43.940912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:24.307449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.011182+0000","last_change":"2024-09-19T11:23:05.237761+0000","last_active":"2024-09-19T11:23:45.011182+0000","last_peered":"2024-09-19T11:23:45.011182+0000","last_clean":"2024-09-19T11:23:45.011182+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:23:45.011182+0000","last_undegraded":"2024-09-19T11:23:45.011182+0000","last_fullsized":"2024-09-19T11:23:45.011182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:29:24.776293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281499+0000","last_change":"2024-09-19T11:23:03.243433+0000","last_active":"2024-09-19T11:23:42.281499+0000","last_peered":"2024-09-19T11:23:42.281499+0000","last_clean":"2024-09-19T11:23:42.281499+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:23:42.281499+0000","last_undegraded":"2024-09-19T11:23:42.281499+0000","last_fullsized":"2024-09-19T11:23:42.281499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:45.982872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338525+0000","last_change":"2024-09-19T11:23:05.240645+0000","last_active":"2024-09-19T11:26:12.338525+0000","last_peered":"2024-09-19T11:26:12.338525+0000","last_clean":"2024-09-19T11:26:12.338525+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:12.338525+0000","last_undegraded":"2024-09-19T11:26:12.338525+0000","last_fullsized":"2024-09-19T11:26:12.338525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.997259+0000","last_change":"2024-09-19T11:23:03.234311+0000","last_active":"2024-09-19T11:26:11.997259+0000","last_peered":"2024-09-19T11:26:11.997259+0000","last_clean":"2024-09-19T11:26:11.997259+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:11.997259+0000","last_undegraded":"2024-09-19T11:26:11.997259+0000","last_fullsized":"2024-09-19T11:26:11.997259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:04:10.195241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342109+0000","last_change":"2024-09-19T11:23:05.251796+0000","last_active":"2024-09-19T11:24:27.342109+0000","last_peered":"2024-09-19T11:24:27.342109+0000","last_clean":"2024-09-19T11:24:27.342109+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:24:27.342109+0000","last_undegraded":"2024-09-19T11:24:27.342109+0000","last_fullsized":"2024-09-19T11:24:27.342109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:53.953637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281786+0000","last_change":"2024-09-19T11:23:03.248461+0000","last_active":"2024-09-19T11:23:42.281786+0000","last_peered":"2024-09-19T11:23:42.281786+0000","last_clean":"2024-09-19T11:23:42.281786+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:23:42.281786+0000","last_undegraded":"2024-09-19T11:23:42.281786+0000","last_fullsized":"2024-09-19T11:23:42.281786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:40.877834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279051+0000","last_change":"2024-09-19T11:23:05.248747+0000","last_active":"2024-09-19T11:23:42.279051+0000","last_peered":"2024-09-19T11:23:42.279051+0000","last_clean":"2024-09-19T11:23:42.279051+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:23:42.279051+0000","last_undegraded":"2024-09-19T11:23:42.279051+0000","last_fullsized":"2024-09-19T11:23:42.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.258514+0000","last_change":"2024-09-19T11:23:03.257927+0000","last_active":"2024-09-19T11:26:12.258514+0000","last_peered":"2024-09-19T11:26:12.258514+0000","last_clean":"2024-09-19T11:26:12.258514+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:12.258514+0000","last_undegraded":"2024-09-19T11:26:12.258514+0000","last_fullsized":"2024-09-19T11:26:12.258514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:24.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374459+0000","last_change":"2024-09-19T11:23:05.248818+0000","last_active":"2024-09-19T11:26:12.374459+0000","last_peered":"2024-09-19T11:26:12.374459+0000","last_clean":"2024-09-19T11:26:12.374459+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:12.374459+0000","last_undegraded":"2024-09-19T11:26:12.374459+0000","last_fullsized":"2024-09-19T11:26:12.374459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:41.343464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280250+0000","last_change":"2024-09-19T11:23:03.238591+0000","last_active":"2024-09-19T11:23:42.280250+0000","last_peered":"2024-09-19T11:23:42.280250+0000","last_clean":"2024-09-19T11:23:42.280250+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:23:42.280250+0000","last_undegraded":"2024-09-19T11:23:42.280250+0000","last_fullsized":"2024-09-19T11:23:42.280250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:42.869904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342282+0000","last_change":"2024-09-19T11:23:05.248696+0000","last_active":"2024-09-19T11:24:27.342282+0000","last_peered":"2024-09-19T11:24:27.342282+0000","last_clean":"2024-09-19T11:24:27.342282+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:24:27.342282+0000","last_undegraded":"2024-09-19T11:24:27.342282+0000","last_fullsized":"2024-09-19T11:24:27.342282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:37:55.133352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944801+0000","last_change":"2024-09-19T11:23:03.255783+0000","last_active":"2024-09-19T11:23:42.944801+0000","last_peered":"2024-09-19T11:23:42.944801+0000","last_clean":"2024-09-19T11:23:42.944801+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:23:42.944801+0000","last_undegraded":"2024-09-19T11:23:42.944801+0000","last_fullsized":"2024-09-19T11:23:42.944801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:24.578577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368100+0000","last_change":"2024-09-19T11:23:05.243797+0000","last_active":"2024-09-19T11:26:12.368100+0000","last_peered":"2024-09-19T11:26:12.368100+0000","last_clean":"2024-09-19T11:26:12.368100+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:26:12.368100+0000","last_undegraded":"2024-09-19T11:26:12.368100+0000","last_fullsized":"2024-09-19T11:26:12.368100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:33.524342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278358+0000","last_change":"2024-09-19T11:23:03.243321+0000","last_active":"2024-09-19T11:23:42.278358+0000","last_peered":"2024-09-19T11:23:42.278358+0000","last_clean":"2024-09-19T11:23:42.278358+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:23:42.278358+0000","last_undegraded":"2024-09-19T11:23:42.278358+0000","last_fullsized":"2024-09-19T11:23:42.278358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:30.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342015+0000","last_change":"2024-09-19T11:23:05.241366+0000","last_active":"2024-09-19T11:24:27.342015+0000","last_peered":"2024-09-19T11:24:27.342015+0000","last_clean":"2024-09-19T11:24:27.342015+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:24:27.342015+0000","last_undegraded":"2024-09-19T11:24:27.342015+0000","last_fullsized":"2024-09-19T11:24:27.342015+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.002195+0000","last_change":"2024-09-19T11:23:03.249920+0000","last_active":"2024-09-19T11:23:45.002195+0000","last_peered":"2024-09-19T11:23:45.002195+0000","last_clean":"2024-09-19T11:23:45.002195+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:23:45.002195+0000","last_undegraded":"2024-09-19T11:23:45.002195+0000","last_fullsized":"2024-09-19T11:23:45.002195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:45.629786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.482864+0000","last_change":"2024-09-19T11:23:05.246852+0000","last_active":"2024-09-19T11:23:45.482864+0000","last_peered":"2024-09-19T11:23:45.482864+0000","last_clean":"2024-09-19T11:23:45.482864+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:23:45.482864+0000","last_undegraded":"2024-09-19T11:23:45.482864+0000","last_fullsized":"2024-09-19T11:23:45.482864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.057981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278908+0000","last_change":"2024-09-19T11:23:03.243727+0000","last_active":"2024-09-19T11:23:42.278908+0000","last_peered":"2024-09-19T11:23:42.278908+0000","last_clean":"2024-09-19T11:23:42.278908+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:23:42.278908+0000","last_undegraded":"2024-09-19T11:23:42.278908+0000","last_fullsized":"2024-09-19T11:23:42.278908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:52:42.605160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433129+0000","last_change":"2024-09-19T11:23:05.245754+0000","last_active":"2024-09-19T11:23:44.433129+0000","last_peered":"2024-09-19T11:23:44.433129+0000","last_clean":"2024-09-19T11:23:44.433129+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:23:44.433129+0000","last_undegraded":"2024-09-19T11:23:44.433129+0000","last_fullsized":"2024-09-19T11:23:44.433129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:22.898396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:03.241382+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:01.554767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266030+0000","last_change":"2024-09-19T11:23:05.248307+0000","last_active":"2024-09-19T11:26:12.266030+0000","last_peered":"2024-09-19T11:26:12.266030+0000","last_clean":"2024-09-19T11:26:12.266030+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:12.266030+0000","last_undegraded":"2024-09-19T11:26:12.266030+0000","last_fullsized":"2024-09-19T11:26:12.266030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:44.091824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946371+0000","last_change":"2024-09-19T11:23:03.256127+0000","last_active":"2024-09-19T11:23:42.946371+0000","last_peered":"2024-09-19T11:23:42.946371+0000","last_clean":"2024-09-19T11:23:42.946371+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:23:42.946371+0000","last_undegraded":"2024-09-19T11:23:42.946371+0000","last_fullsized":"2024-09-19T11:23:42.946371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:03.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123790+0000","last_change":"2024-09-19T11:23:05.250503+0000","last_active":"2024-09-19T11:26:12.123790+0000","last_peered":"2024-09-19T11:26:12.123790+0000","last_clean":"2024-09-19T11:26:12.123790+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:26:12.123790+0000","last_undegraded":"2024-09-19T11:26:12.123790+0000","last_fullsized":"2024-09-19T11:26:12.123790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:46.547197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.993579+0000","last_change":"2024-09-19T11:23:03.259032+0000","last_active":"2024-09-19T11:23:44.993579+0000","last_peered":"2024-09-19T11:23:44.993579+0000","last_clean":"2024-09-19T11:23:44.993579+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:23:44.993579+0000","last_undegraded":"2024-09-19T11:23:44.993579+0000","last_fullsized":"2024-09-19T11:23:44.993579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:06.246922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368312+0000","last_change":"2024-09-19T11:23:05.248701+0000","last_active":"2024-09-19T11:26:12.368312+0000","last_peered":"2024-09-19T11:26:12.368312+0000","last_clean":"2024-09-19T11:26:12.368312+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:12.368312+0000","last_undegraded":"2024-09-19T11:26:12.368312+0000","last_fullsized":"2024-09-19T11:26:12.368312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:20.161230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947585+0000","last_change":"2024-09-19T11:23:03.252640+0000","last_active":"2024-09-19T11:23:42.947585+0000","last_peered":"2024-09-19T11:23:42.947585+0000","last_clean":"2024-09-19T11:23:42.947585+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:23:42.947585+0000","last_undegraded":"2024-09-19T11:23:42.947585+0000","last_fullsized":"2024-09-19T11:23:42.947585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:25.619002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.938414+0000","last_change":"2024-09-19T11:23:03.256515+0000","last_active":"2024-09-19T11:26:11.938414+0000","last_peered":"2024-09-19T11:26:11.938414+0000","last_clean":"2024-09-19T11:26:11.938414+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:26:11.938414+0000","last_undegraded":"2024-09-19T11:26:11.938414+0000","last_fullsized":"2024-09-19T11:26:11.938414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:08.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123873+0000","last_change":"2024-09-19T11:23:05.248691+0000","last_active":"2024-09-19T11:26:12.123873+0000","last_peered":"2024-09-19T11:26:12.123873+0000","last_clean":"2024-09-19T11:26:12.123873+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:12.123873+0000","last_undegraded":"2024-09-19T11:26:12.123873+0000","last_fullsized":"2024-09-19T11:26:12.123873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:20.035436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278475+0000","last_change":"2024-09-19T11:23:03.243676+0000","last_active":"2024-09-19T11:23:42.278475+0000","last_peered":"2024-09-19T11:23:42.278475+0000","last_clean":"2024-09-19T11:23:42.278475+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:23:42.278475+0000","last_undegraded":"2024-09-19T11:23:42.278475+0000","last_fullsized":"2024-09-19T11:23:42.278475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:25:37.230042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510844+0000","last_change":"2024-09-19T11:23:05.245906+0000","last_active":"2024-09-19T11:25:11.510844+0000","last_peered":"2024-09-19T11:25:11.510844+0000","last_clean":"2024-09-19T11:25:11.510844+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:25:11.510844+0000","last_undegraded":"2024-09-19T11:25:11.510844+0000","last_fullsized":"2024-09-19T11:25:11.510844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:19.236446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1374,"ondisk_log_size":1374,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673006,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":150572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673006,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":737565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705710,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40300,"kb_used_data":13276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331540,"statfs":{"total":96636764160,"available":96595496960,"internally_reserved":0,"allocated":13594624,"data_stored":13323694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705710,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":442368,"data_stored":148560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:26:23.815 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.815 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.815 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.815 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.815 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.816 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.817 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.818 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.819 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.820 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.820 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.820 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.820 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.820 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.820 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.821 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.822 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.823 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.824 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.824 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.824 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.824 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.824 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.824 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.825 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.826 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.827 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-19T11:22:45.784999+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=45, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.828 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.829 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.830 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.831 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.832 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-19T11:22:52.801931+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=22, tm_sec=52, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.833 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.834 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.834 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.834 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.834 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.834 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.835 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.836 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.837 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.838 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.839 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.840 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.840 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.840 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.840 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.840 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.840 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.840 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.841 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.842 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.843 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.844 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.845 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:23.846 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T11:26:43.847 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:26:43.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f71640640 1 Processor -- start 2024-09-19T11:26:43.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f71640640 1 -- start start 2024-09-19T11:26:43.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f71640640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f7f6c095a90 0x7f7f6c135040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:43.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f71640640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6c096f00 con 0x7f7f6c094860 2024-09-19T11:26:43.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f71640640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f7f6c074220 con 0x7f7f6c095a90 2024-09-19T11:26:43.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f6affd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f7f6c094860 0x7f7f6c095380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:38342/0 (socket says 172.21.15.22:38342) 2024-09-19T11:26:43.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f6affd640 1 -- 172.21.15.22:0/4026317429 learned_addr learned my addr 172.21.15.22:0/4026317429 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:26:43.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.960+0000 7f7f627fc640 1 --2- 172.21.15.22:0/4026317429 >> v2:172.21.15.22:3300/0 conn(0x7f7f6c095a90 0x7f7f6c135040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:43.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.961+0000 7f7f6a7fc640 1 -- 172.21.15.22:0/4026317429 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 292865883 0 0) 0x7f7f6c096f00 con 0x7f7f6c094860 2024-09-19T11:26:43.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.961+0000 7f7f6a7fc640 1 -- 172.21.15.22:0/4026317429 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f58003520 con 0x7f7f6c094860 2024-09-19T11:26:43.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.961+0000 7f7f627fc640 1 -- 172.21.15.22:0/4026317429 >> v1:172.21.15.22:6789/0 conn(0x7f7f6c094860 legacy=0x7f7f6c095380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:43.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.961+0000 7f7f627fc640 1 -- 172.21.15.22:0/4026317429 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c0696a0 con 0x7f7f6c095a90 2024-09-19T11:26:43.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.961+0000 7f7f627fc640 1 --2- 172.21.15.22:0/4026317429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c095a90 0x7f7f6c135040 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009720 tx=0x7f7f5402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153912eb572e51ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:43.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f6a7fc640 1 -- 172.21.15.22:0/4026317429 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f54037070 con 0x7f7f6c095a90 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f6a7fc640 1 -- 172.21.15.22:0/4026317429 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5402ad90 con 0x7f7f6c095a90 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f6a7fc640 1 -- 172.21.15.22:0/4026317429 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f54032590 con 0x7f7f6c095a90 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f71640640 1 -- 172.21.15.22:0/4026317429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c095a90 msgr2=0x7f7f6c135040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f71640640 1 --2- 172.21.15.22:0/4026317429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c095a90 0x7f7f6c135040 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009720 tx=0x7f7f5402a110 comp rx=0 tx=0).stop 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f71640640 1 -- 172.21.15.22:0/4026317429 shutdown_connections 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f71640640 1 --2- 172.21.15.22:0/4026317429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c095a90 0x7f7f6c135040 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f71640640 1 -- 172.21.15.22:0/4026317429 >> 172.21.15.22:0/4026317429 conn(0x7f7f6c09e060 msgr2=0x7f7f6c09e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f71640640 1 -- 172.21.15.22:0/4026317429 shutdown_connections 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.962+0000 7f7f71640640 1 -- 172.21.15.22:0/4026317429 wait complete. 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.963+0000 7f7f71640640 1 Processor -- start 2024-09-19T11:26:43.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.963+0000 7f7f71640640 1 -- start start 2024-09-19T11:26:43.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f71640640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c094860 0x7f7f6c1a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:43.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f71640640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c095f00 con 0x7f7f6c094860 2024-09-19T11:26:43.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c094860 0x7f7f6c1a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:43.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f6affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c094860 0x7f7f6c1a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53362/0 (socket says 172.21.15.22:53362) 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f6affd640 1 -- 172.21.15.22:0/282950200 learned_addr learned my addr 172.21.15.22:0/282950200 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f6affd640 1 -- 172.21.15.22:0/282950200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c097da0 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f6affd640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c094860 0x7f7f6c1a8f30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c480 tx=0x7f7f5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f63fff640 1 -- 172.21.15.22:0/282950200 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c017070 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f63fff640 1 -- 172.21.15.22:0/282950200 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5c005cd0 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c1ab270 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f63fff640 1 -- 172.21.15.22:0/282950200 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c007200 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.964+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c1aa570 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.965+0000 7f7f63fff640 1 -- 172.21.15.22:0/282950200 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f7f5c01a070 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.965+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c095380 con 0x7f7f6c094860 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.966+0000 7f7f63fff640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f4c03e710 0x7f7f4c040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:26:43.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.966+0000 7f7f63fff640 1 -- 172.21.15.22:0/282950200 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f7f5c052b10 con 0x7f7f6c094860 2024-09-19T11:26:43.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.966+0000 7f7f627fc640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f4c03e710 0x7f7f4c040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:26:43.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.967+0000 7f7f627fc640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f4c03e710 0x7f7f4c040bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7f54034000 tx=0x7f7f5402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:26:43.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:43.971+0000 7f7f63fff640 1 -- 172.21.15.22:0/282950200 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f7f5c01bdc0 con 0x7f7f6c094860 2024-09-19T11:26:44.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.155+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f6c1c18e0 con 0x7f7f4c03e710 2024-09-19T11:26:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.166+0000 7f7f63fff640 1 -- 172.21.15.22:0/282950200 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665739 (secure 0 0 0) 0x7f7f6c1c18e0 con 0x7f7f4c03e710 2024-09-19T11:26:44.178 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:26:44.184 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:26:44.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f4c03e710 msgr2=0x7f7f4c040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:44.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f4c03e710 0x7f7f4c040bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7f54034000 tx=0x7f7f5402e000 comp rx=0 tx=0).stop 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c094860 msgr2=0x7f7f6c1a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c094860 0x7f7f6c1a8f30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c480 tx=0x7f7f5c00c940 comp rx=0 tx=0).stop 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 shutdown_connections 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f7f4c03e710 0x7f7f4c040bd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 --2- 172.21.15.22:0/282950200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f6c094860 0x7f7f6c1a8f30 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 >> 172.21.15.22:0/282950200 conn(0x7f7f6c09e060 msgr2=0x7f7f6c099710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 shutdown_connections 2024-09-19T11:26:44.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:26:44.171+0000 7f7f71640640 1 -- 172.21.15.22:0/282950200 wait complete. 2024-09-19T11:26:44.189 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-19T11:26:42.851051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605410,"num_objects":222,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":0,"num_read":5189,"num_read_kb":4731,"num_write":2121,"num_write_kb":60587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4568,"ondisk_log_size":4568,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":115136,"kb_used_data":2880,"kb_used_omap":6,"kb_used_meta":112185,"kb_avail":377372224,"statfs":{"total":386547056640,"available":386429157376,"internally_reserved":0,"allocated":2949120,"data_stored":1775768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":114878230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004811"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.932354+0000","last_change":"2024-09-19T11:26:37.932354+0000","last_active":"2024-09-19T11:26:37.932354+0000","last_peered":"2024-09-19T11:26:37.932354+0000","last_clean":"2024-09-19T11:26:37.932354+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:26:37.932354+0000","last_undegraded":"2024-09-19T11:26:37.932354+0000","last_fullsized":"2024-09-19T11:26:37.932354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.932266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:41.727377+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333097+0000","last_change":"2024-09-19T11:23:05.248495+0000","last_active":"2024-09-19T11:26:12.333097+0000","last_peered":"2024-09-19T11:26:12.333097+0000","last_clean":"2024-09-19T11:26:12.333097+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:12.333097+0000","last_undegraded":"2024-09-19T11:26:12.333097+0000","last_fullsized":"2024-09-19T11:26:12.333097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:27.929797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.990885+0000","last_change":"2024-09-19T11:26:36.990885+0000","last_active":"2024-09-19T11:26:36.990885+0000","last_peered":"2024-09-19T11:26:36.990885+0000","last_clean":"2024-09-19T11:26:36.990885+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:26:36.990885+0000","last_undegraded":"2024-09-19T11:26:36.990885+0000","last_fullsized":"2024-09-19T11:26:36.990885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.990795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:53.995221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333061+0000","last_change":"2024-09-19T11:23:05.242656+0000","last_active":"2024-09-19T11:26:12.333061+0000","last_peered":"2024-09-19T11:26:12.333061+0000","last_clean":"2024-09-19T11:26:12.333061+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:26:12.333061+0000","last_undegraded":"2024-09-19T11:26:12.333061+0000","last_fullsized":"2024-09-19T11:26:12.333061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.791517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.022905+0000","last_change":"2024-09-19T11:23:03.252002+0000","last_active":"2024-09-19T11:25:08.022905+0000","last_peered":"2024-09-19T11:25:08.022905+0000","last_clean":"2024-09-19T11:25:08.022905+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:25:08.022905+0000","last_undegraded":"2024-09-19T11:25:08.022905+0000","last_fullsized":"2024-09-19T11:25:08.022905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:04:47.977517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'11","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.470023+0000","last_change":"2024-09-19T11:23:42.285084+0000","last_active":"2024-09-19T11:26:12.470023+0000","last_peered":"2024-09-19T11:26:12.470023+0000","last_clean":"2024-09-19T11:26:12.470023+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:12.470023+0000","last_undegraded":"2024-09-19T11:26:12.470023+0000","last_fullsized":"2024-09-19T11:26:12.470023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:31.524562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.958139+0000","last_change":"2024-09-19T11:26:36.958139+0000","last_active":"2024-09-19T11:26:36.958139+0000","last_peered":"2024-09-19T11:26:36.958139+0000","last_clean":"2024-09-19T11:26:36.958139+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:26:36.958139+0000","last_undegraded":"2024-09-19T11:26:36.958139+0000","last_fullsized":"2024-09-19T11:26:36.958139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.958025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:34.544025+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944750+0000","last_change":"2024-09-19T11:23:05.233162+0000","last_active":"2024-09-19T11:23:42.944750+0000","last_peered":"2024-09-19T11:23:42.944750+0000","last_clean":"2024-09-19T11:23:42.944750+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:23:42.944750+0000","last_undegraded":"2024-09-19T11:23:42.944750+0000","last_fullsized":"2024-09-19T11:23:42.944750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:49.026520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.891678+0000","last_change":"2024-09-19T11:23:03.254504+0000","last_active":"2024-09-19T11:26:11.891678+0000","last_peered":"2024-09-19T11:26:11.891678+0000","last_clean":"2024-09-19T11:26:11.891678+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:26:11.891678+0000","last_undegraded":"2024-09-19T11:26:11.891678+0000","last_fullsized":"2024-09-19T11:26:11.891678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:05.223629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'11","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.128818+0000","last_change":"2024-09-19T11:23:05.247147+0000","last_active":"2024-09-19T11:26:12.128818+0000","last_peered":"2024-09-19T11:26:12.128818+0000","last_clean":"2024-09-19T11:26:12.128818+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:12.128818+0000","last_undegraded":"2024-09-19T11:26:12.128818+0000","last_fullsized":"2024-09-19T11:26:12.128818+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:25.162001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.972860+0000","last_change":"2024-09-19T11:26:35.972860+0000","last_active":"2024-09-19T11:26:35.972860+0000","last_peered":"2024-09-19T11:26:35.972860+0000","last_clean":"2024-09-19T11:26:35.972860+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:26:35.972860+0000","last_undegraded":"2024-09-19T11:26:35.972860+0000","last_fullsized":"2024-09-19T11:26:35.972860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.972760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:17.815936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374918+0000","last_change":"2024-09-19T11:23:05.254262+0000","last_active":"2024-09-19T11:26:12.374918+0000","last_peered":"2024-09-19T11:26:12.374918+0000","last_clean":"2024-09-19T11:26:12.374918+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:26:12.374918+0000","last_undegraded":"2024-09-19T11:26:12.374918+0000","last_fullsized":"2024-09-19T11:26:12.374918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:58.352519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.252828+0000","last_change":"2024-09-19T11:23:03.239664+0000","last_active":"2024-09-19T11:26:12.252828+0000","last_peered":"2024-09-19T11:26:12.252828+0000","last_clean":"2024-09-19T11:26:12.252828+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:12.252828+0000","last_undegraded":"2024-09-19T11:26:12.252828+0000","last_fullsized":"2024-09-19T11:26:12.252828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:43.733405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333282+0000","last_change":"2024-09-19T11:23:05.250554+0000","last_active":"2024-09-19T11:26:12.333282+0000","last_peered":"2024-09-19T11:26:12.333282+0000","last_clean":"2024-09-19T11:26:12.333282+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:12.333282+0000","last_undegraded":"2024-09-19T11:26:12.333282+0000","last_fullsized":"2024-09-19T11:26:12.333282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.989858+0000","last_change":"2024-09-19T11:26:35.989858+0000","last_active":"2024-09-19T11:26:35.989858+0000","last_peered":"2024-09-19T11:26:35.989858+0000","last_clean":"2024-09-19T11:26:35.989858+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:26:35.989858+0000","last_undegraded":"2024-09-19T11:26:35.989858+0000","last_fullsized":"2024-09-19T11:26:35.989858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.989770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.409598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.012417+0000","last_change":"2024-09-19T11:23:05.246773+0000","last_active":"2024-09-19T11:23:45.012417+0000","last_peered":"2024-09-19T11:23:45.012417+0000","last_clean":"2024-09-19T11:23:45.012417+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:23:45.012417+0000","last_undegraded":"2024-09-19T11:23:45.012417+0000","last_fullsized":"2024-09-19T11:23:45.012417+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:05:02.041530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:05.241849+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:04.673024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.946461+0000","last_change":"2024-09-19T11:26:34.946461+0000","last_active":"2024-09-19T11:26:34.946461+0000","last_peered":"2024-09-19T11:26:34.946461+0000","last_clean":"2024-09-19T11:26:34.946461+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:26:34.946461+0000","last_undegraded":"2024-09-19T11:26:34.946461+0000","last_fullsized":"2024-09-19T11:26:34.946461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.946370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:48.027671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.483130+0000","last_change":"2024-09-19T11:23:05.246402+0000","last_active":"2024-09-19T11:23:45.483130+0000","last_peered":"2024-09-19T11:23:45.483130+0000","last_clean":"2024-09-19T11:23:45.483130+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:23:45.483130+0000","last_undegraded":"2024-09-19T11:23:45.483130+0000","last_fullsized":"2024-09-19T11:23:45.483130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:44.280015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.952151+0000","last_change":"2024-09-19T11:26:34.952151+0000","last_active":"2024-09-19T11:26:34.952151+0000","last_peered":"2024-09-19T11:26:34.952151+0000","last_clean":"2024-09-19T11:26:34.952151+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:26:34.952151+0000","last_undegraded":"2024-09-19T11:26:34.952151+0000","last_fullsized":"2024-09-19T11:26:34.952151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.952058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:03.872887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342502+0000","last_change":"2024-09-19T11:23:05.248326+0000","last_active":"2024-09-19T11:24:27.342502+0000","last_peered":"2024-09-19T11:24:27.342502+0000","last_clean":"2024-09-19T11:24:27.342502+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:24:27.342502+0000","last_undegraded":"2024-09-19T11:24:27.342502+0000","last_fullsized":"2024-09-19T11:24:27.342502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:49:28.400630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.916754+0000","last_change":"2024-09-19T11:26:33.916754+0000","last_active":"2024-09-19T11:26:33.916754+0000","last_peered":"2024-09-19T11:26:33.916754+0000","last_clean":"2024-09-19T11:26:33.916754+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:33.916754+0000","last_undegraded":"2024-09-19T11:26:33.916754+0000","last_fullsized":"2024-09-19T11:26:33.916754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.916663+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:35.684322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123608+0000","last_change":"2024-09-19T11:23:05.240918+0000","last_active":"2024-09-19T11:26:12.123608+0000","last_peered":"2024-09-19T11:26:12.123608+0000","last_clean":"2024-09-19T11:26:12.123608+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:12.123608+0000","last_undegraded":"2024-09-19T11:26:12.123608+0000","last_fullsized":"2024-09-19T11:26:12.123608+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:19.590735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.987445+0000","last_change":"2024-09-19T11:26:37.987445+0000","last_active":"2024-09-19T11:26:37.987445+0000","last_peered":"2024-09-19T11:26:37.987445+0000","last_clean":"2024-09-19T11:26:37.987445+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:26:37.987445+0000","last_undegraded":"2024-09-19T11:26:37.987445+0000","last_fullsized":"2024-09-19T11:26:37.987445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.987357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:27.173996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266363+0000","last_change":"2024-09-19T11:23:05.245486+0000","last_active":"2024-09-19T11:26:12.266363+0000","last_peered":"2024-09-19T11:26:12.266363+0000","last_clean":"2024-09-19T11:26:12.266363+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:12.266363+0000","last_undegraded":"2024-09-19T11:26:12.266363+0000","last_fullsized":"2024-09-19T11:26:12.266363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.001114+0000","last_change":"2024-09-19T11:26:35.001114+0000","last_active":"2024-09-19T11:26:35.001114+0000","last_peered":"2024-09-19T11:26:35.001114+0000","last_clean":"2024-09-19T11:26:35.001114+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:26:35.001114+0000","last_undegraded":"2024-09-19T11:26:35.001114+0000","last_fullsized":"2024-09-19T11:26:35.001114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.001027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:09.576343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266347+0000","last_change":"2024-09-19T11:23:05.248431+0000","last_active":"2024-09-19T11:26:12.266347+0000","last_peered":"2024-09-19T11:26:12.266347+0000","last_clean":"2024-09-19T11:26:12.266347+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:12.266347+0000","last_undegraded":"2024-09-19T11:26:12.266347+0000","last_fullsized":"2024-09-19T11:26:12.266347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:17.337349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945193+0000","last_change":"2024-09-19T11:23:03.240547+0000","last_active":"2024-09-19T11:23:42.945193+0000","last_peered":"2024-09-19T11:23:42.945193+0000","last_clean":"2024-09-19T11:23:42.945193+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:23:42.945193+0000","last_undegraded":"2024-09-19T11:23:42.945193+0000","last_fullsized":"2024-09-19T11:23:42.945193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:55.218589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'37","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.305388+0000","last_change":"2024-09-19T11:23:13.275524+0000","last_active":"2024-09-19T11:26:12.305388+0000","last_peered":"2024-09-19T11:26:12.305388+0000","last_clean":"2024-09-19T11:26:12.305388+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:26:12.305388+0000","last_undegraded":"2024-09-19T11:26:12.305388+0000","last_fullsized":"2024-09-19T11:26:12.305388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.670701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.018412+0000","last_change":"2024-09-19T11:23:42.283032+0000","last_active":"2024-09-19T11:26:12.018412+0000","last_peered":"2024-09-19T11:26:12.018412+0000","last_clean":"2024-09-19T11:26:12.018412+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:12.018412+0000","last_undegraded":"2024-09-19T11:26:12.018412+0000","last_fullsized":"2024-09-19T11:26:12.018412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.760502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.365713+0000","last_change":"2024-09-19T11:26:37.365713+0000","last_active":"2024-09-19T11:26:37.365713+0000","last_peered":"2024-09-19T11:26:37.365713+0000","last_clean":"2024-09-19T11:26:37.365713+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:26:37.365713+0000","last_undegraded":"2024-09-19T11:26:37.365713+0000","last_fullsized":"2024-09-19T11:26:37.365713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.365625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:20.032640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945113+0000","last_change":"2024-09-19T11:23:07.245040+0000","last_active":"2024-09-19T11:23:42.945113+0000","last_peered":"2024-09-19T11:23:42.945113+0000","last_clean":"2024-09-19T11:23:42.945113+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:23:42.945113+0000","last_undegraded":"2024-09-19T11:23:42.945113+0000","last_fullsized":"2024-09-19T11:23:42.945113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.056714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.204815+0000","last_change":"2024-09-19T11:23:09.256765+0000","last_active":"2024-09-19T11:26:19.204815+0000","last_peered":"2024-09-19T11:26:19.204815+0000","last_clean":"2024-09-19T11:26:19.204815+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:26:19.204815+0000","last_undegraded":"2024-09-19T11:26:19.204815+0000","last_fullsized":"2024-09-19T11:26:19.204815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:39:32.451376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283356+0000","last_change":"2024-09-19T11:23:11.267021+0000","last_active":"2024-09-19T11:23:42.283356+0000","last_peered":"2024-09-19T11:23:42.283356+0000","last_clean":"2024-09-19T11:23:42.283356+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:23:42.283356+0000","last_undegraded":"2024-09-19T11:23:42.283356+0000","last_fullsized":"2024-09-19T11:23:42.283356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.806510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.707113+0000","last_change":"2024-09-19T11:23:13.274325+0000","last_active":"2024-09-19T11:26:11.707113+0000","last_peered":"2024-09-19T11:26:11.707113+0000","last_clean":"2024-09-19T11:26:11.707113+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:26:11.707113+0000","last_undegraded":"2024-09-19T11:26:11.707113+0000","last_fullsized":"2024-09-19T11:26:11.707113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:53:22.517816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.332597+0000","last_change":"2024-09-19T11:23:05.237978+0000","last_active":"2024-09-19T11:26:12.332597+0000","last_peered":"2024-09-19T11:26:12.332597+0000","last_clean":"2024-09-19T11:26:12.332597+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:12.332597+0000","last_undegraded":"2024-09-19T11:26:12.332597+0000","last_fullsized":"2024-09-19T11:26:12.332597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:02.964073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.907786+0000","last_change":"2024-09-19T11:26:30.907786+0000","last_active":"2024-09-19T11:26:30.907786+0000","last_peered":"2024-09-19T11:26:30.907786+0000","last_clean":"2024-09-19T11:26:30.907786+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:26:30.907786+0000","last_undegraded":"2024-09-19T11:26:30.907786+0000","last_fullsized":"2024-09-19T11:26:30.907786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.907688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:01.389670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279716+0000","last_change":"2024-09-19T11:23:07.245213+0000","last_active":"2024-09-19T11:23:42.279716+0000","last_peered":"2024-09-19T11:23:42.279716+0000","last_clean":"2024-09-19T11:23:42.279716+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:23:42.279716+0000","last_undegraded":"2024-09-19T11:23:42.279716+0000","last_fullsized":"2024-09-19T11:23:42.279716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:56.312564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'7","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.551668+0000","last_change":"2024-09-19T11:23:09.248257+0000","last_active":"2024-09-19T11:24:27.551668+0000","last_peered":"2024-09-19T11:24:27.551668+0000","last_clean":"2024-09-19T11:24:27.551668+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:24:27.551668+0000","last_undegraded":"2024-09-19T11:24:27.551668+0000","last_fullsized":"2024-09-19T11:24:27.551668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:18.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281824+0000","last_change":"2024-09-19T11:23:11.263544+0000","last_active":"2024-09-19T11:23:42.281824+0000","last_peered":"2024-09-19T11:23:42.281824+0000","last_clean":"2024-09-19T11:23:42.281824+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:23:42.281824+0000","last_undegraded":"2024-09-19T11:23:42.281824+0000","last_fullsized":"2024-09-19T11:23:42.281824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:44.652949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278313+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:23:42.278313+0000","last_peered":"2024-09-19T11:23:42.278313+0000","last_clean":"2024-09-19T11:23:42.278313+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:23:42.278313+0000","last_undegraded":"2024-09-19T11:23:42.278313+0000","last_fullsized":"2024-09-19T11:23:42.278313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:33:33.618068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510054+0000","last_change":"2024-09-19T11:23:05.236597+0000","last_active":"2024-09-19T11:25:11.510054+0000","last_peered":"2024-09-19T11:25:11.510054+0000","last_clean":"2024-09-19T11:25:11.510054+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:25:11.510054+0000","last_undegraded":"2024-09-19T11:25:11.510054+0000","last_fullsized":"2024-09-19T11:25:11.510054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:11.702021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.378428+0000","last_change":"2024-09-19T11:26:35.378428+0000","last_active":"2024-09-19T11:26:35.378428+0000","last_peered":"2024-09-19T11:26:35.378428+0000","last_clean":"2024-09-19T11:26:35.378428+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:26:35.378428+0000","last_undegraded":"2024-09-19T11:26:35.378428+0000","last_fullsized":"2024-09-19T11:26:35.378428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.378338+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:26.014662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796210+0000","last_change":"2024-09-19T11:23:42.284049+0000","last_active":"2024-09-19T11:23:44.796210+0000","last_peered":"2024-09-19T11:23:44.796210+0000","last_clean":"2024-09-19T11:23:44.796210+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:23:44.796210+0000","last_undegraded":"2024-09-19T11:23:44.796210+0000","last_fullsized":"2024-09-19T11:23:44.796210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279160+0000","last_change":"2024-09-19T11:23:09.253591+0000","last_active":"2024-09-19T11:23:42.279160+0000","last_peered":"2024-09-19T11:23:42.279160+0000","last_clean":"2024-09-19T11:23:42.279160+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:23:42.279160+0000","last_undegraded":"2024-09-19T11:23:42.279160+0000","last_fullsized":"2024-09-19T11:23:42.279160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:52.970909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'285","reported_seq":389,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.073009+0000","last_change":"2024-09-19T11:23:11.271056+0000","last_active":"2024-09-19T11:26:19.073009+0000","last_peered":"2024-09-19T11:26:19.073009+0000","last_clean":"2024-09-19T11:26:19.073009+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:26:19.073009+0000","last_undegraded":"2024-09-19T11:26:19.073009+0000","last_fullsized":"2024-09-19T11:26:19.073009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:07.822761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.312802+0000","last_change":"2024-09-19T11:23:13.270157+0000","last_active":"2024-09-19T11:26:12.312802+0000","last_peered":"2024-09-19T11:26:12.312802+0000","last_clean":"2024-09-19T11:26:12.312802+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:26:12.312802+0000","last_undegraded":"2024-09-19T11:26:12.312802+0000","last_fullsized":"2024-09-19T11:26:12.312802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:32.371537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.781965+0000","last_change":"2024-09-19T11:23:05.248712+0000","last_active":"2024-09-19T11:23:44.781965+0000","last_peered":"2024-09-19T11:23:44.781965+0000","last_clean":"2024-09-19T11:23:44.781965+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:23:44.781965+0000","last_undegraded":"2024-09-19T11:23:44.781965+0000","last_fullsized":"2024-09-19T11:23:44.781965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:01:31.288073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.870975+0000","last_change":"2024-09-19T11:26:24.870975+0000","last_active":"2024-09-19T11:26:24.870975+0000","last_peered":"2024-09-19T11:26:24.870975+0000","last_clean":"2024-09-19T11:26:24.870975+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:26:24.870975+0000","last_undegraded":"2024-09-19T11:26:24.870975+0000","last_fullsized":"2024-09-19T11:26:24.870975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.870884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:14.567909+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947691+0000","last_change":"2024-09-19T11:23:07.248024+0000","last_active":"2024-09-19T11:23:42.947691+0000","last_peered":"2024-09-19T11:23:42.947691+0000","last_clean":"2024-09-19T11:23:42.947691+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:23:42.947691+0000","last_undegraded":"2024-09-19T11:23:42.947691+0000","last_fullsized":"2024-09-19T11:23:42.947691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:02.812050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.189031+0000","last_change":"2024-09-19T11:23:09.260866+0000","last_active":"2024-09-19T11:26:19.189031+0000","last_peered":"2024-09-19T11:26:19.189031+0000","last_clean":"2024-09-19T11:26:19.189031+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:26:19.189031+0000","last_undegraded":"2024-09-19T11:26:19.189031+0000","last_fullsized":"2024-09-19T11:26:19.189031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:22.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'285","reported_seq":405,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.078340+0000","last_change":"2024-09-19T11:23:11.260474+0000","last_active":"2024-09-19T11:26:19.078340+0000","last_peered":"2024-09-19T11:26:19.078340+0000","last_clean":"2024-09-19T11:26:19.078340+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:26:19.078340+0000","last_undegraded":"2024-09-19T11:26:19.078340+0000","last_fullsized":"2024-09-19T11:26:19.078340+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:32.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032873+0000","last_change":"2024-09-19T11:23:05.253843+0000","last_active":"2024-09-19T11:25:08.032873+0000","last_peered":"2024-09-19T11:25:08.032873+0000","last_clean":"2024-09-19T11:25:08.032873+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:25:08.032873+0000","last_undegraded":"2024-09-19T11:25:08.032873+0000","last_fullsized":"2024-09-19T11:25:08.032873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:22.232407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.956170+0000","last_change":"2024-09-19T11:26:27.956170+0000","last_active":"2024-09-19T11:26:27.956170+0000","last_peered":"2024-09-19T11:26:27.956170+0000","last_clean":"2024-09-19T11:26:27.956170+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:26:27.956170+0000","last_undegraded":"2024-09-19T11:26:27.956170+0000","last_fullsized":"2024-09-19T11:26:27.956170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.956081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:59.956862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281170+0000","last_change":"2024-09-19T11:23:07.244371+0000","last_active":"2024-09-19T11:23:42.281170+0000","last_peered":"2024-09-19T11:23:42.281170+0000","last_clean":"2024-09-19T11:23:42.281170+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:23:42.281170+0000","last_undegraded":"2024-09-19T11:23:42.281170+0000","last_fullsized":"2024-09-19T11:23:42.281170+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:53.267571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'43","reported_seq":95,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.217387+0000","last_change":"2024-09-19T11:23:09.249075+0000","last_active":"2024-09-19T11:26:19.217387+0000","last_peered":"2024-09-19T11:26:19.217387+0000","last_clean":"2024-09-19T11:26:19.217387+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:26:19.217387+0000","last_undegraded":"2024-09-19T11:26:19.217387+0000","last_fullsized":"2024-09-19T11:26:19.217387+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:03.176195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281316+0000","last_change":"2024-09-19T11:23:11.258275+0000","last_active":"2024-09-19T11:23:42.281316+0000","last_peered":"2024-09-19T11:23:42.281316+0000","last_clean":"2024-09-19T11:23:42.281316+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:23:42.281316+0000","last_undegraded":"2024-09-19T11:23:42.281316+0000","last_fullsized":"2024-09-19T11:23:42.281316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:14.017912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.349482+0000","last_change":"2024-09-19T11:23:13.274689+0000","last_active":"2024-09-19T11:24:27.349482+0000","last_peered":"2024-09-19T11:24:27.349482+0000","last_clean":"2024-09-19T11:24:27.349482+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:24:27.349482+0000","last_undegraded":"2024-09-19T11:24:27.349482+0000","last_fullsized":"2024-09-19T11:24:27.349482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:05.210803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.014072+0000","last_change":"2024-09-19T11:26:39.014072+0000","last_active":"2024-09-19T11:26:39.014072+0000","last_peered":"2024-09-19T11:26:39.014072+0000","last_clean":"2024-09-19T11:26:39.014072+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:26:39.014072+0000","last_undegraded":"2024-09-19T11:26:39.014072+0000","last_fullsized":"2024-09-19T11:26:39.014072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.013977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:33.627023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.962098+0000","last_change":"2024-09-19T11:26:26.962098+0000","last_active":"2024-09-19T11:26:26.962098+0000","last_peered":"2024-09-19T11:26:26.962098+0000","last_clean":"2024-09-19T11:26:26.962098+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:26:26.962098+0000","last_undegraded":"2024-09-19T11:26:26.962098+0000","last_fullsized":"2024-09-19T11:26:26.962098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.962016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:19.643957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278703+0000","last_change":"2024-09-19T11:23:07.247386+0000","last_active":"2024-09-19T11:23:42.278703+0000","last_peered":"2024-09-19T11:23:42.278703+0000","last_clean":"2024-09-19T11:23:42.278703+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:23:42.278703+0000","last_undegraded":"2024-09-19T11:23:42.278703+0000","last_fullsized":"2024-09-19T11:23:42.278703+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:53.982232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'33","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.158685+0000","last_change":"2024-09-19T11:23:09.262668+0000","last_active":"2024-09-19T11:26:19.158685+0000","last_peered":"2024-09-19T11:26:19.158685+0000","last_clean":"2024-09-19T11:26:19.158685+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:26:19.158685+0000","last_undegraded":"2024-09-19T11:26:19.158685+0000","last_fullsized":"2024-09-19T11:26:19.158685+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:42.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278485+0000","last_change":"2024-09-19T11:23:11.265649+0000","last_active":"2024-09-19T11:23:42.278485+0000","last_peered":"2024-09-19T11:23:42.278485+0000","last_clean":"2024-09-19T11:23:42.278485+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:23:42.278485+0000","last_undegraded":"2024-09-19T11:23:42.278485+0000","last_fullsized":"2024-09-19T11:23:42.278485+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:22.946739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'5","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.124970+0000","last_change":"2024-09-19T11:23:13.268845+0000","last_active":"2024-09-19T11:26:12.124970+0000","last_peered":"2024-09-19T11:26:12.124970+0000","last_clean":"2024-09-19T11:26:12.124970+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:26:12.124970+0000","last_undegraded":"2024-09-19T11:26:12.124970+0000","last_fullsized":"2024-09-19T11:26:12.124970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:30.688522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.889489+0000","last_change":"2024-09-19T11:26:38.889489+0000","last_active":"2024-09-19T11:26:38.889489+0000","last_peered":"2024-09-19T11:26:38.889489+0000","last_clean":"2024-09-19T11:26:38.889489+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:26:38.889489+0000","last_undegraded":"2024-09-19T11:26:38.889489+0000","last_fullsized":"2024-09-19T11:26:38.889489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.889400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:14.595110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.412815+0000","last_change":"2024-09-19T11:26:36.412815+0000","last_active":"2024-09-19T11:26:36.412815+0000","last_peered":"2024-09-19T11:26:36.412815+0000","last_clean":"2024-09-19T11:26:36.412815+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:36.412815+0000","last_undegraded":"2024-09-19T11:26:36.412815+0000","last_fullsized":"2024-09-19T11:26:36.412815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.412703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:59.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281855+0000","last_change":"2024-09-19T11:23:07.247594+0000","last_active":"2024-09-19T11:23:42.281855+0000","last_peered":"2024-09-19T11:23:42.281855+0000","last_clean":"2024-09-19T11:23:42.281855+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:23:42.281855+0000","last_undegraded":"2024-09-19T11:23:42.281855+0000","last_fullsized":"2024-09-19T11:23:42.281855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:41.108832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.222983+0000","last_change":"2024-09-19T11:23:09.259476+0000","last_active":"2024-09-19T11:26:19.222983+0000","last_peered":"2024-09-19T11:26:19.222983+0000","last_clean":"2024-09-19T11:26:19.222983+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:26:19.222983+0000","last_undegraded":"2024-09-19T11:26:19.222983+0000","last_fullsized":"2024-09-19T11:26:19.222983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:21.364894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'285","reported_seq":352,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.079494+0000","last_change":"2024-09-19T11:23:11.267050+0000","last_active":"2024-09-19T11:26:19.079494+0000","last_peered":"2024-09-19T11:26:19.079494+0000","last_clean":"2024-09-19T11:26:19.079494+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:26:19.079494+0000","last_undegraded":"2024-09-19T11:26:19.079494+0000","last_fullsized":"2024-09-19T11:26:19.079494+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.330204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.334604+0000","last_change":"2024-09-19T11:23:13.269663+0000","last_active":"2024-09-19T11:26:12.334604+0000","last_peered":"2024-09-19T11:26:12.334604+0000","last_clean":"2024-09-19T11:26:12.334604+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:26:12.334604+0000","last_undegraded":"2024-09-19T11:26:12.334604+0000","last_fullsized":"2024-09-19T11:26:12.334604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:57:11.742213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368223+0000","last_change":"2024-09-19T11:23:05.235632+0000","last_active":"2024-09-19T11:26:12.368223+0000","last_peered":"2024-09-19T11:26:12.368223+0000","last_clean":"2024-09-19T11:26:12.368223+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:12.368223+0000","last_undegraded":"2024-09-19T11:26:12.368223+0000","last_fullsized":"2024-09-19T11:26:12.368223+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.328550+0000","last_change":"2024-09-19T11:26:34.328550+0000","last_active":"2024-09-19T11:26:34.328550+0000","last_peered":"2024-09-19T11:26:34.328550+0000","last_clean":"2024-09-19T11:26:34.328550+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:26:34.328550+0000","last_undegraded":"2024-09-19T11:26:34.328550+0000","last_fullsized":"2024-09-19T11:26:34.328550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.328463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:54.619678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279784+0000","last_change":"2024-09-19T11:23:07.247239+0000","last_active":"2024-09-19T11:23:42.279784+0000","last_peered":"2024-09-19T11:23:42.279784+0000","last_clean":"2024-09-19T11:23:42.279784+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:23:42.279784+0000","last_undegraded":"2024-09-19T11:23:42.279784+0000","last_fullsized":"2024-09-19T11:23:42.279784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:21.680175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'61","reported_seq":126,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.167477+0000","last_change":"2024-09-19T11:23:09.259576+0000","last_active":"2024-09-19T11:26:19.167477+0000","last_peered":"2024-09-19T11:26:19.167477+0000","last_clean":"2024-09-19T11:26:19.167477+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:26:19.167477+0000","last_undegraded":"2024-09-19T11:26:19.167477+0000","last_fullsized":"2024-09-19T11:26:19.167477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.556001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278323+0000","last_change":"2024-09-19T11:23:11.260403+0000","last_active":"2024-09-19T11:23:42.278323+0000","last_peered":"2024-09-19T11:23:42.278323+0000","last_clean":"2024-09-19T11:23:42.278323+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:23:42.278323+0000","last_undegraded":"2024-09-19T11:23:42.278323+0000","last_fullsized":"2024-09-19T11:23:42.278323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:11.335945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'18","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585256+0000","last_change":"2024-09-19T11:23:13.273720+0000","last_active":"2024-09-19T11:26:12.585256+0000","last_peered":"2024-09-19T11:26:12.585256+0000","last_clean":"2024-09-19T11:26:12.585256+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:26:12.585256+0000","last_undegraded":"2024-09-19T11:26:12.585256+0000","last_fullsized":"2024-09-19T11:26:12.585256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:46.547393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":92,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.048324+0000","last_change":"2024-09-19T11:26:40.048324+0000","last_active":"2024-09-19T11:26:40.048324+0000","last_peered":"2024-09-19T11:26:40.048324+0000","last_clean":"2024-09-19T11:26:40.048324+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:26:40.048324+0000","last_undegraded":"2024-09-19T11:26:40.048324+0000","last_fullsized":"2024-09-19T11:26:40.048324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.048219+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:04:18.644192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.937533+0000","last_change":"2024-09-19T11:26:28.937533+0000","last_active":"2024-09-19T11:26:28.937533+0000","last_peered":"2024-09-19T11:26:28.937533+0000","last_clean":"2024-09-19T11:26:28.937533+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:28.937533+0000","last_undegraded":"2024-09-19T11:26:28.937533+0000","last_fullsized":"2024-09-19T11:26:28.937533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.937444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:51.513741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278469+0000","last_change":"2024-09-19T11:23:07.244885+0000","last_active":"2024-09-19T11:23:42.278469+0000","last_peered":"2024-09-19T11:23:42.278469+0000","last_clean":"2024-09-19T11:23:42.278469+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:23:42.278469+0000","last_undegraded":"2024-09-19T11:23:42.278469+0000","last_fullsized":"2024-09-19T11:23:42.278469+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:45:02.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'36","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.170832+0000","last_change":"2024-09-19T11:23:09.248190+0000","last_active":"2024-09-19T11:26:19.170832+0000","last_peered":"2024-09-19T11:26:19.170832+0000","last_clean":"2024-09-19T11:26:19.170832+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:26:19.170832+0000","last_undegraded":"2024-09-19T11:26:19.170832+0000","last_fullsized":"2024-09-19T11:26:19.170832+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:38.180379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946607+0000","last_change":"2024-09-19T11:23:11.270709+0000","last_active":"2024-09-19T11:23:42.946607+0000","last_peered":"2024-09-19T11:23:42.946607+0000","last_clean":"2024-09-19T11:23:42.946607+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:23:42.946607+0000","last_undegraded":"2024-09-19T11:23:42.946607+0000","last_fullsized":"2024-09-19T11:23:42.946607+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:39.157590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809883+0000","last_change":"2024-09-19T11:23:13.272542+0000","last_active":"2024-09-19T11:25:16.809883+0000","last_peered":"2024-09-19T11:25:16.809883+0000","last_clean":"2024-09-19T11:25:16.809883+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:25:16.809883+0000","last_undegraded":"2024-09-19T11:25:16.809883+0000","last_fullsized":"2024-09-19T11:25:16.809883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:52.793325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433382+0000","last_change":"2024-09-19T11:23:05.254593+0000","last_active":"2024-09-19T11:23:44.433382+0000","last_peered":"2024-09-19T11:23:44.433382+0000","last_clean":"2024-09-19T11:23:44.433382+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:23:44.433382+0000","last_undegraded":"2024-09-19T11:23:44.433382+0000","last_fullsized":"2024-09-19T11:23:44.433382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:29:27.816142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.308889+0000","last_change":"2024-09-19T11:26:33.308889+0000","last_active":"2024-09-19T11:26:33.308889+0000","last_peered":"2024-09-19T11:26:33.308889+0000","last_clean":"2024-09-19T11:26:33.308889+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:26:33.308889+0000","last_undegraded":"2024-09-19T11:26:33.308889+0000","last_fullsized":"2024-09-19T11:26:33.308889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.308801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:19.525046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279479+0000","last_change":"2024-09-19T11:23:07.247411+0000","last_active":"2024-09-19T11:23:42.279479+0000","last_peered":"2024-09-19T11:23:42.279479+0000","last_clean":"2024-09-19T11:23:42.279479+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:23:42.279479+0000","last_undegraded":"2024-09-19T11:23:42.279479+0000","last_fullsized":"2024-09-19T11:23:42.279479+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:20:41.691053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.148513+0000","last_change":"2024-09-19T11:23:09.246515+0000","last_active":"2024-09-19T11:26:19.148513+0000","last_peered":"2024-09-19T11:26:19.148513+0000","last_clean":"2024-09-19T11:26:19.148513+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:26:19.148513+0000","last_undegraded":"2024-09-19T11:26:19.148513+0000","last_fullsized":"2024-09-19T11:26:19.148513+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:54.969396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'285","reported_seq":407,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.074087+0000","last_change":"2024-09-19T11:23:11.263693+0000","last_active":"2024-09-19T11:26:19.074087+0000","last_peered":"2024-09-19T11:26:19.074087+0000","last_clean":"2024-09-19T11:26:19.074087+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:26:19.074087+0000","last_undegraded":"2024-09-19T11:26:19.074087+0000","last_fullsized":"2024-09-19T11:26:19.074087+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:29.980859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'14","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.475120+0000","last_change":"2024-09-19T11:23:13.265074+0000","last_active":"2024-09-19T11:26:12.475120+0000","last_peered":"2024-09-19T11:26:12.475120+0000","last_clean":"2024-09-19T11:26:12.475120+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:26:12.475120+0000","last_undegraded":"2024-09-19T11:26:12.475120+0000","last_fullsized":"2024-09-19T11:26:12.475120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:47.118156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368115+0000","last_change":"2024-09-19T11:23:05.250216+0000","last_active":"2024-09-19T11:26:12.368115+0000","last_peered":"2024-09-19T11:26:12.368115+0000","last_clean":"2024-09-19T11:26:12.368115+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:12.368115+0000","last_undegraded":"2024-09-19T11:26:12.368115+0000","last_fullsized":"2024-09-19T11:26:12.368115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.268981+0000","last_change":"2024-09-19T11:26:31.268981+0000","last_active":"2024-09-19T11:26:31.268981+0000","last_peered":"2024-09-19T11:26:31.268981+0000","last_clean":"2024-09-19T11:26:31.268981+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:26:31.268981+0000","last_undegraded":"2024-09-19T11:26:31.268981+0000","last_fullsized":"2024-09-19T11:26:31.268981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.268893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:31.227733+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'3","reported_seq":418,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.054439+0000","last_change":"2024-09-19T11:23:07.242151+0000","last_active":"2024-09-19T11:26:19.054439+0000","last_peered":"2024-09-19T11:26:19.054439+0000","last_clean":"2024-09-19T11:26:19.054439+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:26:19.054439+0000","last_undegraded":"2024-09-19T11:26:19.054439+0000","last_fullsized":"2024-09-19T11:26:19.054439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:09:15.211742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946255+0000","last_change":"2024-09-19T11:23:09.255928+0000","last_active":"2024-09-19T11:23:42.946255+0000","last_peered":"2024-09-19T11:23:42.946255+0000","last_clean":"2024-09-19T11:23:42.946255+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:23:42.946255+0000","last_undegraded":"2024-09-19T11:23:42.946255+0000","last_fullsized":"2024-09-19T11:23:42.946255+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:04.689951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'285","reported_seq":469,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.075845+0000","last_change":"2024-09-19T11:23:11.255933+0000","last_active":"2024-09-19T11:26:19.075845+0000","last_peered":"2024-09-19T11:26:19.075845+0000","last_clean":"2024-09-19T11:26:19.075845+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:26:19.075845+0000","last_undegraded":"2024-09-19T11:26:19.075845+0000","last_fullsized":"2024-09-19T11:26:19.075845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:01.160503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.141728+0000","last_change":"2024-09-19T11:23:13.266957+0000","last_active":"2024-09-19T11:25:11.141728+0000","last_peered":"2024-09-19T11:25:11.141728+0000","last_clean":"2024-09-19T11:25:11.141728+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:25:11.141728+0000","last_undegraded":"2024-09-19T11:25:11.141728+0000","last_fullsized":"2024-09-19T11:25:11.141728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:59.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":101,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338578+0000","last_change":"2024-09-19T11:23:05.255344+0000","last_active":"2024-09-19T11:26:12.338578+0000","last_peered":"2024-09-19T11:26:12.338578+0000","last_clean":"2024-09-19T11:26:12.338578+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:12.338578+0000","last_undegraded":"2024-09-19T11:26:12.338578+0000","last_fullsized":"2024-09-19T11:26:12.338578+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:22.119627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.225653+0000","last_change":"2024-09-19T11:26:29.225653+0000","last_active":"2024-09-19T11:26:29.225653+0000","last_peered":"2024-09-19T11:26:29.225653+0000","last_clean":"2024-09-19T11:26:29.225653+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:26:29.225653+0000","last_undegraded":"2024-09-19T11:26:29.225653+0000","last_fullsized":"2024-09-19T11:26:29.225653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.225549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:47.310948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":420,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.055152+0000","last_change":"2024-09-19T11:23:07.250416+0000","last_active":"2024-09-19T11:26:19.055152+0000","last_peered":"2024-09-19T11:26:19.055152+0000","last_clean":"2024-09-19T11:26:19.055152+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:26:19.055152+0000","last_undegraded":"2024-09-19T11:26:19.055152+0000","last_fullsized":"2024-09-19T11:26:19.055152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":586,"num_read_kb":586,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'41","reported_seq":92,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.208185+0000","last_change":"2024-09-19T11:23:09.247733+0000","last_active":"2024-09-19T11:26:19.208185+0000","last_peered":"2024-09-19T11:26:19.208185+0000","last_clean":"2024-09-19T11:26:19.208185+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:26:19.208185+0000","last_undegraded":"2024-09-19T11:26:19.208185+0000","last_fullsized":"2024-09-19T11:26:19.208185+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:38.921761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279060+0000","last_change":"2024-09-19T11:23:11.266008+0000","last_active":"2024-09-19T11:23:42.279060+0000","last_peered":"2024-09-19T11:23:42.279060+0000","last_clean":"2024-09-19T11:23:42.279060+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:23:42.279060+0000","last_undegraded":"2024-09-19T11:23:42.279060+0000","last_fullsized":"2024-09-19T11:23:42.279060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:38:43.871781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.426809+0000","last_change":"2024-09-19T11:23:13.263896+0000","last_active":"2024-09-19T11:26:12.426809+0000","last_peered":"2024-09-19T11:26:12.426809+0000","last_clean":"2024-09-19T11:26:12.426809+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:26:12.426809+0000","last_undegraded":"2024-09-19T11:26:12.426809+0000","last_fullsized":"2024-09-19T11:26:12.426809+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:35.364893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'7","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.273438+0000","last_change":"2024-09-19T11:23:05.254167+0000","last_active":"2024-09-19T11:26:12.273438+0000","last_peered":"2024-09-19T11:26:12.273438+0000","last_clean":"2024-09-19T11:26:12.273438+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:12.273438+0000","last_undegraded":"2024-09-19T11:26:12.273438+0000","last_fullsized":"2024-09-19T11:26:12.273438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:44.425539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"33'74","reported_seq":156,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.856409+0000","last_change":"2024-09-19T11:26:23.856409+0000","last_active":"2024-09-19T11:26:23.856409+0000","last_peered":"2024-09-19T11:26:23.856409+0000","last_clean":"2024-09-19T11:26:23.856409+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:26:23.856409+0000","last_undegraded":"2024-09-19T11:26:23.856409+0000","last_fullsized":"2024-09-19T11:26:23.856409+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'74","last_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_deep_scrub":"33'74","last_deep_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.856290+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:02.254545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'2","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.142962+0000","last_change":"2024-09-19T11:26:27.142962+0000","last_active":"2024-09-19T11:26:27.142962+0000","last_peered":"2024-09-19T11:26:27.142962+0000","last_clean":"2024-09-19T11:26:27.142962+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:26:27.142962+0000","last_undegraded":"2024-09-19T11:26:27.142962+0000","last_fullsized":"2024-09-19T11:26:27.142962+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.142873+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.914383+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.273990+0000","last_change":"2024-09-19T11:26:30.273990+0000","last_active":"2024-09-19T11:26:30.273990+0000","last_peered":"2024-09-19T11:26:30.273990+0000","last_clean":"2024-09-19T11:26:30.273990+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:26:30.273990+0000","last_undegraded":"2024-09-19T11:26:30.273990+0000","last_fullsized":"2024-09-19T11:26:30.273990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.273902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:45.300652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281740+0000","last_change":"2024-09-19T11:23:07.238893+0000","last_active":"2024-09-19T11:23:42.281740+0000","last_peered":"2024-09-19T11:23:42.281740+0000","last_clean":"2024-09-19T11:23:42.281740+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:23:42.281740+0000","last_undegraded":"2024-09-19T11:23:42.281740+0000","last_fullsized":"2024-09-19T11:23:42.281740+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:35.034936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'85","reported_seq":167,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.214097+0000","last_change":"2024-09-19T11:23:09.260843+0000","last_active":"2024-09-19T11:26:19.214097+0000","last_peered":"2024-09-19T11:26:19.214097+0000","last_clean":"2024-09-19T11:26:19.214097+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:26:19.214097+0000","last_undegraded":"2024-09-19T11:26:19.214097+0000","last_fullsized":"2024-09-19T11:26:19.214097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:17.846685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280169+0000","last_change":"2024-09-19T11:23:11.266314+0000","last_active":"2024-09-19T11:23:42.280169+0000","last_peered":"2024-09-19T11:23:42.280169+0000","last_clean":"2024-09-19T11:23:42.280169+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:23:42.280169+0000","last_undegraded":"2024-09-19T11:23:42.280169+0000","last_fullsized":"2024-09-19T11:23:42.280169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:28.832170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":163,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.377387+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:26:12.377387+0000","last_peered":"2024-09-19T11:26:12.377387+0000","last_clean":"2024-09-19T11:26:12.377387+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:26:12.377387+0000","last_undegraded":"2024-09-19T11:26:12.377387+0000","last_fullsized":"2024-09-19T11:26:12.377387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:51.720881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'5","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.810212+0000","last_change":"2024-09-19T11:23:05.233784+0000","last_active":"2024-09-19T11:25:16.810212+0000","last_peered":"2024-09-19T11:25:16.810212+0000","last_clean":"2024-09-19T11:25:16.810212+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:25:16.810212+0000","last_undegraded":"2024-09-19T11:25:16.810212+0000","last_fullsized":"2024-09-19T11:25:16.810212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:49.702956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.079800+0000","last_change":"2024-09-19T11:26:24.079800+0000","last_active":"2024-09-19T11:26:24.079800+0000","last_peered":"2024-09-19T11:26:24.079800+0000","last_clean":"2024-09-19T11:26:24.079800+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:26:24.079800+0000","last_undegraded":"2024-09-19T11:26:24.079800+0000","last_fullsized":"2024-09-19T11:26:24.079800+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.079699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:48:18.789176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.923493+0000","last_change":"2024-09-19T11:26:25.923493+0000","last_active":"2024-09-19T11:26:25.923493+0000","last_peered":"2024-09-19T11:26:25.923493+0000","last_clean":"2024-09-19T11:26:25.923493+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:26:25.923493+0000","last_undegraded":"2024-09-19T11:26:25.923493+0000","last_fullsized":"2024-09-19T11:26:25.923493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.923407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:56.735995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948430+0000","last_change":"2024-09-19T11:23:07.246124+0000","last_active":"2024-09-19T11:23:42.948430+0000","last_peered":"2024-09-19T11:23:42.948430+0000","last_clean":"2024-09-19T11:23:42.948430+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:23:42.948430+0000","last_undegraded":"2024-09-19T11:23:42.948430+0000","last_fullsized":"2024-09-19T11:23:42.948430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'284","reported_seq":501,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.212063+0000","last_change":"2024-09-19T11:23:09.257240+0000","last_active":"2024-09-19T11:26:19.212063+0000","last_peered":"2024-09-19T11:26:19.212063+0000","last_clean":"2024-09-19T11:26:19.212063+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:26:19.212063+0000","last_undegraded":"2024-09-19T11:26:19.212063+0000","last_fullsized":"2024-09-19T11:26:19.212063+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:14.514960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944824+0000","last_change":"2024-09-19T11:23:11.270807+0000","last_active":"2024-09-19T11:23:42.944824+0000","last_peered":"2024-09-19T11:23:42.944824+0000","last_clean":"2024-09-19T11:23:42.944824+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:23:42.944824+0000","last_undegraded":"2024-09-19T11:23:42.944824+0000","last_fullsized":"2024-09-19T11:23:42.944824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:23.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.346118+0000","last_change":"2024-09-19T11:23:13.269927+0000","last_active":"2024-09-19T11:24:27.346118+0000","last_peered":"2024-09-19T11:24:27.346118+0000","last_clean":"2024-09-19T11:24:27.346118+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:24:27.346118+0000","last_undegraded":"2024-09-19T11:24:27.346118+0000","last_fullsized":"2024-09-19T11:24:27.346118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:59.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.967834+0000","last_change":"2024-09-19T11:26:39.967834+0000","last_active":"2024-09-19T11:26:39.967834+0000","last_peered":"2024-09-19T11:26:39.967834+0000","last_clean":"2024-09-19T11:26:39.967834+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:39.967834+0000","last_undegraded":"2024-09-19T11:26:39.967834+0000","last_fullsized":"2024-09-19T11:26:39.967834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.967748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:48.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.952524+0000","last_change":"2024-09-19T11:26:23.952524+0000","last_active":"2024-09-19T11:26:23.952524+0000","last_peered":"2024-09-19T11:26:23.952524+0000","last_clean":"2024-09-19T11:26:23.952524+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:26:23.952524+0000","last_undegraded":"2024-09-19T11:26:23.952524+0000","last_fullsized":"2024-09-19T11:26:23.952524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.952436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:33.569870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.922822+0000","last_change":"2024-09-19T11:26:24.922822+0000","last_active":"2024-09-19T11:26:24.922822+0000","last_peered":"2024-09-19T11:26:24.922822+0000","last_clean":"2024-09-19T11:26:24.922822+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:26:24.922822+0000","last_undegraded":"2024-09-19T11:26:24.922822+0000","last_fullsized":"2024-09-19T11:26:24.922822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.922736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:47.473806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280552+0000","last_change":"2024-09-19T11:23:07.245296+0000","last_active":"2024-09-19T11:23:42.280552+0000","last_peered":"2024-09-19T11:23:42.280552+0000","last_clean":"2024-09-19T11:23:42.280552+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:23:42.280552+0000","last_undegraded":"2024-09-19T11:23:42.280552+0000","last_fullsized":"2024-09-19T11:23:42.280552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:23:05.997935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'16","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327294+0000","last_change":"2024-09-19T11:23:09.249984+0000","last_active":"2024-09-19T11:26:12.327294+0000","last_peered":"2024-09-19T11:26:12.327294+0000","last_clean":"2024-09-19T11:26:12.327294+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:12.327294+0000","last_undegraded":"2024-09-19T11:26:12.327294+0000","last_fullsized":"2024-09-19T11:26:12.327294+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:47.233637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280391+0000","last_change":"2024-09-19T11:23:11.265633+0000","last_active":"2024-09-19T11:23:42.280391+0000","last_peered":"2024-09-19T11:23:42.280391+0000","last_clean":"2024-09-19T11:23:42.280391+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:23:42.280391+0000","last_undegraded":"2024-09-19T11:23:42.280391+0000","last_fullsized":"2024-09-19T11:23:42.280391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:16.181121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943722+0000","last_change":"2024-09-19T11:23:13.273373+0000","last_active":"2024-09-19T11:23:42.943722+0000","last_peered":"2024-09-19T11:23:42.943722+0000","last_clean":"2024-09-19T11:23:42.943722+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:23:42.943722+0000","last_undegraded":"2024-09-19T11:23:42.943722+0000","last_fullsized":"2024-09-19T11:23:42.943722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:11:13.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.977150+0000","last_change":"2024-09-19T11:26:37.977150+0000","last_active":"2024-09-19T11:26:37.977150+0000","last_peered":"2024-09-19T11:26:37.977150+0000","last_clean":"2024-09-19T11:26:37.977150+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:37.977150+0000","last_undegraded":"2024-09-19T11:26:37.977150+0000","last_fullsized":"2024-09-19T11:26:37.977150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.977048+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:20.097270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"33'3","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.109916+0000","last_change":"2024-09-19T11:26:25.109916+0000","last_active":"2024-09-19T11:26:25.109916+0000","last_peered":"2024-09-19T11:26:25.109916+0000","last_clean":"2024-09-19T11:26:25.109916+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:26:25.109916+0000","last_undegraded":"2024-09-19T11:26:25.109916+0000","last_fullsized":"2024-09-19T11:26:25.109916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.109824+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:48.020763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.938045+0000","last_change":"2024-09-19T11:26:25.938045+0000","last_active":"2024-09-19T11:26:25.938045+0000","last_peered":"2024-09-19T11:26:25.938045+0000","last_clean":"2024-09-19T11:26:25.938045+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:25.938045+0000","last_undegraded":"2024-09-19T11:26:25.938045+0000","last_fullsized":"2024-09-19T11:26:25.938045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.937963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:58:57.848583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279722+0000","last_change":"2024-09-19T11:23:07.243946+0000","last_active":"2024-09-19T11:23:42.279722+0000","last_peered":"2024-09-19T11:23:42.279722+0000","last_clean":"2024-09-19T11:23:42.279722+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:23:42.279722+0000","last_undegraded":"2024-09-19T11:23:42.279722+0000","last_fullsized":"2024-09-19T11:23:42.279722+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:02.383974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'13","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.530475+0000","last_change":"2024-09-19T11:23:09.254499+0000","last_active":"2024-09-19T11:26:11.530475+0000","last_peered":"2024-09-19T11:26:11.530475+0000","last_clean":"2024-09-19T11:26:11.530475+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:26:11.530475+0000","last_undegraded":"2024-09-19T11:26:11.530475+0000","last_fullsized":"2024-09-19T11:26:11.530475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:18.957107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947826+0000","last_change":"2024-09-19T11:23:11.271073+0000","last_active":"2024-09-19T11:23:42.947826+0000","last_peered":"2024-09-19T11:23:42.947826+0000","last_clean":"2024-09-19T11:23:42.947826+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:23:42.947826+0000","last_undegraded":"2024-09-19T11:23:42.947826+0000","last_fullsized":"2024-09-19T11:23:42.947826+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:52.079937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.088603+0000","last_change":"2024-09-19T11:23:13.273910+0000","last_active":"2024-09-19T11:25:11.088603+0000","last_peered":"2024-09-19T11:25:11.088603+0000","last_clean":"2024-09-19T11:25:11.088603+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:25:11.088603+0000","last_undegraded":"2024-09-19T11:25:11.088603+0000","last_fullsized":"2024-09-19T11:25:11.088603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:02.581618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.916064+0000","last_change":"2024-09-19T11:26:35.916064+0000","last_active":"2024-09-19T11:26:35.916064+0000","last_peered":"2024-09-19T11:26:35.916064+0000","last_clean":"2024-09-19T11:26:35.916064+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:35.916064+0000","last_undegraded":"2024-09-19T11:26:35.916064+0000","last_fullsized":"2024-09-19T11:26:35.916064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.915972+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:33.729135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.933741+0000","last_change":"2024-09-19T11:26:24.933741+0000","last_active":"2024-09-19T11:26:24.933741+0000","last_peered":"2024-09-19T11:26:24.933741+0000","last_clean":"2024-09-19T11:26:24.933741+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:26:24.933741+0000","last_undegraded":"2024-09-19T11:26:24.933741+0000","last_fullsized":"2024-09-19T11:26:24.933741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.933645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.620054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.910571+0000","last_change":"2024-09-19T11:26:22.910571+0000","last_active":"2024-09-19T11:26:22.910571+0000","last_peered":"2024-09-19T11:26:22.910571+0000","last_clean":"2024-09-19T11:26:22.910571+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:26:22.910571+0000","last_undegraded":"2024-09-19T11:26:22.910571+0000","last_fullsized":"2024-09-19T11:26:22.910571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.910497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:20.068073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'1","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.641108+0000","last_change":"2024-09-19T11:23:07.247913+0000","last_active":"2024-09-19T11:26:12.641108+0000","last_peered":"2024-09-19T11:26:12.641108+0000","last_clean":"2024-09-19T11:26:12.641108+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:26:12.641108+0000","last_undegraded":"2024-09-19T11:26:12.641108+0000","last_fullsized":"2024-09-19T11:26:12.641108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:03.208183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'68","reported_seq":136,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.192730+0000","last_change":"2024-09-19T11:23:09.257853+0000","last_active":"2024-09-19T11:26:19.192730+0000","last_peered":"2024-09-19T11:26:19.192730+0000","last_clean":"2024-09-19T11:26:19.192730+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:26:19.192730+0000","last_undegraded":"2024-09-19T11:26:19.192730+0000","last_fullsized":"2024-09-19T11:26:19.192730+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:02.366161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277438+0000","last_change":"2024-09-19T11:23:11.268600+0000","last_active":"2024-09-19T11:23:42.277438+0000","last_peered":"2024-09-19T11:23:42.277438+0000","last_clean":"2024-09-19T11:23:42.277438+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:23:42.277438+0000","last_undegraded":"2024-09-19T11:23:42.277438+0000","last_fullsized":"2024-09-19T11:23:42.277438+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:50.540232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.422953+0000","last_change":"2024-09-19T11:23:13.267518+0000","last_active":"2024-09-19T11:25:11.422953+0000","last_peered":"2024-09-19T11:25:11.422953+0000","last_clean":"2024-09-19T11:25:11.422953+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:25:11.422953+0000","last_undegraded":"2024-09-19T11:25:11.422953+0000","last_fullsized":"2024-09-19T11:25:11.422953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:35.030653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.952752+0000","last_change":"2024-09-19T11:26:36.952752+0000","last_active":"2024-09-19T11:26:36.952752+0000","last_peered":"2024-09-19T11:26:36.952752+0000","last_clean":"2024-09-19T11:26:36.952752+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:36.952752+0000","last_undegraded":"2024-09-19T11:26:36.952752+0000","last_fullsized":"2024-09-19T11:26:36.952752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.952660+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.496448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.969511+0000","last_change":"2024-09-19T11:26:38.969511+0000","last_active":"2024-09-19T11:26:38.969511+0000","last_peered":"2024-09-19T11:26:38.969511+0000","last_clean":"2024-09-19T11:26:38.969511+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:26:38.969511+0000","last_undegraded":"2024-09-19T11:26:38.969511+0000","last_fullsized":"2024-09-19T11:26:38.969511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.969423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:49.370950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.293468+0000","last_change":"2024-09-19T11:26:32.293468+0000","last_active":"2024-09-19T11:26:32.293468+0000","last_peered":"2024-09-19T11:26:32.293468+0000","last_clean":"2024-09-19T11:26:32.293468+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:26:32.293468+0000","last_undegraded":"2024-09-19T11:26:32.293468+0000","last_fullsized":"2024-09-19T11:26:32.293468+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.293378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:48.996006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943947+0000","last_change":"2024-09-19T11:23:07.243184+0000","last_active":"2024-09-19T11:23:42.943947+0000","last_peered":"2024-09-19T11:23:42.943947+0000","last_clean":"2024-09-19T11:23:42.943947+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:23:42.943947+0000","last_undegraded":"2024-09-19T11:23:42.943947+0000","last_fullsized":"2024-09-19T11:23:42.943947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:32:59.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'15","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809866+0000","last_change":"2024-09-19T11:23:09.261039+0000","last_active":"2024-09-19T11:25:16.809866+0000","last_peered":"2024-09-19T11:25:16.809866+0000","last_clean":"2024-09-19T11:25:16.809866+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:25:16.809866+0000","last_undegraded":"2024-09-19T11:25:16.809866+0000","last_fullsized":"2024-09-19T11:25:16.809866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:42.183684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'285","reported_seq":359,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.071730+0000","last_change":"2024-09-19T11:23:11.263125+0000","last_active":"2024-09-19T11:26:19.071730+0000","last_peered":"2024-09-19T11:26:19.071730+0000","last_clean":"2024-09-19T11:26:19.071730+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:26:19.071730+0000","last_undegraded":"2024-09-19T11:26:19.071730+0000","last_fullsized":"2024-09-19T11:26:19.071730+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:10.328770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946621+0000","last_change":"2024-09-19T11:23:13.269759+0000","last_active":"2024-09-19T11:23:42.946621+0000","last_peered":"2024-09-19T11:23:42.946621+0000","last_clean":"2024-09-19T11:23:42.946621+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:23:42.946621+0000","last_undegraded":"2024-09-19T11:23:42.946621+0000","last_fullsized":"2024-09-19T11:23:42.946621+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:46.820239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'11","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.025445+0000","last_change":"2024-09-19T11:23:05.254850+0000","last_active":"2024-09-19T11:26:12.025445+0000","last_peered":"2024-09-19T11:26:12.025445+0000","last_clean":"2024-09-19T11:26:12.025445+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:12.025445+0000","last_undegraded":"2024-09-19T11:26:12.025445+0000","last_fullsized":"2024-09-19T11:26:12.025445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:44.121192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.185387+0000","last_change":"2024-09-19T11:26:28.185387+0000","last_active":"2024-09-19T11:26:28.185387+0000","last_peered":"2024-09-19T11:26:28.185387+0000","last_clean":"2024-09-19T11:26:28.185387+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:26:28.185387+0000","last_undegraded":"2024-09-19T11:26:28.185387+0000","last_fullsized":"2024-09-19T11:26:28.185387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.185291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:48.217740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.897455+0000","last_change":"2024-09-19T11:26:23.897455+0000","last_active":"2024-09-19T11:26:23.897455+0000","last_peered":"2024-09-19T11:26:23.897455+0000","last_clean":"2024-09-19T11:26:23.897455+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:26:23.897455+0000","last_undegraded":"2024-09-19T11:26:23.897455+0000","last_fullsized":"2024-09-19T11:26:23.897455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.897373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:13.527667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946584+0000","last_change":"2024-09-19T11:23:07.245138+0000","last_active":"2024-09-19T11:23:42.946584+0000","last_peered":"2024-09-19T11:23:42.946584+0000","last_clean":"2024-09-19T11:23:42.946584+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:23:42.946584+0000","last_undegraded":"2024-09-19T11:23:42.946584+0000","last_fullsized":"2024-09-19T11:23:42.946584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:31.680535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'11","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945354+0000","last_change":"2024-09-19T11:23:09.256698+0000","last_active":"2024-09-19T11:23:42.945354+0000","last_peered":"2024-09-19T11:23:42.945354+0000","last_clean":"2024-09-19T11:23:42.945354+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:23:42.945354+0000","last_undegraded":"2024-09-19T11:23:42.945354+0000","last_fullsized":"2024-09-19T11:23:42.945354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:21.852554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'285","reported_seq":433,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.076916+0000","last_change":"2024-09-19T11:23:11.258279+0000","last_active":"2024-09-19T11:26:19.076916+0000","last_peered":"2024-09-19T11:26:19.076916+0000","last_clean":"2024-09-19T11:26:19.076916+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:26:19.076916+0000","last_undegraded":"2024-09-19T11:26:19.076916+0000","last_fullsized":"2024-09-19T11:26:19.076916+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:24.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281484+0000","last_change":"2024-09-19T11:23:13.274470+0000","last_active":"2024-09-19T11:23:42.281484+0000","last_peered":"2024-09-19T11:23:42.281484+0000","last_clean":"2024-09-19T11:23:42.281484+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:23:42.281484+0000","last_undegraded":"2024-09-19T11:23:42.281484+0000","last_fullsized":"2024-09-19T11:23:42.281484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:33.500152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550363+0000","last_change":"2024-09-19T11:23:05.250199+0000","last_active":"2024-09-19T11:24:27.550363+0000","last_peered":"2024-09-19T11:24:27.550363+0000","last_clean":"2024-09-19T11:24:27.550363+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:24:27.550363+0000","last_undegraded":"2024-09-19T11:24:27.550363+0000","last_fullsized":"2024-09-19T11:24:27.550363+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:20.188966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.886518+0000","last_change":"2024-09-19T11:26:22.886518+0000","last_active":"2024-09-19T11:26:22.886518+0000","last_peered":"2024-09-19T11:26:22.886518+0000","last_clean":"2024-09-19T11:26:22.886518+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:26:22.886518+0000","last_undegraded":"2024-09-19T11:26:22.886518+0000","last_fullsized":"2024-09-19T11:26:22.886518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.886419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:07.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.146468+0000","last_change":"2024-09-19T11:26:26.146468+0000","last_active":"2024-09-19T11:26:26.146468+0000","last_peered":"2024-09-19T11:26:26.146468+0000","last_clean":"2024-09-19T11:26:26.146468+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:26:26.146468+0000","last_undegraded":"2024-09-19T11:26:26.146468+0000","last_fullsized":"2024-09-19T11:26:26.146468+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.146376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:37.768766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945462+0000","last_change":"2024-09-19T11:23:07.248324+0000","last_active":"2024-09-19T11:23:42.945462+0000","last_peered":"2024-09-19T11:23:42.945462+0000","last_clean":"2024-09-19T11:23:42.945462+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:23:42.945462+0000","last_undegraded":"2024-09-19T11:23:42.945462+0000","last_fullsized":"2024-09-19T11:23:42.945462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.767188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'91","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.201893+0000","last_change":"2024-09-19T11:23:09.258292+0000","last_active":"2024-09-19T11:26:19.201893+0000","last_peered":"2024-09-19T11:26:19.201893+0000","last_clean":"2024-09-19T11:26:19.201893+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:26:19.201893+0000","last_undegraded":"2024-09-19T11:26:19.201893+0000","last_fullsized":"2024-09-19T11:26:19.201893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:51.017060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280109+0000","last_change":"2024-09-19T11:23:11.263614+0000","last_active":"2024-09-19T11:23:42.280109+0000","last_peered":"2024-09-19T11:23:42.280109+0000","last_clean":"2024-09-19T11:23:42.280109+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:23:42.280109+0000","last_undegraded":"2024-09-19T11:23:42.280109+0000","last_fullsized":"2024-09-19T11:23:42.280109+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.842570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278855+0000","last_change":"2024-09-19T11:23:13.265949+0000","last_active":"2024-09-19T11:23:42.278855+0000","last_peered":"2024-09-19T11:23:42.278855+0000","last_clean":"2024-09-19T11:23:42.278855+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:23:42.278855+0000","last_undegraded":"2024-09-19T11:23:42.278855+0000","last_fullsized":"2024-09-19T11:23:42.278855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:37.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278953+0000","last_change":"2024-09-19T11:23:13.271066+0000","last_active":"2024-09-19T11:23:42.278953+0000","last_peered":"2024-09-19T11:23:42.278953+0000","last_clean":"2024-09-19T11:23:42.278953+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:23:42.278953+0000","last_undegraded":"2024-09-19T11:23:42.278953+0000","last_fullsized":"2024-09-19T11:23:42.278953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:34.307629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277914+0000","last_change":"2024-09-19T11:23:05.245620+0000","last_active":"2024-09-19T11:23:42.277914+0000","last_peered":"2024-09-19T11:23:42.277914+0000","last_clean":"2024-09-19T11:23:42.277914+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:23:42.277914+0000","last_undegraded":"2024-09-19T11:23:42.277914+0000","last_fullsized":"2024-09-19T11:23:42.277914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.396108+0000","last_change":"2024-09-19T11:26:38.396108+0000","last_active":"2024-09-19T11:26:38.396108+0000","last_peered":"2024-09-19T11:26:38.396108+0000","last_clean":"2024-09-19T11:26:38.396108+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:26:38.396108+0000","last_undegraded":"2024-09-19T11:26:38.396108+0000","last_fullsized":"2024-09-19T11:26:38.396108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.396006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:14.129195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282381+0000","last_change":"2024-09-19T11:23:07.240696+0000","last_active":"2024-09-19T11:23:42.282381+0000","last_peered":"2024-09-19T11:23:42.282381+0000","last_clean":"2024-09-19T11:23:42.282381+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:23:42.282381+0000","last_undegraded":"2024-09-19T11:23:42.282381+0000","last_fullsized":"2024-09-19T11:23:42.282381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:02.659138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'35","reported_seq":83,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.220186+0000","last_change":"2024-09-19T11:23:09.259148+0000","last_active":"2024-09-19T11:26:19.220186+0000","last_peered":"2024-09-19T11:26:19.220186+0000","last_clean":"2024-09-19T11:26:19.220186+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:26:19.220186+0000","last_undegraded":"2024-09-19T11:26:19.220186+0000","last_fullsized":"2024-09-19T11:26:19.220186+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:29.000573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945264+0000","last_change":"2024-09-19T11:23:11.271128+0000","last_active":"2024-09-19T11:23:42.945264+0000","last_peered":"2024-09-19T11:23:42.945264+0000","last_clean":"2024-09-19T11:23:42.945264+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:23:42.945264+0000","last_undegraded":"2024-09-19T11:23:42.945264+0000","last_fullsized":"2024-09-19T11:23:42.945264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:35.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.427118+0000","last_change":"2024-09-19T11:23:13.273576+0000","last_active":"2024-09-19T11:25:09.427118+0000","last_peered":"2024-09-19T11:25:09.427118+0000","last_clean":"2024-09-19T11:25:09.427118+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:25:09.427118+0000","last_undegraded":"2024-09-19T11:25:09.427118+0000","last_fullsized":"2024-09-19T11:25:09.427118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:43.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.036281+0000","last_change":"2024-09-19T11:26:41.036281+0000","last_active":"2024-09-19T11:26:41.036281+0000","last_peered":"2024-09-19T11:26:41.036281+0000","last_clean":"2024-09-19T11:26:41.036281+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:26:41.036281+0000","last_undegraded":"2024-09-19T11:26:41.036281+0000","last_fullsized":"2024-09-19T11:26:41.036281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.036169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:34.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.921998+0000","last_change":"2024-09-19T11:26:28.921998+0000","last_active":"2024-09-19T11:26:28.921998+0000","last_peered":"2024-09-19T11:26:28.921998+0000","last_clean":"2024-09-19T11:26:28.921998+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:26:28.921998+0000","last_undegraded":"2024-09-19T11:26:28.921998+0000","last_fullsized":"2024-09-19T11:26:28.921998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.921906+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:00.637589+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945177+0000","last_change":"2024-09-19T11:23:07.248908+0000","last_active":"2024-09-19T11:23:42.945177+0000","last_peered":"2024-09-19T11:23:42.945177+0000","last_clean":"2024-09-19T11:23:42.945177+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:23:42.945177+0000","last_undegraded":"2024-09-19T11:23:42.945177+0000","last_fullsized":"2024-09-19T11:23:42.945177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:57.835222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'5","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.535761+0000","last_change":"2024-09-19T11:23:09.244959+0000","last_active":"2024-09-19T11:26:11.535761+0000","last_peered":"2024-09-19T11:26:11.535761+0000","last_clean":"2024-09-19T11:26:11.535761+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:26:11.535761+0000","last_undegraded":"2024-09-19T11:26:11.535761+0000","last_fullsized":"2024-09-19T11:26:11.535761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:13.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277093+0000","last_change":"2024-09-19T11:23:11.268038+0000","last_active":"2024-09-19T11:23:42.277093+0000","last_peered":"2024-09-19T11:23:42.277093+0000","last_clean":"2024-09-19T11:23:42.277093+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:23:42.277093+0000","last_undegraded":"2024-09-19T11:23:42.277093+0000","last_fullsized":"2024-09-19T11:23:42.277093+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:40.232592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279462+0000","last_change":"2024-09-19T11:23:13.267238+0000","last_active":"2024-09-19T11:23:42.279462+0000","last_peered":"2024-09-19T11:23:42.279462+0000","last_clean":"2024-09-19T11:23:42.279462+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:23:42.279462+0000","last_undegraded":"2024-09-19T11:23:42.279462+0000","last_fullsized":"2024-09-19T11:23:42.279462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:36.252981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123419+0000","last_change":"2024-09-19T11:23:05.240162+0000","last_active":"2024-09-19T11:26:12.123419+0000","last_peered":"2024-09-19T11:26:12.123419+0000","last_clean":"2024-09-19T11:26:12.123419+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:12.123419+0000","last_undegraded":"2024-09-19T11:26:12.123419+0000","last_fullsized":"2024-09-19T11:26:12.123419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:45.738004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.937976+0000","last_change":"2024-09-19T11:26:29.937976+0000","last_active":"2024-09-19T11:26:29.937976+0000","last_peered":"2024-09-19T11:26:29.937976+0000","last_clean":"2024-09-19T11:26:29.937976+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:29.937976+0000","last_undegraded":"2024-09-19T11:26:29.937976+0000","last_fullsized":"2024-09-19T11:26:29.937976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.937882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:14.058304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277308+0000","last_change":"2024-09-19T11:23:07.243467+0000","last_active":"2024-09-19T11:23:42.277308+0000","last_peered":"2024-09-19T11:23:42.277308+0000","last_clean":"2024-09-19T11:23:42.277308+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:23:42.277308+0000","last_undegraded":"2024-09-19T11:23:42.277308+0000","last_fullsized":"2024-09-19T11:23:42.277308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:48.792369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'43","reported_seq":107,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.186175+0000","last_change":"2024-09-19T11:23:09.248715+0000","last_active":"2024-09-19T11:26:19.186175+0000","last_peered":"2024-09-19T11:26:19.186175+0000","last_clean":"2024-09-19T11:26:19.186175+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:26:19.186175+0000","last_undegraded":"2024-09-19T11:26:19.186175+0000","last_fullsized":"2024-09-19T11:26:19.186175+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:51.723342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280357+0000","last_change":"2024-09-19T11:23:11.262690+0000","last_active":"2024-09-19T11:23:42.280357+0000","last_peered":"2024-09-19T11:23:42.280357+0000","last_clean":"2024-09-19T11:23:42.280357+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:23:42.280357+0000","last_undegraded":"2024-09-19T11:23:42.280357+0000","last_fullsized":"2024-09-19T11:23:42.280357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:55.778119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.511055+0000","last_change":"2024-09-19T11:23:05.253736+0000","last_active":"2024-09-19T11:25:11.511055+0000","last_peered":"2024-09-19T11:25:11.511055+0000","last_clean":"2024-09-19T11:25:11.511055+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:25:11.511055+0000","last_undegraded":"2024-09-19T11:25:11.511055+0000","last_fullsized":"2024-09-19T11:25:11.511055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:09.050323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.886507+0000","last_change":"2024-09-19T11:26:25.886507+0000","last_active":"2024-09-19T11:26:25.886507+0000","last_peered":"2024-09-19T11:26:25.886507+0000","last_clean":"2024-09-19T11:26:25.886507+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:26:25.886507+0000","last_undegraded":"2024-09-19T11:26:25.886507+0000","last_fullsized":"2024-09-19T11:26:25.886507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.886414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:38.573945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944123+0000","last_change":"2024-09-19T11:23:07.245862+0000","last_active":"2024-09-19T11:23:42.944123+0000","last_peered":"2024-09-19T11:23:42.944123+0000","last_clean":"2024-09-19T11:23:42.944123+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:23:42.944123+0000","last_undegraded":"2024-09-19T11:23:42.944123+0000","last_fullsized":"2024-09-19T11:23:42.944123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:09.692396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.177030+0000","last_change":"2024-09-19T11:23:09.247041+0000","last_active":"2024-09-19T11:26:19.177030+0000","last_peered":"2024-09-19T11:26:19.177030+0000","last_clean":"2024-09-19T11:26:19.177030+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:26:19.177030+0000","last_undegraded":"2024-09-19T11:26:19.177030+0000","last_fullsized":"2024-09-19T11:26:19.177030+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:21.727511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277107+0000","last_change":"2024-09-19T11:23:11.253755+0000","last_active":"2024-09-19T11:23:42.277107+0000","last_peered":"2024-09-19T11:23:42.277107+0000","last_clean":"2024-09-19T11:23:42.277107+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:23:42.277107+0000","last_undegraded":"2024-09-19T11:23:42.277107+0000","last_fullsized":"2024-09-19T11:23:42.277107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:51.209331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510416+0000","last_change":"2024-09-19T11:23:05.236615+0000","last_active":"2024-09-19T11:25:11.510416+0000","last_peered":"2024-09-19T11:25:11.510416+0000","last_clean":"2024-09-19T11:25:11.510416+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:25:11.510416+0000","last_undegraded":"2024-09-19T11:25:11.510416+0000","last_fullsized":"2024-09-19T11:25:11.510416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:36.749643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'4","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.386729+0000","last_change":"2024-09-19T11:26:39.386729+0000","last_active":"2024-09-19T11:26:39.386729+0000","last_peered":"2024-09-19T11:26:39.386729+0000","last_clean":"2024-09-19T11:26:39.386729+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:26:39.386729+0000","last_undegraded":"2024-09-19T11:26:39.386729+0000","last_fullsized":"2024-09-19T11:26:39.386729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.386636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:26.742196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.796174+0000","last_change":"2024-09-19T11:23:42.945035+0000","last_active":"2024-09-19T11:23:44.796174+0000","last_peered":"2024-09-19T11:23:44.796174+0000","last_clean":"2024-09-19T11:23:44.796174+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:23:44.796174+0000","last_undegraded":"2024-09-19T11:23:44.796174+0000","last_fullsized":"2024-09-19T11:23:44.796174+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'62","reported_seq":127,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.174069+0000","last_change":"2024-09-19T11:23:09.246587+0000","last_active":"2024-09-19T11:26:19.174069+0000","last_peered":"2024-09-19T11:26:19.174069+0000","last_clean":"2024-09-19T11:26:19.174069+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:26:19.174069+0000","last_undegraded":"2024-09-19T11:26:19.174069+0000","last_fullsized":"2024-09-19T11:26:19.174069+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:53.040162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":65,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278167+0000","last_change":"2024-09-19T11:23:11.260449+0000","last_active":"2024-09-19T11:23:42.278167+0000","last_peered":"2024-09-19T11:23:42.278167+0000","last_clean":"2024-09-19T11:23:42.278167+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:23:42.278167+0000","last_undegraded":"2024-09-19T11:23:42.278167+0000","last_fullsized":"2024-09-19T11:23:42.278167+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:30.983027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.419187+0000","last_change":"2024-09-19T11:23:13.268105+0000","last_active":"2024-09-19T11:25:11.419187+0000","last_peered":"2024-09-19T11:25:11.419187+0000","last_clean":"2024-09-19T11:25:11.419187+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:25:11.419187+0000","last_undegraded":"2024-09-19T11:25:11.419187+0000","last_fullsized":"2024-09-19T11:25:11.419187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:21.113571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368061+0000","last_change":"2024-09-19T11:23:05.246352+0000","last_active":"2024-09-19T11:26:12.368061+0000","last_peered":"2024-09-19T11:26:12.368061+0000","last_clean":"2024-09-19T11:26:12.368061+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:12.368061+0000","last_undegraded":"2024-09-19T11:26:12.368061+0000","last_fullsized":"2024-09-19T11:26:12.368061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:53:39.878491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.929509+0000","last_change":"2024-09-19T11:26:26.929509+0000","last_active":"2024-09-19T11:26:26.929509+0000","last_peered":"2024-09-19T11:26:26.929509+0000","last_clean":"2024-09-19T11:26:26.929509+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:26:26.929509+0000","last_undegraded":"2024-09-19T11:26:26.929509+0000","last_fullsized":"2024-09-19T11:26:26.929509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.929422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:53.997533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279359+0000","last_change":"2024-09-19T11:23:07.243165+0000","last_active":"2024-09-19T11:23:42.279359+0000","last_peered":"2024-09-19T11:23:42.279359+0000","last_clean":"2024-09-19T11:23:42.279359+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:23:42.279359+0000","last_undegraded":"2024-09-19T11:23:42.279359+0000","last_fullsized":"2024-09-19T11:23:42.279359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:21.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.179930+0000","last_change":"2024-09-19T11:23:09.256642+0000","last_active":"2024-09-19T11:26:19.179930+0000","last_peered":"2024-09-19T11:26:19.179930+0000","last_clean":"2024-09-19T11:26:19.179930+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:26:19.179930+0000","last_undegraded":"2024-09-19T11:26:19.179930+0000","last_fullsized":"2024-09-19T11:26:19.179930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:42.797325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282300+0000","last_change":"2024-09-19T11:23:11.266817+0000","last_active":"2024-09-19T11:23:42.282300+0000","last_peered":"2024-09-19T11:23:42.282300+0000","last_clean":"2024-09-19T11:23:42.282300+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:23:42.282300+0000","last_undegraded":"2024-09-19T11:23:42.282300+0000","last_fullsized":"2024-09-19T11:23:42.282300+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:41:28.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947653+0000","last_change":"2024-09-19T11:23:05.254589+0000","last_active":"2024-09-19T11:23:42.947653+0000","last_peered":"2024-09-19T11:23:42.947653+0000","last_clean":"2024-09-19T11:23:42.947653+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:23:42.947653+0000","last_undegraded":"2024-09-19T11:23:42.947653+0000","last_fullsized":"2024-09-19T11:23:42.947653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:11.078441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.877728+0000","last_change":"2024-09-19T11:26:28.877728+0000","last_active":"2024-09-19T11:26:28.877728+0000","last_peered":"2024-09-19T11:26:28.877728+0000","last_clean":"2024-09-19T11:26:28.877728+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:26:28.877728+0000","last_undegraded":"2024-09-19T11:26:28.877728+0000","last_fullsized":"2024-09-19T11:26:28.877728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.877640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:09.086406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278448+0000","last_change":"2024-09-19T11:23:07.242009+0000","last_active":"2024-09-19T11:23:42.278448+0000","last_peered":"2024-09-19T11:23:42.278448+0000","last_clean":"2024-09-19T11:23:42.278448+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:23:42.278448+0000","last_undegraded":"2024-09-19T11:23:42.278448+0000","last_fullsized":"2024-09-19T11:23:42.278448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:37.498928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'37","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.126834+0000","last_change":"2024-09-19T11:23:09.258190+0000","last_active":"2024-09-19T11:26:19.126834+0000","last_peered":"2024-09-19T11:26:19.126834+0000","last_clean":"2024-09-19T11:26:19.126834+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:26:19.126834+0000","last_undegraded":"2024-09-19T11:26:19.126834+0000","last_fullsized":"2024-09-19T11:26:19.126834+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:31.989630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280329+0000","last_change":"2024-09-19T11:23:11.261698+0000","last_active":"2024-09-19T11:23:42.280329+0000","last_peered":"2024-09-19T11:23:42.280329+0000","last_clean":"2024-09-19T11:23:42.280329+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:23:42.280329+0000","last_undegraded":"2024-09-19T11:23:42.280329+0000","last_fullsized":"2024-09-19T11:23:42.280329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:48.537146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032904+0000","last_change":"2024-09-19T11:23:05.238234+0000","last_active":"2024-09-19T11:25:08.032904+0000","last_peered":"2024-09-19T11:25:08.032904+0000","last_clean":"2024-09-19T11:25:08.032904+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:25:08.032904+0000","last_undegraded":"2024-09-19T11:25:08.032904+0000","last_fullsized":"2024-09-19T11:25:08.032904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:03:26.967357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.342895+0000","last_change":"2024-09-19T11:26:40.342895+0000","last_active":"2024-09-19T11:26:40.342895+0000","last_peered":"2024-09-19T11:26:40.342895+0000","last_clean":"2024-09-19T11:26:40.342895+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:26:40.342895+0000","last_undegraded":"2024-09-19T11:26:40.342895+0000","last_fullsized":"2024-09-19T11:26:40.342895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.342808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.126397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.286788+0000","last_change":"2024-09-19T11:23:42.286200+0000","last_active":"2024-09-19T11:23:42.286788+0000","last_peered":"2024-09-19T11:23:42.286788+0000","last_clean":"2024-09-19T11:23:42.286788+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:23:42.286788+0000","last_undegraded":"2024-09-19T11:23:42.286788+0000","last_fullsized":"2024-09-19T11:23:42.286788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.397616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946537+0000","last_change":"2024-09-19T11:23:09.257194+0000","last_active":"2024-09-19T11:23:42.946537+0000","last_peered":"2024-09-19T11:23:42.946537+0000","last_clean":"2024-09-19T11:23:42.946537+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:23:42.946537+0000","last_undegraded":"2024-09-19T11:23:42.946537+0000","last_fullsized":"2024-09-19T11:23:42.946537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:10.690183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'285","reported_seq":421,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.070038+0000","last_change":"2024-09-19T11:23:11.268055+0000","last_active":"2024-09-19T11:26:19.070038+0000","last_peered":"2024-09-19T11:26:19.070038+0000","last_clean":"2024-09-19T11:26:19.070038+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:26:19.070038+0000","last_undegraded":"2024-09-19T11:26:19.070038+0000","last_fullsized":"2024-09-19T11:26:19.070038+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:33.997140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943313+0000","last_change":"2024-09-19T11:23:13.272264+0000","last_active":"2024-09-19T11:23:42.943313+0000","last_peered":"2024-09-19T11:23:42.943313+0000","last_clean":"2024-09-19T11:23:42.943313+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:23:42.943313+0000","last_undegraded":"2024-09-19T11:23:42.943313+0000","last_fullsized":"2024-09-19T11:23:42.943313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:34.821219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123209+0000","last_change":"2024-09-19T11:23:05.235764+0000","last_active":"2024-09-19T11:26:12.123209+0000","last_peered":"2024-09-19T11:26:12.123209+0000","last_clean":"2024-09-19T11:26:12.123209+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:12.123209+0000","last_undegraded":"2024-09-19T11:26:12.123209+0000","last_fullsized":"2024-09-19T11:26:12.123209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:41.326878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.907839+0000","last_change":"2024-09-19T11:26:27.907839+0000","last_active":"2024-09-19T11:26:27.907839+0000","last_peered":"2024-09-19T11:26:27.907839+0000","last_clean":"2024-09-19T11:26:27.907839+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:26:27.907839+0000","last_undegraded":"2024-09-19T11:26:27.907839+0000","last_fullsized":"2024-09-19T11:26:27.907839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.907726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:48.160182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281352+0000","last_change":"2024-09-19T11:23:07.247890+0000","last_active":"2024-09-19T11:23:42.281352+0000","last_peered":"2024-09-19T11:23:42.281352+0000","last_clean":"2024-09-19T11:23:42.281352+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:23:42.281352+0000","last_undegraded":"2024-09-19T11:23:42.281352+0000","last_fullsized":"2024-09-19T11:23:42.281352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.412033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.151902+0000","last_change":"2024-09-19T11:23:09.246755+0000","last_active":"2024-09-19T11:26:19.151902+0000","last_peered":"2024-09-19T11:26:19.151902+0000","last_clean":"2024-09-19T11:26:19.151902+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:26:19.151902+0000","last_undegraded":"2024-09-19T11:26:19.151902+0000","last_fullsized":"2024-09-19T11:26:19.151902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.229678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278129+0000","last_change":"2024-09-19T11:23:11.266713+0000","last_active":"2024-09-19T11:23:42.278129+0000","last_peered":"2024-09-19T11:23:42.278129+0000","last_clean":"2024-09-19T11:23:42.278129+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:23:42.278129+0000","last_undegraded":"2024-09-19T11:23:42.278129+0000","last_fullsized":"2024-09-19T11:23:42.278129+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:38.162577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283516+0000","last_change":"2024-09-19T11:23:13.268771+0000","last_active":"2024-09-19T11:23:42.283516+0000","last_peered":"2024-09-19T11:23:42.283516+0000","last_clean":"2024-09-19T11:23:42.283516+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:23:42.283516+0000","last_undegraded":"2024-09-19T11:23:42.283516+0000","last_fullsized":"2024-09-19T11:23:42.283516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:19.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266078+0000","last_change":"2024-09-19T11:23:05.236610+0000","last_active":"2024-09-19T11:26:12.266078+0000","last_peered":"2024-09-19T11:26:12.266078+0000","last_clean":"2024-09-19T11:26:12.266078+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:12.266078+0000","last_undegraded":"2024-09-19T11:26:12.266078+0000","last_fullsized":"2024-09-19T11:26:12.266078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:44.218259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.382940+0000","last_change":"2024-09-19T11:26:41.382940+0000","last_active":"2024-09-19T11:26:41.382940+0000","last_peered":"2024-09-19T11:26:41.382940+0000","last_clean":"2024-09-19T11:26:41.382940+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:26:41.382940+0000","last_undegraded":"2024-09-19T11:26:41.382940+0000","last_fullsized":"2024-09-19T11:26:41.382940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.382839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:55.560428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278151+0000","last_change":"2024-09-19T11:23:07.247875+0000","last_active":"2024-09-19T11:23:42.278151+0000","last_peered":"2024-09-19T11:23:42.278151+0000","last_clean":"2024-09-19T11:23:42.278151+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:23:42.278151+0000","last_undegraded":"2024-09-19T11:23:42.278151+0000","last_fullsized":"2024-09-19T11:23:42.278151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:20.420374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.155681+0000","last_change":"2024-09-19T11:23:09.257850+0000","last_active":"2024-09-19T11:26:19.155681+0000","last_peered":"2024-09-19T11:26:19.155681+0000","last_clean":"2024-09-19T11:26:19.155681+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:26:19.155681+0000","last_undegraded":"2024-09-19T11:26:19.155681+0000","last_fullsized":"2024-09-19T11:26:19.155681+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:48.134834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943735+0000","last_change":"2024-09-19T11:23:11.271462+0000","last_active":"2024-09-19T11:23:42.943735+0000","last_peered":"2024-09-19T11:23:42.943735+0000","last_clean":"2024-09-19T11:23:42.943735+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:23:42.943735+0000","last_undegraded":"2024-09-19T11:23:42.943735+0000","last_fullsized":"2024-09-19T11:23:42.943735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.194246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368608+0000","last_change":"2024-09-19T11:23:42.282153+0000","last_active":"2024-09-19T11:26:12.368608+0000","last_peered":"2024-09-19T11:26:12.368608+0000","last_clean":"2024-09-19T11:26:12.368608+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:12.368608+0000","last_undegraded":"2024-09-19T11:26:12.368608+0000","last_fullsized":"2024-09-19T11:26:12.368608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.828536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.898841+0000","last_change":"2024-09-19T11:26:26.898841+0000","last_active":"2024-09-19T11:26:26.898841+0000","last_peered":"2024-09-19T11:26:26.898841+0000","last_clean":"2024-09-19T11:26:26.898841+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:26:26.898841+0000","last_undegraded":"2024-09-19T11:26:26.898841+0000","last_fullsized":"2024-09-19T11:26:26.898841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.898758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:04.348903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280358+0000","last_change":"2024-09-19T11:23:07.245080+0000","last_active":"2024-09-19T11:23:42.280358+0000","last_peered":"2024-09-19T11:23:42.280358+0000","last_clean":"2024-09-19T11:23:42.280358+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:23:42.280358+0000","last_undegraded":"2024-09-19T11:23:42.280358+0000","last_fullsized":"2024-09-19T11:23:42.280358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:41.815514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'29","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.120252+0000","last_change":"2024-09-19T11:23:09.258685+0000","last_active":"2024-09-19T11:26:19.120252+0000","last_peered":"2024-09-19T11:26:19.120252+0000","last_clean":"2024-09-19T11:26:19.120252+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:26:19.120252+0000","last_undegraded":"2024-09-19T11:26:19.120252+0000","last_fullsized":"2024-09-19T11:26:19.120252+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:33.508526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944317+0000","last_change":"2024-09-19T11:23:11.267904+0000","last_active":"2024-09-19T11:23:42.944317+0000","last_peered":"2024-09-19T11:23:42.944317+0000","last_clean":"2024-09-19T11:23:42.944317+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:23:42.944317+0000","last_undegraded":"2024-09-19T11:23:42.944317+0000","last_fullsized":"2024-09-19T11:23:42.944317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:16.874318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.032486+0000","last_change":"2024-09-19T11:23:05.235573+0000","last_active":"2024-09-19T11:25:08.032486+0000","last_peered":"2024-09-19T11:25:08.032486+0000","last_clean":"2024-09-19T11:25:08.032486+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:25:08.032486+0000","last_undegraded":"2024-09-19T11:25:08.032486+0000","last_fullsized":"2024-09-19T11:25:08.032486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:07.062938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.916292+0000","last_change":"2024-09-19T11:26:29.916292+0000","last_active":"2024-09-19T11:26:29.916292+0000","last_peered":"2024-09-19T11:26:29.916292+0000","last_clean":"2024-09-19T11:26:29.916292+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:26:29.916292+0000","last_undegraded":"2024-09-19T11:26:29.916292+0000","last_fullsized":"2024-09-19T11:26:29.916292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.916202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:50.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'2","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.365932+0000","last_change":"2024-09-19T11:23:07.247476+0000","last_active":"2024-09-19T11:25:08.365932+0000","last_peered":"2024-09-19T11:25:08.365932+0000","last_clean":"2024-09-19T11:25:08.365932+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:25:08.365932+0000","last_undegraded":"2024-09-19T11:25:08.365932+0000","last_fullsized":"2024-09-19T11:25:08.365932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:03.848364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'37","reported_seq":86,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.129686+0000","last_change":"2024-09-19T11:23:09.255745+0000","last_active":"2024-09-19T11:26:19.129686+0000","last_peered":"2024-09-19T11:26:19.129686+0000","last_clean":"2024-09-19T11:26:19.129686+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:26:19.129686+0000","last_undegraded":"2024-09-19T11:26:19.129686+0000","last_fullsized":"2024-09-19T11:26:19.129686+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:35.538204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279071+0000","last_change":"2024-09-19T11:23:11.266117+0000","last_active":"2024-09-19T11:23:42.279071+0000","last_peered":"2024-09-19T11:23:42.279071+0000","last_clean":"2024-09-19T11:23:42.279071+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:23:42.279071+0000","last_undegraded":"2024-09-19T11:23:42.279071+0000","last_fullsized":"2024-09-19T11:23:42.279071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:22.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.550414+0000","last_change":"2024-09-19T11:23:05.252243+0000","last_active":"2024-09-19T11:24:27.550414+0000","last_peered":"2024-09-19T11:24:27.550414+0000","last_clean":"2024-09-19T11:24:27.550414+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:24:27.550414+0000","last_undegraded":"2024-09-19T11:24:27.550414+0000","last_fullsized":"2024-09-19T11:24:27.550414+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:18.179391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.949825+0000","last_change":"2024-09-19T11:26:31.949825+0000","last_active":"2024-09-19T11:26:31.949825+0000","last_peered":"2024-09-19T11:26:31.949825+0000","last_clean":"2024-09-19T11:26:31.949825+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:26:31.949825+0000","last_undegraded":"2024-09-19T11:26:31.949825+0000","last_fullsized":"2024-09-19T11:26:31.949825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.949702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:38.587259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.024977+0000","last_change":"2024-09-19T11:23:05.238242+0000","last_active":"2024-09-19T11:26:12.024977+0000","last_peered":"2024-09-19T11:26:12.024977+0000","last_clean":"2024-09-19T11:26:12.024977+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:12.024977+0000","last_undegraded":"2024-09-19T11:26:12.024977+0000","last_fullsized":"2024-09-19T11:26:12.024977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:58.517138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.905294+0000","last_change":"2024-09-19T11:26:27.905294+0000","last_active":"2024-09-19T11:26:27.905294+0000","last_peered":"2024-09-19T11:26:27.905294+0000","last_clean":"2024-09-19T11:26:27.905294+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:26:27.905294+0000","last_undegraded":"2024-09-19T11:26:27.905294+0000","last_fullsized":"2024-09-19T11:26:27.905294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.905202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:43:38.666093+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342000+0000","last_change":"2024-09-19T11:23:05.237372+0000","last_active":"2024-09-19T11:24:27.342000+0000","last_peered":"2024-09-19T11:24:27.342000+0000","last_clean":"2024-09-19T11:24:27.342000+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:24:27.342000+0000","last_undegraded":"2024-09-19T11:24:27.342000+0000","last_fullsized":"2024-09-19T11:24:27.342000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:27.975988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:43.940912+0000","last_change":"2024-09-19T11:23:03.256664+0000","last_active":"2024-09-19T11:23:43.940912+0000","last_peered":"2024-09-19T11:23:43.940912+0000","last_clean":"2024-09-19T11:23:43.940912+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:23:43.940912+0000","last_undegraded":"2024-09-19T11:23:43.940912+0000","last_fullsized":"2024-09-19T11:23:43.940912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:24.307449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.011182+0000","last_change":"2024-09-19T11:23:05.237761+0000","last_active":"2024-09-19T11:23:45.011182+0000","last_peered":"2024-09-19T11:23:45.011182+0000","last_clean":"2024-09-19T11:23:45.011182+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:23:45.011182+0000","last_undegraded":"2024-09-19T11:23:45.011182+0000","last_fullsized":"2024-09-19T11:23:45.011182+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:29:24.776293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.953783+0000","last_change":"2024-09-19T11:26:30.953783+0000","last_active":"2024-09-19T11:26:30.953783+0000","last_peered":"2024-09-19T11:26:30.953783+0000","last_clean":"2024-09-19T11:26:30.953783+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:26:30.953783+0000","last_undegraded":"2024-09-19T11:26:30.953783+0000","last_fullsized":"2024-09-19T11:26:30.953783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.953693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:32.144901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.338525+0000","last_change":"2024-09-19T11:23:05.240645+0000","last_active":"2024-09-19T11:26:12.338525+0000","last_peered":"2024-09-19T11:26:12.338525+0000","last_clean":"2024-09-19T11:26:12.338525+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:12.338525+0000","last_undegraded":"2024-09-19T11:26:12.338525+0000","last_fullsized":"2024-09-19T11:26:12.338525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:18.188580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.952079+0000","last_change":"2024-09-19T11:26:29.952079+0000","last_active":"2024-09-19T11:26:29.952079+0000","last_peered":"2024-09-19T11:26:29.952079+0000","last_clean":"2024-09-19T11:26:29.952079+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:29.952079+0000","last_undegraded":"2024-09-19T11:26:29.952079+0000","last_fullsized":"2024-09-19T11:26:29.952079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.951980+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:30.180814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342109+0000","last_change":"2024-09-19T11:23:05.251796+0000","last_active":"2024-09-19T11:24:27.342109+0000","last_peered":"2024-09-19T11:24:27.342109+0000","last_clean":"2024-09-19T11:24:27.342109+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:24:27.342109+0000","last_undegraded":"2024-09-19T11:24:27.342109+0000","last_fullsized":"2024-09-19T11:24:27.342109+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:53.953637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.867275+0000","last_change":"2024-09-19T11:26:30.867275+0000","last_active":"2024-09-19T11:26:30.867275+0000","last_peered":"2024-09-19T11:26:30.867275+0000","last_clean":"2024-09-19T11:26:30.867275+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:26:30.867275+0000","last_undegraded":"2024-09-19T11:26:30.867275+0000","last_fullsized":"2024-09-19T11:26:30.867275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.867184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:38.055814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279051+0000","last_change":"2024-09-19T11:23:05.248747+0000","last_active":"2024-09-19T11:23:42.279051+0000","last_peered":"2024-09-19T11:23:42.279051+0000","last_clean":"2024-09-19T11:23:42.279051+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:23:42.279051+0000","last_undegraded":"2024-09-19T11:23:42.279051+0000","last_fullsized":"2024-09-19T11:23:42.279051+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:39.771789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.977981+0000","last_change":"2024-09-19T11:26:32.977981+0000","last_active":"2024-09-19T11:26:32.977981+0000","last_peered":"2024-09-19T11:26:32.977981+0000","last_clean":"2024-09-19T11:26:32.977981+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:32.977981+0000","last_undegraded":"2024-09-19T11:26:32.977981+0000","last_fullsized":"2024-09-19T11:26:32.977981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.977891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:36.076149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374459+0000","last_change":"2024-09-19T11:23:05.248818+0000","last_active":"2024-09-19T11:26:12.374459+0000","last_peered":"2024-09-19T11:26:12.374459+0000","last_clean":"2024-09-19T11:26:12.374459+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:12.374459+0000","last_undegraded":"2024-09-19T11:26:12.374459+0000","last_fullsized":"2024-09-19T11:26:12.374459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:41.343464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.990845+0000","last_change":"2024-09-19T11:26:31.990845+0000","last_active":"2024-09-19T11:26:31.990845+0000","last_peered":"2024-09-19T11:26:31.990845+0000","last_clean":"2024-09-19T11:26:31.990845+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:26:31.990845+0000","last_undegraded":"2024-09-19T11:26:31.990845+0000","last_fullsized":"2024-09-19T11:26:31.990845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.990752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:33.522127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342282+0000","last_change":"2024-09-19T11:23:05.248696+0000","last_active":"2024-09-19T11:24:27.342282+0000","last_peered":"2024-09-19T11:24:27.342282+0000","last_clean":"2024-09-19T11:24:27.342282+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:24:27.342282+0000","last_undegraded":"2024-09-19T11:24:27.342282+0000","last_fullsized":"2024-09-19T11:24:27.342282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:37:55.133352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944801+0000","last_change":"2024-09-19T11:23:03.255783+0000","last_active":"2024-09-19T11:23:42.944801+0000","last_peered":"2024-09-19T11:23:42.944801+0000","last_clean":"2024-09-19T11:23:42.944801+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:23:42.944801+0000","last_undegraded":"2024-09-19T11:23:42.944801+0000","last_fullsized":"2024-09-19T11:23:42.944801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:24.578577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368100+0000","last_change":"2024-09-19T11:23:05.243797+0000","last_active":"2024-09-19T11:26:12.368100+0000","last_peered":"2024-09-19T11:26:12.368100+0000","last_clean":"2024-09-19T11:26:12.368100+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:26:12.368100+0000","last_undegraded":"2024-09-19T11:26:12.368100+0000","last_fullsized":"2024-09-19T11:26:12.368100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:33.524342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.017041+0000","last_change":"2024-09-19T11:26:33.017041+0000","last_active":"2024-09-19T11:26:33.017041+0000","last_peered":"2024-09-19T11:26:33.017041+0000","last_clean":"2024-09-19T11:26:33.017041+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:26:33.017041+0000","last_undegraded":"2024-09-19T11:26:33.017041+0000","last_fullsized":"2024-09-19T11:26:33.017041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.016950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:06.549550+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342015+0000","last_change":"2024-09-19T11:23:05.241366+0000","last_active":"2024-09-19T11:24:27.342015+0000","last_peered":"2024-09-19T11:24:27.342015+0000","last_clean":"2024-09-19T11:24:27.342015+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:24:27.342015+0000","last_undegraded":"2024-09-19T11:24:27.342015+0000","last_fullsized":"2024-09-19T11:24:27.342015+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:20:56.281842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.853602+0000","last_change":"2024-09-19T11:26:31.853602+0000","last_active":"2024-09-19T11:26:31.853602+0000","last_peered":"2024-09-19T11:26:31.853602+0000","last_clean":"2024-09-19T11:26:31.853602+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:26:31.853602+0000","last_undegraded":"2024-09-19T11:26:31.853602+0000","last_fullsized":"2024-09-19T11:26:31.853602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.853479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:16:07.826853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:45.482864+0000","last_change":"2024-09-19T11:23:05.246852+0000","last_active":"2024-09-19T11:23:45.482864+0000","last_peered":"2024-09-19T11:23:45.482864+0000","last_clean":"2024-09-19T11:23:45.482864+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:23:45.482864+0000","last_undegraded":"2024-09-19T11:23:45.482864+0000","last_fullsized":"2024-09-19T11:23:45.482864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.057981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.886892+0000","last_change":"2024-09-19T11:26:32.886892+0000","last_active":"2024-09-19T11:26:32.886892+0000","last_peered":"2024-09-19T11:26:32.886892+0000","last_clean":"2024-09-19T11:26:32.886892+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:26:32.886892+0000","last_undegraded":"2024-09-19T11:26:32.886892+0000","last_fullsized":"2024-09-19T11:26:32.886892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.886787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:19.114834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:44.433129+0000","last_change":"2024-09-19T11:23:05.245754+0000","last_active":"2024-09-19T11:23:44.433129+0000","last_peered":"2024-09-19T11:23:44.433129+0000","last_clean":"2024-09-19T11:23:44.433129+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:23:44.433129+0000","last_undegraded":"2024-09-19T11:23:44.433129+0000","last_fullsized":"2024-09-19T11:23:44.433129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:22.898396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:03.241382+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:01.554767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.266030+0000","last_change":"2024-09-19T11:23:05.248307+0000","last_active":"2024-09-19T11:26:12.266030+0000","last_peered":"2024-09-19T11:26:12.266030+0000","last_clean":"2024-09-19T11:26:12.266030+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:12.266030+0000","last_undegraded":"2024-09-19T11:26:12.266030+0000","last_fullsized":"2024-09-19T11:26:12.266030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:01:44.091824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946371+0000","last_change":"2024-09-19T11:23:03.256127+0000","last_active":"2024-09-19T11:23:42.946371+0000","last_peered":"2024-09-19T11:23:42.946371+0000","last_clean":"2024-09-19T11:23:42.946371+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:23:42.946371+0000","last_undegraded":"2024-09-19T11:23:42.946371+0000","last_fullsized":"2024-09-19T11:23:42.946371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:03.696227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123790+0000","last_change":"2024-09-19T11:23:05.250503+0000","last_active":"2024-09-19T11:26:12.123790+0000","last_peered":"2024-09-19T11:26:12.123790+0000","last_clean":"2024-09-19T11:26:12.123790+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:26:12.123790+0000","last_undegraded":"2024-09-19T11:26:12.123790+0000","last_fullsized":"2024-09-19T11:26:12.123790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:46.547197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.979119+0000","last_change":"2024-09-19T11:26:33.979119+0000","last_active":"2024-09-19T11:26:33.979119+0000","last_peered":"2024-09-19T11:26:33.979119+0000","last_clean":"2024-09-19T11:26:33.979119+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:26:33.979119+0000","last_undegraded":"2024-09-19T11:26:33.979119+0000","last_fullsized":"2024-09-19T11:26:33.979119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.979000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:24.446915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368312+0000","last_change":"2024-09-19T11:23:05.248701+0000","last_active":"2024-09-19T11:26:12.368312+0000","last_peered":"2024-09-19T11:26:12.368312+0000","last_clean":"2024-09-19T11:26:12.368312+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:12.368312+0000","last_undegraded":"2024-09-19T11:26:12.368312+0000","last_fullsized":"2024-09-19T11:26:12.368312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:20.161230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947585+0000","last_change":"2024-09-19T11:23:03.252640+0000","last_active":"2024-09-19T11:23:42.947585+0000","last_peered":"2024-09-19T11:23:42.947585+0000","last_clean":"2024-09-19T11:23:42.947585+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:23:42.947585+0000","last_undegraded":"2024-09-19T11:23:42.947585+0000","last_fullsized":"2024-09-19T11:23:42.947585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:25.619002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.938414+0000","last_change":"2024-09-19T11:23:03.256515+0000","last_active":"2024-09-19T11:26:11.938414+0000","last_peered":"2024-09-19T11:26:11.938414+0000","last_clean":"2024-09-19T11:26:11.938414+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:26:11.938414+0000","last_undegraded":"2024-09-19T11:26:11.938414+0000","last_fullsized":"2024-09-19T11:26:11.938414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:02.211011+0000","last_clean_scrub_stamp":"2024-09-19T11:23:02.211011+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:08.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123873+0000","last_change":"2024-09-19T11:23:05.248691+0000","last_active":"2024-09-19T11:26:12.123873+0000","last_peered":"2024-09-19T11:26:12.123873+0000","last_clean":"2024-09-19T11:26:12.123873+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:12.123873+0000","last_undegraded":"2024-09-19T11:26:12.123873+0000","last_fullsized":"2024-09-19T11:26:12.123873+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:20.035436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.990707+0000","last_change":"2024-09-19T11:26:33.990707+0000","last_active":"2024-09-19T11:26:33.990707+0000","last_peered":"2024-09-19T11:26:33.990707+0000","last_clean":"2024-09-19T11:26:33.990707+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:26:33.990707+0000","last_undegraded":"2024-09-19T11:26:33.990707+0000","last_fullsized":"2024-09-19T11:26:33.990707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.990617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:35.040214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.510844+0000","last_change":"2024-09-19T11:23:05.245906+0000","last_active":"2024-09-19T11:25:11.510844+0000","last_peered":"2024-09-19T11:25:11.510844+0000","last_clean":"2024-09-19T11:25:11.510844+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:25:11.510844+0000","last_undegraded":"2024-09-19T11:25:11.510844+0000","last_fullsized":"2024-09-19T11:25:11.510844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:19.236446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1374,"ondisk_log_size":1374,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673010,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":150572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673010,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":737565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705714,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":739071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":148560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:26:44.209 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.209 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.210 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.210 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.211 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.211 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.211 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.212 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.212 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.213 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.213 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.213 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.213 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.214 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.214 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.214 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.214 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.215 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.215 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.215 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.216 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.216 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.217 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.217 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.217 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.218 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.218 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.218 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.218 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.219 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.219 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.219 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.219 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.220 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.220 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.220 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.220 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.221 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.221 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.221 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.221 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.222 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.222 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.222 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.222 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.223 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.223 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.223 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.224 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.224 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.224 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.224 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.224 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.224 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.225 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.225 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.225 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.225 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.225 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.226 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.226 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.226 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.226 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.226 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.227 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.227 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.227 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.227 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.227 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.227 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.228 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.228 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.228 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.228 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.228 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.229 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.229 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.229 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.229 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.230 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.230 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.230 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.230 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.230 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.230 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.230 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.231 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.231 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.231 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.231 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.231 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.232 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.232 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.232 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.232 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.233 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.233 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.233 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.233 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.233 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.234 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.235 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.236 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.237 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.237 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.237 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.237 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.237 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.237 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.238 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.238 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.238 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.238 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.238 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.239 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.239 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.239 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.239 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.239 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.240 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.240 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.240 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.241 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.241 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.241 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.241 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.241 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.241 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.242 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.242 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.243 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.244 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.244 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.244 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.244 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.244 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.244 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.245 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.245 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.245 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.245 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.245 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.245 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.246 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.246 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.246 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.247 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.247 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.247 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.247 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.248 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.248 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.248 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.248 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.249 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.249 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.249 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.249 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.249 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.250 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.250 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.250 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.250 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-19T11:23:02.211011+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=2, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.250 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.251 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:26:44.251 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T11:27:04.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:27:04.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.365+0000 7fb162afe640 1 Processor -- start 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.365+0000 7fb162afe640 1 -- start start 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb162afe640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb15c12cba0 0x7fb15c1315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb162afe640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb15c09a290 con 0x7fb15c12c0b0 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb162afe640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb15c071860 con 0x7fb15c12cba0 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb16089a640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb15c12c0b0 0x7fb15c12c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:59776/0 (socket says 172.21.15.22:59776) 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb16089a640 1 -- 172.21.15.22:0/2247450870 learned_addr learned my addr 172.21.15.22:0/2247450870 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb15bfff640 1 --2- 172.21.15.22:0/2247450870 >> v2:172.21.15.22:3300/0 conn(0x7fb15c12cba0 0x7fb15c1315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb15b7fe640 1 -- 172.21.15.22:0/2247450870 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 222802135 0 0) 0x7fb15c09a290 con 0x7fb15c12c0b0 2024-09-19T11:27:04.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb15b7fe640 1 -- 172.21.15.22:0/2247450870 --> v1:172.21.15.22:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb144003520 con 0x7fb15c12c0b0 2024-09-19T11:27:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb15bfff640 1 -- 172.21.15.22:0/2247450870 >> v1:172.21.15.22:6789/0 conn(0x7fb15c12c0b0 legacy=0x7fb15c12c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:04.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.366+0000 7fb15bfff640 1 -- 172.21.15.22:0/2247450870 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15c099e20 con 0x7fb15c12cba0 2024-09-19T11:27:04.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.367+0000 7fb15bfff640 1 --2- 172.21.15.22:0/2247450870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12cba0 0x7fb15c1315e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb150009750 tx=0x7fb15002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ce846de777ee5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:04.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.367+0000 7fb15b7fe640 1 -- 172.21.15.22:0/2247450870 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb150037070 con 0x7fb15c12cba0 2024-09-19T11:27:04.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.367+0000 7fb15b7fe640 1 -- 172.21.15.22:0/2247450870 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15002ad80 con 0x7fb15c12cba0 2024-09-19T11:27:04.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.367+0000 7fb15b7fe640 1 -- 172.21.15.22:0/2247450870 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb150032590 con 0x7fb15c12cba0 2024-09-19T11:27:04.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.368+0000 7fb162afe640 1 -- 172.21.15.22:0/2247450870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12cba0 msgr2=0x7fb15c1315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.368+0000 7fb162afe640 1 --2- 172.21.15.22:0/2247450870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12cba0 0x7fb15c1315e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb150009750 tx=0x7fb15002a140 comp rx=0 tx=0).stop 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.368+0000 7fb162afe640 1 -- 172.21.15.22:0/2247450870 shutdown_connections 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.368+0000 7fb162afe640 1 --2- 172.21.15.22:0/2247450870 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12cba0 0x7fb15c1315e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.368+0000 7fb162afe640 1 -- 172.21.15.22:0/2247450870 >> 172.21.15.22:0/2247450870 conn(0x7fb15c09abc0 msgr2=0x7fb15c098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.368+0000 7fb162afe640 1 -- 172.21.15.22:0/2247450870 shutdown_connections 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.368+0000 7fb162afe640 1 -- 172.21.15.22:0/2247450870 wait complete. 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.369+0000 7fb162afe640 1 Processor -- start 2024-09-19T11:27:04.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.369+0000 7fb162afe640 1 -- start start 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.369+0000 7fb162afe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12c0b0 0x7fb15c1c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.369+0000 7fb162afe640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb15c070c40 con 0x7fb15c12c0b0 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.369+0000 7fb16089a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12c0b0 0x7fb15c1c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.369+0000 7fb16089a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12c0b0 0x7fb15c1c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55198/0 (socket says 172.21.15.22:55198) 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.369+0000 7fb16089a640 1 -- 172.21.15.22:0/2647089487 learned_addr learned my addr 172.21.15.22:0/2647089487 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.370+0000 7fb16089a640 1 -- 172.21.15.22:0/2647089487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15c1285e0 con 0x7fb15c12c0b0 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.370+0000 7fb16089a640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12c0b0 0x7fb15c1c1c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c480 tx=0x7fb14c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.370+0000 7fb1597fa640 1 -- 172.21.15.22:0/2647089487 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb14c017070 con 0x7fb15c12c0b0 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.370+0000 7fb1597fa640 1 -- 172.21.15.22:0/2647089487 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14c005cd0 con 0x7fb15c12c0b0 2024-09-19T11:27:04.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.370+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15c126890 con 0x7fb15c12c0b0 2024-09-19T11:27:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.370+0000 7fb1597fa640 1 -- 172.21.15.22:0/2647089487 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb14c007200 con 0x7fb15c12c0b0 2024-09-19T11:27:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.370+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb15c1c23a0 con 0x7fb15c12c0b0 2024-09-19T11:27:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.371+0000 7fb1597fa640 1 -- 172.21.15.22:0/2647089487 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fb14c01a070 con 0x7fb15c12c0b0 2024-09-19T11:27:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.371+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb15c12c490 con 0x7fb15c12c0b0 2024-09-19T11:27:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.372+0000 7fb1597fa640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb14803e710 0x7fb148040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.372+0000 7fb1597fa640 1 -- 172.21.15.22:0/2647089487 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fb14c052ac0 con 0x7fb15c12c0b0 2024-09-19T11:27:04.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.372+0000 7fb15bfff640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb14803e710 0x7fb148040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:04.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.372+0000 7fb15bfff640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb14803e710 0x7fb148040bd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb150034000 tx=0x7fb15002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:04.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.376+0000 7fb1597fa640 1 -- 172.21.15.22:0/2647089487 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fb14c01b3b0 con 0x7fb15c12c0b0 2024-09-19T11:27:04.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.559+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb15c050ee0 con 0x7fb14803e710 2024-09-19T11:27:04.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.570+0000 7fb1597fa640 1 -- 172.21.15.22:0/2647089487 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665904 (secure 0 0 0) 0x7fb15c050ee0 con 0x7fb14803e710 2024-09-19T11:27:04.579 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:27:04.584 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb14803e710 msgr2=0x7fb148040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb14803e710 0x7fb148040bd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb150034000 tx=0x7fb15002e000 comp rx=0 tx=0).stop 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12c0b0 msgr2=0x7fb15c1c1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12c0b0 0x7fb15c1c1c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00c480 tx=0x7fb14c00c940 comp rx=0 tx=0).stop 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 shutdown_connections 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb14803e710 0x7fb148040bd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 --2- 172.21.15.22:0/2647089487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb15c12c0b0 0x7fb15c1c1c80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 >> 172.21.15.22:0/2647089487 conn(0x7fb15c09abc0 msgr2=0x7fb15c06dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 shutdown_connections 2024-09-19T11:27:04.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:04.575+0000 7fb162afe640 1 -- 172.21.15.22:0/2647089487 wait complete. 2024-09-19T11:27:04.588 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-19T11:27:02.859270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605410,"num_objects":222,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":0,"num_read":5189,"num_read_kb":4731,"num_write":2121,"num_write_kb":60587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1042,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4601,"ondisk_log_size":4601,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":115136,"kb_used_data":2880,"kb_used_omap":6,"kb_used_meta":112185,"kb_avail":377372224,"statfs":{"total":386547056640,"available":386429157376,"internally_reserved":0,"allocated":2949120,"data_stored":1775768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":114878230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1042,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005016"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.932354+0000","last_change":"2024-09-19T11:26:37.932354+0000","last_active":"2024-09-19T11:26:37.932354+0000","last_peered":"2024-09-19T11:26:37.932354+0000","last_clean":"2024-09-19T11:26:37.932354+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:26:37.932354+0000","last_undegraded":"2024-09-19T11:26:37.932354+0000","last_fullsized":"2024-09-19T11:26:37.932354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.932266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:41.727377+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.069528+0000","last_change":"2024-09-19T11:26:55.069528+0000","last_active":"2024-09-19T11:26:55.069528+0000","last_peered":"2024-09-19T11:26:55.069528+0000","last_clean":"2024-09-19T11:26:55.069528+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:55.069528+0000","last_undegraded":"2024-09-19T11:26:55.069528+0000","last_fullsized":"2024-09-19T11:26:55.069528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.069440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:55.494351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.990885+0000","last_change":"2024-09-19T11:26:36.990885+0000","last_active":"2024-09-19T11:26:36.990885+0000","last_peered":"2024-09-19T11:26:36.990885+0000","last_clean":"2024-09-19T11:26:36.990885+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:26:36.990885+0000","last_undegraded":"2024-09-19T11:26:36.990885+0000","last_fullsized":"2024-09-19T11:26:36.990885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.990795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:53.995221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.333061+0000","last_change":"2024-09-19T11:23:05.242656+0000","last_active":"2024-09-19T11:26:12.333061+0000","last_peered":"2024-09-19T11:26:12.333061+0000","last_clean":"2024-09-19T11:26:12.333061+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:26:12.333061+0000","last_undegraded":"2024-09-19T11:26:12.333061+0000","last_fullsized":"2024-09-19T11:26:12.333061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.791517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.390768+0000","last_change":"2024-09-19T11:26:51.390768+0000","last_active":"2024-09-19T11:26:51.390768+0000","last_peered":"2024-09-19T11:26:51.390768+0000","last_clean":"2024-09-19T11:26:51.390768+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:26:51.390768+0000","last_undegraded":"2024-09-19T11:26:51.390768+0000","last_fullsized":"2024-09-19T11:26:51.390768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.390678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:06.584463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'12","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.945164+0000","last_change":"2024-09-19T11:26:50.945164+0000","last_active":"2024-09-19T11:26:50.945164+0000","last_peered":"2024-09-19T11:26:50.945164+0000","last_clean":"2024-09-19T11:26:50.945164+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:50.945164+0000","last_undegraded":"2024-09-19T11:26:50.945164+0000","last_fullsized":"2024-09-19T11:26:50.945164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.945069+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:26:56.726703+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.958139+0000","last_change":"2024-09-19T11:26:36.958139+0000","last_active":"2024-09-19T11:26:36.958139+0000","last_peered":"2024-09-19T11:26:36.958139+0000","last_clean":"2024-09-19T11:26:36.958139+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:26:36.958139+0000","last_undegraded":"2024-09-19T11:26:36.958139+0000","last_fullsized":"2024-09-19T11:26:36.958139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.958025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:34.544025+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944750+0000","last_change":"2024-09-19T11:23:05.233162+0000","last_active":"2024-09-19T11:23:42.944750+0000","last_peered":"2024-09-19T11:23:42.944750+0000","last_clean":"2024-09-19T11:23:42.944750+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:23:42.944750+0000","last_undegraded":"2024-09-19T11:23:42.944750+0000","last_fullsized":"2024-09-19T11:23:42.944750+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:49.026520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.373571+0000","last_change":"2024-09-19T11:26:50.373571+0000","last_active":"2024-09-19T11:26:50.373571+0000","last_peered":"2024-09-19T11:26:50.373571+0000","last_clean":"2024-09-19T11:26:50.373571+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:26:50.373571+0000","last_undegraded":"2024-09-19T11:26:50.373571+0000","last_fullsized":"2024-09-19T11:26:50.373571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.373464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:22:03.662429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'12","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.036130+0000","last_change":"2024-09-19T11:26:56.036130+0000","last_active":"2024-09-19T11:26:56.036130+0000","last_peered":"2024-09-19T11:26:56.036130+0000","last_clean":"2024-09-19T11:26:56.036130+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:56.036130+0000","last_undegraded":"2024-09-19T11:26:56.036130+0000","last_fullsized":"2024-09-19T11:26:56.036130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.036029+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.756679+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.972860+0000","last_change":"2024-09-19T11:26:35.972860+0000","last_active":"2024-09-19T11:26:35.972860+0000","last_peered":"2024-09-19T11:26:35.972860+0000","last_clean":"2024-09-19T11:26:35.972860+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:26:35.972860+0000","last_undegraded":"2024-09-19T11:26:35.972860+0000","last_fullsized":"2024-09-19T11:26:35.972860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.972760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:17.815936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.374918+0000","last_change":"2024-09-19T11:23:05.254262+0000","last_active":"2024-09-19T11:26:12.374918+0000","last_peered":"2024-09-19T11:26:12.374918+0000","last_clean":"2024-09-19T11:26:12.374918+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:26:12.374918+0000","last_undegraded":"2024-09-19T11:26:12.374918+0000","last_fullsized":"2024-09-19T11:26:12.374918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:34:58.352519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.431257+0000","last_change":"2024-09-19T11:26:48.431257+0000","last_active":"2024-09-19T11:26:48.431257+0000","last_peered":"2024-09-19T11:26:48.431257+0000","last_clean":"2024-09-19T11:26:48.431257+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:48.431257+0000","last_undegraded":"2024-09-19T11:26:48.431257+0000","last_fullsized":"2024-09-19T11:26:48.431257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.431171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:57.441652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.745135+0000","last_change":"2024-09-19T11:26:49.745135+0000","last_active":"2024-09-19T11:26:49.745135+0000","last_peered":"2024-09-19T11:26:49.745135+0000","last_clean":"2024-09-19T11:26:49.745135+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:49.745135+0000","last_undegraded":"2024-09-19T11:26:49.745135+0000","last_fullsized":"2024-09-19T11:26:49.745135+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.745054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:51.276429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.989858+0000","last_change":"2024-09-19T11:26:35.989858+0000","last_active":"2024-09-19T11:26:35.989858+0000","last_peered":"2024-09-19T11:26:35.989858+0000","last_clean":"2024-09-19T11:26:35.989858+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:26:35.989858+0000","last_undegraded":"2024-09-19T11:26:35.989858+0000","last_fullsized":"2024-09-19T11:26:35.989858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.989770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.409598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.783644+0000","last_change":"2024-09-19T11:26:50.783644+0000","last_active":"2024-09-19T11:26:50.783644+0000","last_peered":"2024-09-19T11:26:50.783644+0000","last_clean":"2024-09-19T11:26:50.783644+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:26:50.783644+0000","last_undegraded":"2024-09-19T11:26:50.783644+0000","last_fullsized":"2024-09-19T11:26:50.783644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.783546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:56.110032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945568+0000","last_change":"2024-09-19T11:23:05.241849+0000","last_active":"2024-09-19T11:23:42.945568+0000","last_peered":"2024-09-19T11:23:42.945568+0000","last_clean":"2024-09-19T11:23:42.945568+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:23:42.945568+0000","last_undegraded":"2024-09-19T11:23:42.945568+0000","last_fullsized":"2024-09-19T11:23:42.945568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:04.673024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.946461+0000","last_change":"2024-09-19T11:26:34.946461+0000","last_active":"2024-09-19T11:26:34.946461+0000","last_peered":"2024-09-19T11:26:34.946461+0000","last_clean":"2024-09-19T11:26:34.946461+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:26:34.946461+0000","last_undegraded":"2024-09-19T11:26:34.946461+0000","last_fullsized":"2024-09-19T11:26:34.946461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.946370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:48.027671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.017232+0000","last_change":"2024-09-19T11:26:51.017232+0000","last_active":"2024-09-19T11:26:51.017232+0000","last_peered":"2024-09-19T11:26:51.017232+0000","last_clean":"2024-09-19T11:26:51.017232+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:26:51.017232+0000","last_undegraded":"2024-09-19T11:26:51.017232+0000","last_fullsized":"2024-09-19T11:26:51.017232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.017146+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.668961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.952151+0000","last_change":"2024-09-19T11:26:34.952151+0000","last_active":"2024-09-19T11:26:34.952151+0000","last_peered":"2024-09-19T11:26:34.952151+0000","last_clean":"2024-09-19T11:26:34.952151+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:26:34.952151+0000","last_undegraded":"2024-09-19T11:26:34.952151+0000","last_fullsized":"2024-09-19T11:26:34.952151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.952058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:03.872887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.018264+0000","last_change":"2024-09-19T11:26:52.018264+0000","last_active":"2024-09-19T11:26:52.018264+0000","last_peered":"2024-09-19T11:26:52.018264+0000","last_clean":"2024-09-19T11:26:52.018264+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:26:52.018264+0000","last_undegraded":"2024-09-19T11:26:52.018264+0000","last_fullsized":"2024-09-19T11:26:52.018264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.018180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:31.898878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.916754+0000","last_change":"2024-09-19T11:26:33.916754+0000","last_active":"2024-09-19T11:26:33.916754+0000","last_peered":"2024-09-19T11:26:33.916754+0000","last_clean":"2024-09-19T11:26:33.916754+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:33.916754+0000","last_undegraded":"2024-09-19T11:26:33.916754+0000","last_fullsized":"2024-09-19T11:26:33.916754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.916663+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:35.684322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":68,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.898475+0000","last_change":"2024-09-19T11:26:51.898475+0000","last_active":"2024-09-19T11:26:51.898475+0000","last_peered":"2024-09-19T11:26:51.898475+0000","last_clean":"2024-09-19T11:26:51.898475+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:51.898475+0000","last_undegraded":"2024-09-19T11:26:51.898475+0000","last_fullsized":"2024-09-19T11:26:51.898475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.898386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:37.296273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.987445+0000","last_change":"2024-09-19T11:26:37.987445+0000","last_active":"2024-09-19T11:26:37.987445+0000","last_peered":"2024-09-19T11:26:37.987445+0000","last_clean":"2024-09-19T11:26:37.987445+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:26:37.987445+0000","last_undegraded":"2024-09-19T11:26:37.987445+0000","last_fullsized":"2024-09-19T11:26:37.987445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.987357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:27.173996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.779288+0000","last_change":"2024-09-19T11:26:48.779288+0000","last_active":"2024-09-19T11:26:48.779288+0000","last_peered":"2024-09-19T11:26:48.779288+0000","last_clean":"2024-09-19T11:26:48.779288+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:48.779288+0000","last_undegraded":"2024-09-19T11:26:48.779288+0000","last_fullsized":"2024-09-19T11:26:48.779288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.779185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:49.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.001114+0000","last_change":"2024-09-19T11:26:35.001114+0000","last_active":"2024-09-19T11:26:35.001114+0000","last_peered":"2024-09-19T11:26:35.001114+0000","last_clean":"2024-09-19T11:26:35.001114+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:26:35.001114+0000","last_undegraded":"2024-09-19T11:26:35.001114+0000","last_fullsized":"2024-09-19T11:26:35.001114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.001027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:09.576343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.033350+0000","last_change":"2024-09-19T11:26:53.033350+0000","last_active":"2024-09-19T11:26:53.033350+0000","last_peered":"2024-09-19T11:26:53.033350+0000","last_clean":"2024-09-19T11:26:53.033350+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:53.033350+0000","last_undegraded":"2024-09-19T11:26:53.033350+0000","last_fullsized":"2024-09-19T11:26:53.033350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.033252+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:11.509923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.392963+0000","last_change":"2024-09-19T11:26:49.392963+0000","last_active":"2024-09-19T11:26:49.392963+0000","last_peered":"2024-09-19T11:26:49.392963+0000","last_clean":"2024-09-19T11:26:49.392963+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:26:49.392963+0000","last_undegraded":"2024-09-19T11:26:49.392963+0000","last_fullsized":"2024-09-19T11:26:49.392963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.392873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:20.014824+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'37","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.305388+0000","last_change":"2024-09-19T11:23:13.275524+0000","last_active":"2024-09-19T11:26:12.305388+0000","last_peered":"2024-09-19T11:26:12.305388+0000","last_clean":"2024-09-19T11:26:12.305388+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:26:12.305388+0000","last_undegraded":"2024-09-19T11:26:12.305388+0000","last_fullsized":"2024-09-19T11:26:12.305388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.670701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.766852+0000","last_change":"2024-09-19T11:26:43.766852+0000","last_active":"2024-09-19T11:26:43.766852+0000","last_peered":"2024-09-19T11:26:43.766852+0000","last_clean":"2024-09-19T11:26:43.766852+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:43.766852+0000","last_undegraded":"2024-09-19T11:26:43.766852+0000","last_fullsized":"2024-09-19T11:26:43.766852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.766755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:14.579342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.365713+0000","last_change":"2024-09-19T11:26:37.365713+0000","last_active":"2024-09-19T11:26:37.365713+0000","last_peered":"2024-09-19T11:26:37.365713+0000","last_clean":"2024-09-19T11:26:37.365713+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:26:37.365713+0000","last_undegraded":"2024-09-19T11:26:37.365713+0000","last_fullsized":"2024-09-19T11:26:37.365713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.365625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:20.032640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945113+0000","last_change":"2024-09-19T11:23:07.245040+0000","last_active":"2024-09-19T11:23:42.945113+0000","last_peered":"2024-09-19T11:23:42.945113+0000","last_clean":"2024-09-19T11:23:42.945113+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:23:42.945113+0000","last_undegraded":"2024-09-19T11:23:42.945113+0000","last_fullsized":"2024-09-19T11:23:42.945113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:17.056714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.204815+0000","last_change":"2024-09-19T11:23:09.256765+0000","last_active":"2024-09-19T11:26:19.204815+0000","last_peered":"2024-09-19T11:26:19.204815+0000","last_clean":"2024-09-19T11:26:19.204815+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:26:19.204815+0000","last_undegraded":"2024-09-19T11:26:19.204815+0000","last_fullsized":"2024-09-19T11:26:19.204815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:39:32.451376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283356+0000","last_change":"2024-09-19T11:23:11.267021+0000","last_active":"2024-09-19T11:23:42.283356+0000","last_peered":"2024-09-19T11:23:42.283356+0000","last_clean":"2024-09-19T11:23:42.283356+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:23:42.283356+0000","last_undegraded":"2024-09-19T11:23:42.283356+0000","last_fullsized":"2024-09-19T11:23:42.283356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:23.806510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.707113+0000","last_change":"2024-09-19T11:23:13.274325+0000","last_active":"2024-09-19T11:26:11.707113+0000","last_peered":"2024-09-19T11:26:11.707113+0000","last_clean":"2024-09-19T11:26:11.707113+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:26:11.707113+0000","last_undegraded":"2024-09-19T11:26:11.707113+0000","last_fullsized":"2024-09-19T11:26:11.707113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:53:22.517816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.402917+0000","last_change":"2024-09-19T11:26:57.402917+0000","last_active":"2024-09-19T11:26:57.402917+0000","last_peered":"2024-09-19T11:26:57.402917+0000","last_clean":"2024-09-19T11:26:57.402917+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:57.402917+0000","last_undegraded":"2024-09-19T11:26:57.402917+0000","last_fullsized":"2024-09-19T11:26:57.402917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.402830+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:15.259544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.907786+0000","last_change":"2024-09-19T11:26:30.907786+0000","last_active":"2024-09-19T11:26:30.907786+0000","last_peered":"2024-09-19T11:26:30.907786+0000","last_clean":"2024-09-19T11:26:30.907786+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:26:30.907786+0000","last_undegraded":"2024-09-19T11:26:30.907786+0000","last_fullsized":"2024-09-19T11:26:30.907786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.907688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:01.389670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.767856+0000","last_change":"2024-09-19T11:26:59.767856+0000","last_active":"2024-09-19T11:26:59.767856+0000","last_peered":"2024-09-19T11:26:59.767856+0000","last_clean":"2024-09-19T11:26:59.767856+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:26:59.767856+0000","last_undegraded":"2024-09-19T11:26:59.767856+0000","last_fullsized":"2024-09-19T11:26:59.767856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.767773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:00:22.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'7","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.551668+0000","last_change":"2024-09-19T11:23:09.248257+0000","last_active":"2024-09-19T11:24:27.551668+0000","last_peered":"2024-09-19T11:24:27.551668+0000","last_clean":"2024-09-19T11:24:27.551668+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:24:27.551668+0000","last_undegraded":"2024-09-19T11:24:27.551668+0000","last_fullsized":"2024-09-19T11:24:27.551668+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:18.342558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281824+0000","last_change":"2024-09-19T11:23:11.263544+0000","last_active":"2024-09-19T11:23:42.281824+0000","last_peered":"2024-09-19T11:23:42.281824+0000","last_clean":"2024-09-19T11:23:42.281824+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:23:42.281824+0000","last_undegraded":"2024-09-19T11:23:42.281824+0000","last_fullsized":"2024-09-19T11:23:42.281824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:32:44.652949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278313+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:23:42.278313+0000","last_peered":"2024-09-19T11:23:42.278313+0000","last_clean":"2024-09-19T11:23:42.278313+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:23:42.278313+0000","last_undegraded":"2024-09-19T11:23:42.278313+0000","last_fullsized":"2024-09-19T11:23:42.278313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:33:33.618068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.434595+0000","last_change":"2024-09-19T11:26:59.434595+0000","last_active":"2024-09-19T11:26:59.434595+0000","last_peered":"2024-09-19T11:26:59.434595+0000","last_clean":"2024-09-19T11:26:59.434595+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:26:59.434595+0000","last_undegraded":"2024-09-19T11:26:59.434595+0000","last_fullsized":"2024-09-19T11:26:59.434595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.434505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:29.091390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.378428+0000","last_change":"2024-09-19T11:26:35.378428+0000","last_active":"2024-09-19T11:26:35.378428+0000","last_peered":"2024-09-19T11:26:35.378428+0000","last_clean":"2024-09-19T11:26:35.378428+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:26:35.378428+0000","last_undegraded":"2024-09-19T11:26:35.378428+0000","last_fullsized":"2024-09-19T11:26:35.378428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.378338+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:26.014662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.815608+0000","last_change":"2024-09-19T11:26:54.815608+0000","last_active":"2024-09-19T11:26:54.815608+0000","last_peered":"2024-09-19T11:26:54.815608+0000","last_clean":"2024-09-19T11:26:54.815608+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:26:54.815608+0000","last_undegraded":"2024-09-19T11:26:54.815608+0000","last_fullsized":"2024-09-19T11:26:54.815608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.815526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:36.935748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279160+0000","last_change":"2024-09-19T11:23:09.253591+0000","last_active":"2024-09-19T11:23:42.279160+0000","last_peered":"2024-09-19T11:23:42.279160+0000","last_clean":"2024-09-19T11:23:42.279160+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:23:42.279160+0000","last_undegraded":"2024-09-19T11:23:42.279160+0000","last_fullsized":"2024-09-19T11:23:42.279160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:52.970909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'285","reported_seq":389,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.073009+0000","last_change":"2024-09-19T11:23:11.271056+0000","last_active":"2024-09-19T11:26:19.073009+0000","last_peered":"2024-09-19T11:26:19.073009+0000","last_clean":"2024-09-19T11:26:19.073009+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:26:19.073009+0000","last_undegraded":"2024-09-19T11:26:19.073009+0000","last_fullsized":"2024-09-19T11:26:19.073009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:07.822761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.312802+0000","last_change":"2024-09-19T11:23:13.270157+0000","last_active":"2024-09-19T11:26:12.312802+0000","last_peered":"2024-09-19T11:26:12.312802+0000","last_clean":"2024-09-19T11:26:12.312802+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:26:12.312802+0000","last_undegraded":"2024-09-19T11:26:12.312802+0000","last_fullsized":"2024-09-19T11:26:12.312802+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:32.371537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.025004+0000","last_change":"2024-09-19T11:26:42.025004+0000","last_active":"2024-09-19T11:26:42.025004+0000","last_peered":"2024-09-19T11:26:42.025004+0000","last_clean":"2024-09-19T11:26:42.025004+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:26:42.025004+0000","last_undegraded":"2024-09-19T11:26:42.025004+0000","last_fullsized":"2024-09-19T11:26:42.025004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.024919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:31.267851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.870975+0000","last_change":"2024-09-19T11:26:24.870975+0000","last_active":"2024-09-19T11:26:24.870975+0000","last_peered":"2024-09-19T11:26:24.870975+0000","last_clean":"2024-09-19T11:26:24.870975+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:26:24.870975+0000","last_undegraded":"2024-09-19T11:26:24.870975+0000","last_fullsized":"2024-09-19T11:26:24.870975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.870884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:14.567909+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947691+0000","last_change":"2024-09-19T11:23:07.248024+0000","last_active":"2024-09-19T11:23:42.947691+0000","last_peered":"2024-09-19T11:23:42.947691+0000","last_clean":"2024-09-19T11:23:42.947691+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:23:42.947691+0000","last_undegraded":"2024-09-19T11:23:42.947691+0000","last_fullsized":"2024-09-19T11:23:42.947691+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:02.812050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.189031+0000","last_change":"2024-09-19T11:23:09.260866+0000","last_active":"2024-09-19T11:26:19.189031+0000","last_peered":"2024-09-19T11:26:19.189031+0000","last_clean":"2024-09-19T11:26:19.189031+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:26:19.189031+0000","last_undegraded":"2024-09-19T11:26:19.189031+0000","last_fullsized":"2024-09-19T11:26:19.189031+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:22.062690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'285","reported_seq":405,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.078340+0000","last_change":"2024-09-19T11:23:11.260474+0000","last_active":"2024-09-19T11:26:19.078340+0000","last_peered":"2024-09-19T11:26:19.078340+0000","last_clean":"2024-09-19T11:26:19.078340+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:26:19.078340+0000","last_undegraded":"2024-09-19T11:26:19.078340+0000","last_fullsized":"2024-09-19T11:26:19.078340+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:32.540955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.381320+0000","last_change":"2024-09-19T11:26:55.381320+0000","last_active":"2024-09-19T11:26:55.381320+0000","last_peered":"2024-09-19T11:26:55.381320+0000","last_clean":"2024-09-19T11:26:55.381320+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:26:55.381320+0000","last_undegraded":"2024-09-19T11:26:55.381320+0000","last_fullsized":"2024-09-19T11:26:55.381320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.381228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:24:06.217118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.956170+0000","last_change":"2024-09-19T11:26:27.956170+0000","last_active":"2024-09-19T11:26:27.956170+0000","last_peered":"2024-09-19T11:26:27.956170+0000","last_clean":"2024-09-19T11:26:27.956170+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:26:27.956170+0000","last_undegraded":"2024-09-19T11:26:27.956170+0000","last_fullsized":"2024-09-19T11:26:27.956170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.956081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:59.956862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.969498+0000","last_change":"2024-09-19T11:26:57.969498+0000","last_active":"2024-09-19T11:26:57.969498+0000","last_peered":"2024-09-19T11:26:57.969498+0000","last_clean":"2024-09-19T11:26:57.969498+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:26:57.969498+0000","last_undegraded":"2024-09-19T11:26:57.969498+0000","last_fullsized":"2024-09-19T11:26:57.969498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.969412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.712881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'43","reported_seq":95,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.217387+0000","last_change":"2024-09-19T11:23:09.249075+0000","last_active":"2024-09-19T11:26:19.217387+0000","last_peered":"2024-09-19T11:26:19.217387+0000","last_clean":"2024-09-19T11:26:19.217387+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:26:19.217387+0000","last_undegraded":"2024-09-19T11:26:19.217387+0000","last_fullsized":"2024-09-19T11:26:19.217387+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:03.176195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281316+0000","last_change":"2024-09-19T11:23:11.258275+0000","last_active":"2024-09-19T11:23:42.281316+0000","last_peered":"2024-09-19T11:23:42.281316+0000","last_clean":"2024-09-19T11:23:42.281316+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:23:42.281316+0000","last_undegraded":"2024-09-19T11:23:42.281316+0000","last_fullsized":"2024-09-19T11:23:42.281316+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:14.017912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.349482+0000","last_change":"2024-09-19T11:23:13.274689+0000","last_active":"2024-09-19T11:24:27.349482+0000","last_peered":"2024-09-19T11:24:27.349482+0000","last_clean":"2024-09-19T11:24:27.349482+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:24:27.349482+0000","last_undegraded":"2024-09-19T11:24:27.349482+0000","last_fullsized":"2024-09-19T11:24:27.349482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:05.210803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.014072+0000","last_change":"2024-09-19T11:26:39.014072+0000","last_active":"2024-09-19T11:26:39.014072+0000","last_peered":"2024-09-19T11:26:39.014072+0000","last_clean":"2024-09-19T11:26:39.014072+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:26:39.014072+0000","last_undegraded":"2024-09-19T11:26:39.014072+0000","last_fullsized":"2024-09-19T11:26:39.014072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.013977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:33.627023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.962098+0000","last_change":"2024-09-19T11:26:26.962098+0000","last_active":"2024-09-19T11:26:26.962098+0000","last_peered":"2024-09-19T11:26:26.962098+0000","last_clean":"2024-09-19T11:26:26.962098+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:26:26.962098+0000","last_undegraded":"2024-09-19T11:26:26.962098+0000","last_fullsized":"2024-09-19T11:26:26.962098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.962016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:19.643957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.852269+0000","last_change":"2024-09-19T11:26:53.852269+0000","last_active":"2024-09-19T11:26:53.852269+0000","last_peered":"2024-09-19T11:26:53.852269+0000","last_clean":"2024-09-19T11:26:53.852269+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:26:53.852269+0000","last_undegraded":"2024-09-19T11:26:53.852269+0000","last_fullsized":"2024-09-19T11:26:53.852269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.852186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:52.995416+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'33","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.158685+0000","last_change":"2024-09-19T11:23:09.262668+0000","last_active":"2024-09-19T11:26:19.158685+0000","last_peered":"2024-09-19T11:26:19.158685+0000","last_clean":"2024-09-19T11:26:19.158685+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:26:19.158685+0000","last_undegraded":"2024-09-19T11:26:19.158685+0000","last_fullsized":"2024-09-19T11:26:19.158685+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:42.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278485+0000","last_change":"2024-09-19T11:23:11.265649+0000","last_active":"2024-09-19T11:23:42.278485+0000","last_peered":"2024-09-19T11:23:42.278485+0000","last_clean":"2024-09-19T11:23:42.278485+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:23:42.278485+0000","last_undegraded":"2024-09-19T11:23:42.278485+0000","last_fullsized":"2024-09-19T11:23:42.278485+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:22.946739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'5","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.124970+0000","last_change":"2024-09-19T11:23:13.268845+0000","last_active":"2024-09-19T11:26:12.124970+0000","last_peered":"2024-09-19T11:26:12.124970+0000","last_clean":"2024-09-19T11:26:12.124970+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:26:12.124970+0000","last_undegraded":"2024-09-19T11:26:12.124970+0000","last_fullsized":"2024-09-19T11:26:12.124970+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:30.688522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.889489+0000","last_change":"2024-09-19T11:26:38.889489+0000","last_active":"2024-09-19T11:26:38.889489+0000","last_peered":"2024-09-19T11:26:38.889489+0000","last_clean":"2024-09-19T11:26:38.889489+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:26:38.889489+0000","last_undegraded":"2024-09-19T11:26:38.889489+0000","last_fullsized":"2024-09-19T11:26:38.889489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.889400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:14.595110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.412815+0000","last_change":"2024-09-19T11:26:36.412815+0000","last_active":"2024-09-19T11:26:36.412815+0000","last_peered":"2024-09-19T11:26:36.412815+0000","last_clean":"2024-09-19T11:26:36.412815+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:36.412815+0000","last_undegraded":"2024-09-19T11:26:36.412815+0000","last_fullsized":"2024-09-19T11:26:36.412815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.412703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:59.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.839126+0000","last_change":"2024-09-19T11:26:55.839126+0000","last_active":"2024-09-19T11:26:55.839126+0000","last_peered":"2024-09-19T11:26:55.839126+0000","last_clean":"2024-09-19T11:26:55.839126+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:26:55.839126+0000","last_undegraded":"2024-09-19T11:26:55.839126+0000","last_fullsized":"2024-09-19T11:26:55.839126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:08.800957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.222983+0000","last_change":"2024-09-19T11:23:09.259476+0000","last_active":"2024-09-19T11:26:19.222983+0000","last_peered":"2024-09-19T11:26:19.222983+0000","last_clean":"2024-09-19T11:26:19.222983+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:26:19.222983+0000","last_undegraded":"2024-09-19T11:26:19.222983+0000","last_fullsized":"2024-09-19T11:26:19.222983+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:21.364894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'285","reported_seq":352,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.079494+0000","last_change":"2024-09-19T11:23:11.267050+0000","last_active":"2024-09-19T11:26:19.079494+0000","last_peered":"2024-09-19T11:26:19.079494+0000","last_clean":"2024-09-19T11:26:19.079494+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:26:19.079494+0000","last_undegraded":"2024-09-19T11:26:19.079494+0000","last_fullsized":"2024-09-19T11:26:19.079494+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:55.330204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.334604+0000","last_change":"2024-09-19T11:23:13.269663+0000","last_active":"2024-09-19T11:26:12.334604+0000","last_peered":"2024-09-19T11:26:12.334604+0000","last_clean":"2024-09-19T11:26:12.334604+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:26:12.334604+0000","last_undegraded":"2024-09-19T11:26:12.334604+0000","last_fullsized":"2024-09-19T11:26:12.334604+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:57:11.742213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.810354+0000","last_change":"2024-09-19T11:26:40.810354+0000","last_active":"2024-09-19T11:26:40.810354+0000","last_peered":"2024-09-19T11:26:40.810354+0000","last_clean":"2024-09-19T11:26:40.810354+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:40.810354+0000","last_undegraded":"2024-09-19T11:26:40.810354+0000","last_fullsized":"2024-09-19T11:26:40.810354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.810272+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.328550+0000","last_change":"2024-09-19T11:26:34.328550+0000","last_active":"2024-09-19T11:26:34.328550+0000","last_peered":"2024-09-19T11:26:34.328550+0000","last_clean":"2024-09-19T11:26:34.328550+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:26:34.328550+0000","last_undegraded":"2024-09-19T11:26:34.328550+0000","last_fullsized":"2024-09-19T11:26:34.328550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.328463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:54.619678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.868748+0000","last_change":"2024-09-19T11:26:54.868748+0000","last_active":"2024-09-19T11:26:54.868748+0000","last_peered":"2024-09-19T11:26:54.868748+0000","last_clean":"2024-09-19T11:26:54.868748+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:26:54.868748+0000","last_undegraded":"2024-09-19T11:26:54.868748+0000","last_fullsized":"2024-09-19T11:26:54.868748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.868649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:17.358870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'61","reported_seq":126,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.167477+0000","last_change":"2024-09-19T11:23:09.259576+0000","last_active":"2024-09-19T11:26:19.167477+0000","last_peered":"2024-09-19T11:26:19.167477+0000","last_clean":"2024-09-19T11:26:19.167477+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:26:19.167477+0000","last_undegraded":"2024-09-19T11:26:19.167477+0000","last_fullsized":"2024-09-19T11:26:19.167477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.556001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278323+0000","last_change":"2024-09-19T11:23:11.260403+0000","last_active":"2024-09-19T11:23:42.278323+0000","last_peered":"2024-09-19T11:23:42.278323+0000","last_clean":"2024-09-19T11:23:42.278323+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:23:42.278323+0000","last_undegraded":"2024-09-19T11:23:42.278323+0000","last_fullsized":"2024-09-19T11:23:42.278323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:11.335945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'18","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.585256+0000","last_change":"2024-09-19T11:23:13.273720+0000","last_active":"2024-09-19T11:26:12.585256+0000","last_peered":"2024-09-19T11:26:12.585256+0000","last_clean":"2024-09-19T11:26:12.585256+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:26:12.585256+0000","last_undegraded":"2024-09-19T11:26:12.585256+0000","last_fullsized":"2024-09-19T11:26:12.585256+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:46.547393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":92,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.048324+0000","last_change":"2024-09-19T11:26:40.048324+0000","last_active":"2024-09-19T11:26:40.048324+0000","last_peered":"2024-09-19T11:26:40.048324+0000","last_clean":"2024-09-19T11:26:40.048324+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:26:40.048324+0000","last_undegraded":"2024-09-19T11:26:40.048324+0000","last_fullsized":"2024-09-19T11:26:40.048324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.048219+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:04:18.644192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.937533+0000","last_change":"2024-09-19T11:26:28.937533+0000","last_active":"2024-09-19T11:26:28.937533+0000","last_peered":"2024-09-19T11:26:28.937533+0000","last_clean":"2024-09-19T11:26:28.937533+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:28.937533+0000","last_undegraded":"2024-09-19T11:26:28.937533+0000","last_fullsized":"2024-09-19T11:26:28.937533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.937444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:51.513741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.828665+0000","last_change":"2024-09-19T11:26:52.828665+0000","last_active":"2024-09-19T11:26:52.828665+0000","last_peered":"2024-09-19T11:26:52.828665+0000","last_clean":"2024-09-19T11:26:52.828665+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:26:52.828665+0000","last_undegraded":"2024-09-19T11:26:52.828665+0000","last_fullsized":"2024-09-19T11:26:52.828665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.828574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'36","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.170832+0000","last_change":"2024-09-19T11:23:09.248190+0000","last_active":"2024-09-19T11:26:19.170832+0000","last_peered":"2024-09-19T11:26:19.170832+0000","last_clean":"2024-09-19T11:26:19.170832+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:26:19.170832+0000","last_undegraded":"2024-09-19T11:26:19.170832+0000","last_fullsized":"2024-09-19T11:26:19.170832+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:38.180379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946607+0000","last_change":"2024-09-19T11:23:11.270709+0000","last_active":"2024-09-19T11:23:42.946607+0000","last_peered":"2024-09-19T11:23:42.946607+0000","last_clean":"2024-09-19T11:23:42.946607+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:23:42.946607+0000","last_undegraded":"2024-09-19T11:23:42.946607+0000","last_fullsized":"2024-09-19T11:23:42.946607+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:39.157590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809883+0000","last_change":"2024-09-19T11:23:13.272542+0000","last_active":"2024-09-19T11:25:16.809883+0000","last_peered":"2024-09-19T11:25:16.809883+0000","last_clean":"2024-09-19T11:25:16.809883+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:25:16.809883+0000","last_undegraded":"2024-09-19T11:25:16.809883+0000","last_fullsized":"2024-09-19T11:25:16.809883+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:52.793325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":70,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.393655+0000","last_change":"2024-09-19T11:26:56.393655+0000","last_active":"2024-09-19T11:26:56.393655+0000","last_peered":"2024-09-19T11:26:56.393655+0000","last_clean":"2024-09-19T11:26:56.393655+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:26:56.393655+0000","last_undegraded":"2024-09-19T11:26:56.393655+0000","last_fullsized":"2024-09-19T11:26:56.393655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.393557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:06.327682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.308889+0000","last_change":"2024-09-19T11:26:33.308889+0000","last_active":"2024-09-19T11:26:33.308889+0000","last_peered":"2024-09-19T11:26:33.308889+0000","last_clean":"2024-09-19T11:26:33.308889+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:26:33.308889+0000","last_undegraded":"2024-09-19T11:26:33.308889+0000","last_fullsized":"2024-09-19T11:26:33.308889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.308801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:19.525046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.829806+0000","last_change":"2024-09-19T11:26:57.829806+0000","last_active":"2024-09-19T11:26:57.829806+0000","last_peered":"2024-09-19T11:26:57.829806+0000","last_clean":"2024-09-19T11:26:57.829806+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:26:57.829806+0000","last_undegraded":"2024-09-19T11:26:57.829806+0000","last_fullsized":"2024-09-19T11:26:57.829806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.829722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:47.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.148513+0000","last_change":"2024-09-19T11:23:09.246515+0000","last_active":"2024-09-19T11:26:19.148513+0000","last_peered":"2024-09-19T11:26:19.148513+0000","last_clean":"2024-09-19T11:26:19.148513+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:26:19.148513+0000","last_undegraded":"2024-09-19T11:26:19.148513+0000","last_fullsized":"2024-09-19T11:26:19.148513+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:01:54.969396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'285","reported_seq":407,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.074087+0000","last_change":"2024-09-19T11:23:11.263693+0000","last_active":"2024-09-19T11:26:19.074087+0000","last_peered":"2024-09-19T11:26:19.074087+0000","last_clean":"2024-09-19T11:26:19.074087+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:26:19.074087+0000","last_undegraded":"2024-09-19T11:26:19.074087+0000","last_fullsized":"2024-09-19T11:26:19.074087+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:29.980859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'14","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.475120+0000","last_change":"2024-09-19T11:23:13.265074+0000","last_active":"2024-09-19T11:26:12.475120+0000","last_peered":"2024-09-19T11:26:12.475120+0000","last_clean":"2024-09-19T11:26:12.475120+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:26:12.475120+0000","last_undegraded":"2024-09-19T11:26:12.475120+0000","last_fullsized":"2024-09-19T11:26:12.475120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:47.118156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.810833+0000","last_change":"2024-09-19T11:26:41.810833+0000","last_active":"2024-09-19T11:26:41.810833+0000","last_peered":"2024-09-19T11:26:41.810833+0000","last_clean":"2024-09-19T11:26:41.810833+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:41.810833+0000","last_undegraded":"2024-09-19T11:26:41.810833+0000","last_fullsized":"2024-09-19T11:26:41.810833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.810721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:31.441015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.268981+0000","last_change":"2024-09-19T11:26:31.268981+0000","last_active":"2024-09-19T11:26:31.268981+0000","last_peered":"2024-09-19T11:26:31.268981+0000","last_clean":"2024-09-19T11:26:31.268981+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:26:31.268981+0000","last_undegraded":"2024-09-19T11:26:31.268981+0000","last_fullsized":"2024-09-19T11:26:31.268981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.268893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:31.227733+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'5","reported_seq":428,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.945300+0000","last_change":"2024-09-19T11:26:58.945300+0000","last_active":"2024-09-19T11:26:58.945300+0000","last_peered":"2024-09-19T11:26:58.945300+0000","last_clean":"2024-09-19T11:26:58.945300+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:26:58.945300+0000","last_undegraded":"2024-09-19T11:26:58.945300+0000","last_fullsized":"2024-09-19T11:26:58.945300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.945214+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:11.607957+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946255+0000","last_change":"2024-09-19T11:23:09.255928+0000","last_active":"2024-09-19T11:23:42.946255+0000","last_peered":"2024-09-19T11:23:42.946255+0000","last_clean":"2024-09-19T11:23:42.946255+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:23:42.946255+0000","last_undegraded":"2024-09-19T11:23:42.946255+0000","last_fullsized":"2024-09-19T11:23:42.946255+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:04.689951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'285","reported_seq":469,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.075845+0000","last_change":"2024-09-19T11:23:11.255933+0000","last_active":"2024-09-19T11:26:19.075845+0000","last_peered":"2024-09-19T11:26:19.075845+0000","last_clean":"2024-09-19T11:26:19.075845+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:26:19.075845+0000","last_undegraded":"2024-09-19T11:26:19.075845+0000","last_fullsized":"2024-09-19T11:26:19.075845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:01.160503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.141728+0000","last_change":"2024-09-19T11:23:13.266957+0000","last_active":"2024-09-19T11:25:11.141728+0000","last_peered":"2024-09-19T11:25:11.141728+0000","last_clean":"2024-09-19T11:25:11.141728+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:25:11.141728+0000","last_undegraded":"2024-09-19T11:25:11.141728+0000","last_fullsized":"2024-09-19T11:25:11.141728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:59.867268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":109,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.028189+0000","last_change":"2024-09-19T11:26:42.028189+0000","last_active":"2024-09-19T11:26:42.028189+0000","last_peered":"2024-09-19T11:26:42.028189+0000","last_clean":"2024-09-19T11:26:42.028189+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:42.028189+0000","last_undegraded":"2024-09-19T11:26:42.028189+0000","last_fullsized":"2024-09-19T11:26:42.028189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'23","last_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_deep_scrub":"33'23","last_deep_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.028101+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.215685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.225653+0000","last_change":"2024-09-19T11:26:29.225653+0000","last_active":"2024-09-19T11:26:29.225653+0000","last_peered":"2024-09-19T11:26:29.225653+0000","last_clean":"2024-09-19T11:26:29.225653+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:26:29.225653+0000","last_undegraded":"2024-09-19T11:26:29.225653+0000","last_fullsized":"2024-09-19T11:26:29.225653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.225549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:47.310948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'5","reported_seq":420,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.055152+0000","last_change":"2024-09-19T11:23:07.250416+0000","last_active":"2024-09-19T11:26:19.055152+0000","last_peered":"2024-09-19T11:26:19.055152+0000","last_clean":"2024-09-19T11:26:19.055152+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:26:19.055152+0000","last_undegraded":"2024-09-19T11:26:19.055152+0000","last_fullsized":"2024-09-19T11:26:19.055152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:20.445331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":586,"num_read_kb":586,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'41","reported_seq":92,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.208185+0000","last_change":"2024-09-19T11:23:09.247733+0000","last_active":"2024-09-19T11:26:19.208185+0000","last_peered":"2024-09-19T11:26:19.208185+0000","last_clean":"2024-09-19T11:26:19.208185+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:26:19.208185+0000","last_undegraded":"2024-09-19T11:26:19.208185+0000","last_fullsized":"2024-09-19T11:26:19.208185+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:38.921761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279060+0000","last_change":"2024-09-19T11:23:11.266008+0000","last_active":"2024-09-19T11:23:42.279060+0000","last_peered":"2024-09-19T11:23:42.279060+0000","last_clean":"2024-09-19T11:23:42.279060+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:23:42.279060+0000","last_undegraded":"2024-09-19T11:23:42.279060+0000","last_fullsized":"2024-09-19T11:23:42.279060+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:38:43.871781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'6","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.426809+0000","last_change":"2024-09-19T11:23:13.263896+0000","last_active":"2024-09-19T11:26:12.426809+0000","last_peered":"2024-09-19T11:26:12.426809+0000","last_clean":"2024-09-19T11:26:12.426809+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:26:12.426809+0000","last_undegraded":"2024-09-19T11:26:12.426809+0000","last_fullsized":"2024-09-19T11:26:12.426809+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:35.364893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'8","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.369670+0000","last_change":"2024-09-19T11:26:52.369670+0000","last_active":"2024-09-19T11:26:52.369670+0000","last_peered":"2024-09-19T11:26:52.369670+0000","last_clean":"2024-09-19T11:26:52.369670+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:52.369670+0000","last_undegraded":"2024-09-19T11:26:52.369670+0000","last_fullsized":"2024-09-19T11:26:52.369670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.369580+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:43.275474+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"33'74","reported_seq":157,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.856490+0000","last_change":"2024-09-19T11:26:23.856409+0000","last_active":"2024-09-19T11:26:23.856490+0000","last_peered":"2024-09-19T11:26:23.856490+0000","last_clean":"2024-09-19T11:26:23.856490+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:26:23.856490+0000","last_undegraded":"2024-09-19T11:26:23.856490+0000","last_fullsized":"2024-09-19T11:26:23.856490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'74","last_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_deep_scrub":"33'74","last_deep_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.856290+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:02.254545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'2","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.142962+0000","last_change":"2024-09-19T11:26:27.142962+0000","last_active":"2024-09-19T11:26:27.142962+0000","last_peered":"2024-09-19T11:26:27.142962+0000","last_clean":"2024-09-19T11:26:27.142962+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:26:27.142962+0000","last_undegraded":"2024-09-19T11:26:27.142962+0000","last_fullsized":"2024-09-19T11:26:27.142962+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.142873+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.914383+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.273990+0000","last_change":"2024-09-19T11:26:30.273990+0000","last_active":"2024-09-19T11:26:30.273990+0000","last_peered":"2024-09-19T11:26:30.273990+0000","last_clean":"2024-09-19T11:26:30.273990+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:26:30.273990+0000","last_undegraded":"2024-09-19T11:26:30.273990+0000","last_fullsized":"2024-09-19T11:26:30.273990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.273902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:45.300652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.985644+0000","last_change":"2024-09-19T11:26:56.985644+0000","last_active":"2024-09-19T11:26:56.985644+0000","last_peered":"2024-09-19T11:26:56.985644+0000","last_clean":"2024-09-19T11:26:56.985644+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:26:56.985644+0000","last_undegraded":"2024-09-19T11:26:56.985644+0000","last_fullsized":"2024-09-19T11:26:56.985644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.985544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:47.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'85","reported_seq":167,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.214097+0000","last_change":"2024-09-19T11:23:09.260843+0000","last_active":"2024-09-19T11:26:19.214097+0000","last_peered":"2024-09-19T11:26:19.214097+0000","last_clean":"2024-09-19T11:26:19.214097+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:26:19.214097+0000","last_undegraded":"2024-09-19T11:26:19.214097+0000","last_fullsized":"2024-09-19T11:26:19.214097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:17.846685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280169+0000","last_change":"2024-09-19T11:23:11.266314+0000","last_active":"2024-09-19T11:23:42.280169+0000","last_peered":"2024-09-19T11:23:42.280169+0000","last_clean":"2024-09-19T11:23:42.280169+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:23:42.280169+0000","last_undegraded":"2024-09-19T11:23:42.280169+0000","last_fullsized":"2024-09-19T11:23:42.280169+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:28.832170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":163,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.377387+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:26:12.377387+0000","last_peered":"2024-09-19T11:26:12.377387+0000","last_clean":"2024-09-19T11:26:12.377387+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:26:12.377387+0000","last_undegraded":"2024-09-19T11:26:12.377387+0000","last_fullsized":"2024-09-19T11:26:12.377387+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:51.720881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.415515+0000","last_change":"2024-09-19T11:26:54.415515+0000","last_active":"2024-09-19T11:26:54.415515+0000","last_peered":"2024-09-19T11:26:54.415515+0000","last_clean":"2024-09-19T11:26:54.415515+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:26:54.415515+0000","last_undegraded":"2024-09-19T11:26:54.415515+0000","last_fullsized":"2024-09-19T11:26:54.415515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.415426+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:50.714030+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.079877+0000","last_change":"2024-09-19T11:26:24.079800+0000","last_active":"2024-09-19T11:26:24.079877+0000","last_peered":"2024-09-19T11:26:24.079877+0000","last_clean":"2024-09-19T11:26:24.079877+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:26:24.079877+0000","last_undegraded":"2024-09-19T11:26:24.079877+0000","last_fullsized":"2024-09-19T11:26:24.079877+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.079699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:48:18.789176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.923493+0000","last_change":"2024-09-19T11:26:25.923493+0000","last_active":"2024-09-19T11:26:25.923493+0000","last_peered":"2024-09-19T11:26:25.923493+0000","last_clean":"2024-09-19T11:26:25.923493+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:26:25.923493+0000","last_undegraded":"2024-09-19T11:26:25.923493+0000","last_fullsized":"2024-09-19T11:26:25.923493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.923407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:56.735995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.948430+0000","last_change":"2024-09-19T11:23:07.246124+0000","last_active":"2024-09-19T11:23:42.948430+0000","last_peered":"2024-09-19T11:23:42.948430+0000","last_clean":"2024-09-19T11:23:42.948430+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:23:42.948430+0000","last_undegraded":"2024-09-19T11:23:42.948430+0000","last_fullsized":"2024-09-19T11:23:42.948430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:52.791058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'284","reported_seq":501,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.212063+0000","last_change":"2024-09-19T11:23:09.257240+0000","last_active":"2024-09-19T11:26:19.212063+0000","last_peered":"2024-09-19T11:26:19.212063+0000","last_clean":"2024-09-19T11:26:19.212063+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:26:19.212063+0000","last_undegraded":"2024-09-19T11:26:19.212063+0000","last_fullsized":"2024-09-19T11:26:19.212063+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:14.514960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944824+0000","last_change":"2024-09-19T11:23:11.270807+0000","last_active":"2024-09-19T11:23:42.944824+0000","last_peered":"2024-09-19T11:23:42.944824+0000","last_clean":"2024-09-19T11:23:42.944824+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:23:42.944824+0000","last_undegraded":"2024-09-19T11:23:42.944824+0000","last_fullsized":"2024-09-19T11:23:42.944824+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:23.667174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.346118+0000","last_change":"2024-09-19T11:23:13.269927+0000","last_active":"2024-09-19T11:24:27.346118+0000","last_peered":"2024-09-19T11:24:27.346118+0000","last_clean":"2024-09-19T11:24:27.346118+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:24:27.346118+0000","last_undegraded":"2024-09-19T11:24:27.346118+0000","last_fullsized":"2024-09-19T11:24:27.346118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:59.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.967834+0000","last_change":"2024-09-19T11:26:39.967834+0000","last_active":"2024-09-19T11:26:39.967834+0000","last_peered":"2024-09-19T11:26:39.967834+0000","last_clean":"2024-09-19T11:26:39.967834+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:39.967834+0000","last_undegraded":"2024-09-19T11:26:39.967834+0000","last_fullsized":"2024-09-19T11:26:39.967834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.967748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:48.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.952524+0000","last_change":"2024-09-19T11:26:23.952524+0000","last_active":"2024-09-19T11:26:23.952524+0000","last_peered":"2024-09-19T11:26:23.952524+0000","last_clean":"2024-09-19T11:26:23.952524+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:26:23.952524+0000","last_undegraded":"2024-09-19T11:26:23.952524+0000","last_fullsized":"2024-09-19T11:26:23.952524+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.952436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:33.569870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.922822+0000","last_change":"2024-09-19T11:26:24.922822+0000","last_active":"2024-09-19T11:26:24.922822+0000","last_peered":"2024-09-19T11:26:24.922822+0000","last_clean":"2024-09-19T11:26:24.922822+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:26:24.922822+0000","last_undegraded":"2024-09-19T11:26:24.922822+0000","last_fullsized":"2024-09-19T11:26:24.922822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.922736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:47.473806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"33'2","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.833236+0000","last_change":"2024-09-19T11:26:53.833236+0000","last_active":"2024-09-19T11:26:53.833236+0000","last_peered":"2024-09-19T11:26:53.833236+0000","last_clean":"2024-09-19T11:26:53.833236+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:26:53.833236+0000","last_undegraded":"2024-09-19T11:26:53.833236+0000","last_fullsized":"2024-09-19T11:26:53.833236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.833148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:46.323076+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'16","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.327294+0000","last_change":"2024-09-19T11:23:09.249984+0000","last_active":"2024-09-19T11:26:12.327294+0000","last_peered":"2024-09-19T11:26:12.327294+0000","last_clean":"2024-09-19T11:26:12.327294+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:12.327294+0000","last_undegraded":"2024-09-19T11:26:12.327294+0000","last_fullsized":"2024-09-19T11:26:12.327294+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:47.233637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280391+0000","last_change":"2024-09-19T11:23:11.265633+0000","last_active":"2024-09-19T11:23:42.280391+0000","last_peered":"2024-09-19T11:23:42.280391+0000","last_clean":"2024-09-19T11:23:42.280391+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:23:42.280391+0000","last_undegraded":"2024-09-19T11:23:42.280391+0000","last_fullsized":"2024-09-19T11:23:42.280391+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:16.181121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943722+0000","last_change":"2024-09-19T11:23:13.273373+0000","last_active":"2024-09-19T11:23:42.943722+0000","last_peered":"2024-09-19T11:23:42.943722+0000","last_clean":"2024-09-19T11:23:42.943722+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:23:42.943722+0000","last_undegraded":"2024-09-19T11:23:42.943722+0000","last_fullsized":"2024-09-19T11:23:42.943722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:11:13.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.977150+0000","last_change":"2024-09-19T11:26:37.977150+0000","last_active":"2024-09-19T11:26:37.977150+0000","last_peered":"2024-09-19T11:26:37.977150+0000","last_clean":"2024-09-19T11:26:37.977150+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:37.977150+0000","last_undegraded":"2024-09-19T11:26:37.977150+0000","last_fullsized":"2024-09-19T11:26:37.977150+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.977048+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:20.097270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"33'3","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.109916+0000","last_change":"2024-09-19T11:26:25.109916+0000","last_active":"2024-09-19T11:26:25.109916+0000","last_peered":"2024-09-19T11:26:25.109916+0000","last_clean":"2024-09-19T11:26:25.109916+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:26:25.109916+0000","last_undegraded":"2024-09-19T11:26:25.109916+0000","last_fullsized":"2024-09-19T11:26:25.109916+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.109824+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:48.020763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.938045+0000","last_change":"2024-09-19T11:26:25.938045+0000","last_active":"2024-09-19T11:26:25.938045+0000","last_peered":"2024-09-19T11:26:25.938045+0000","last_clean":"2024-09-19T11:26:25.938045+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:25.938045+0000","last_undegraded":"2024-09-19T11:26:25.938045+0000","last_fullsized":"2024-09-19T11:26:25.938045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.937963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:58:57.848583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.801257+0000","last_change":"2024-09-19T11:26:56.801257+0000","last_active":"2024-09-19T11:26:56.801257+0000","last_peered":"2024-09-19T11:26:56.801257+0000","last_clean":"2024-09-19T11:26:56.801257+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:26:56.801257+0000","last_undegraded":"2024-09-19T11:26:56.801257+0000","last_fullsized":"2024-09-19T11:26:56.801257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.801170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:01.667240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'13","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.530475+0000","last_change":"2024-09-19T11:23:09.254499+0000","last_active":"2024-09-19T11:26:11.530475+0000","last_peered":"2024-09-19T11:26:11.530475+0000","last_clean":"2024-09-19T11:26:11.530475+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:26:11.530475+0000","last_undegraded":"2024-09-19T11:26:11.530475+0000","last_fullsized":"2024-09-19T11:26:11.530475+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:18.957107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947826+0000","last_change":"2024-09-19T11:23:11.271073+0000","last_active":"2024-09-19T11:23:42.947826+0000","last_peered":"2024-09-19T11:23:42.947826+0000","last_clean":"2024-09-19T11:23:42.947826+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:23:42.947826+0000","last_undegraded":"2024-09-19T11:23:42.947826+0000","last_fullsized":"2024-09-19T11:23:42.947826+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:52.079937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.088603+0000","last_change":"2024-09-19T11:23:13.273910+0000","last_active":"2024-09-19T11:25:11.088603+0000","last_peered":"2024-09-19T11:25:11.088603+0000","last_clean":"2024-09-19T11:25:11.088603+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:25:11.088603+0000","last_undegraded":"2024-09-19T11:25:11.088603+0000","last_fullsized":"2024-09-19T11:25:11.088603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:22:02.581618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.916064+0000","last_change":"2024-09-19T11:26:35.916064+0000","last_active":"2024-09-19T11:26:35.916064+0000","last_peered":"2024-09-19T11:26:35.916064+0000","last_clean":"2024-09-19T11:26:35.916064+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:35.916064+0000","last_undegraded":"2024-09-19T11:26:35.916064+0000","last_fullsized":"2024-09-19T11:26:35.916064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.915972+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:33.729135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.933741+0000","last_change":"2024-09-19T11:26:24.933741+0000","last_active":"2024-09-19T11:26:24.933741+0000","last_peered":"2024-09-19T11:26:24.933741+0000","last_clean":"2024-09-19T11:26:24.933741+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:26:24.933741+0000","last_undegraded":"2024-09-19T11:26:24.933741+0000","last_fullsized":"2024-09-19T11:26:24.933741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.933645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.620054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.910571+0000","last_change":"2024-09-19T11:26:22.910571+0000","last_active":"2024-09-19T11:26:22.910571+0000","last_peered":"2024-09-19T11:26:22.910571+0000","last_clean":"2024-09-19T11:26:22.910571+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:26:22.910571+0000","last_undegraded":"2024-09-19T11:26:22.910571+0000","last_fullsized":"2024-09-19T11:26:22.910571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.910497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:20.068073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.866167+0000","last_change":"2024-09-19T11:26:52.866167+0000","last_active":"2024-09-19T11:26:52.866167+0000","last_peered":"2024-09-19T11:26:52.866167+0000","last_clean":"2024-09-19T11:26:52.866167+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:26:52.866167+0000","last_undegraded":"2024-09-19T11:26:52.866167+0000","last_fullsized":"2024-09-19T11:26:52.866167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.866082+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:47.631328+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'77","reported_seq":153,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.785636+0000","last_change":"2024-09-19T11:26:58.785636+0000","last_active":"2024-09-19T11:26:58.785636+0000","last_peered":"2024-09-19T11:26:58.785636+0000","last_clean":"2024-09-19T11:26:58.785636+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:26:58.785636+0000","last_undegraded":"2024-09-19T11:26:58.785636+0000","last_fullsized":"2024-09-19T11:26:58.785636+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'77","last_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_deep_scrub":"33'77","last_deep_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.785526+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:40.190489+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277438+0000","last_change":"2024-09-19T11:23:11.268600+0000","last_active":"2024-09-19T11:23:42.277438+0000","last_peered":"2024-09-19T11:23:42.277438+0000","last_clean":"2024-09-19T11:23:42.277438+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:23:42.277438+0000","last_undegraded":"2024-09-19T11:23:42.277438+0000","last_fullsized":"2024-09-19T11:23:42.277438+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:50.540232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.422953+0000","last_change":"2024-09-19T11:23:13.267518+0000","last_active":"2024-09-19T11:25:11.422953+0000","last_peered":"2024-09-19T11:25:11.422953+0000","last_clean":"2024-09-19T11:25:11.422953+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:25:11.422953+0000","last_undegraded":"2024-09-19T11:25:11.422953+0000","last_fullsized":"2024-09-19T11:25:11.422953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:35.030653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.952752+0000","last_change":"2024-09-19T11:26:36.952752+0000","last_active":"2024-09-19T11:26:36.952752+0000","last_peered":"2024-09-19T11:26:36.952752+0000","last_clean":"2024-09-19T11:26:36.952752+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:36.952752+0000","last_undegraded":"2024-09-19T11:26:36.952752+0000","last_fullsized":"2024-09-19T11:26:36.952752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.952660+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.496448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.969511+0000","last_change":"2024-09-19T11:26:38.969511+0000","last_active":"2024-09-19T11:26:38.969511+0000","last_peered":"2024-09-19T11:26:38.969511+0000","last_clean":"2024-09-19T11:26:38.969511+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:26:38.969511+0000","last_undegraded":"2024-09-19T11:26:38.969511+0000","last_fullsized":"2024-09-19T11:26:38.969511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.969423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:49.370950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.293468+0000","last_change":"2024-09-19T11:26:32.293468+0000","last_active":"2024-09-19T11:26:32.293468+0000","last_peered":"2024-09-19T11:26:32.293468+0000","last_clean":"2024-09-19T11:26:32.293468+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:26:32.293468+0000","last_undegraded":"2024-09-19T11:26:32.293468+0000","last_fullsized":"2024-09-19T11:26:32.293468+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.293378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:48.996006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943947+0000","last_change":"2024-09-19T11:23:07.243184+0000","last_active":"2024-09-19T11:23:42.943947+0000","last_peered":"2024-09-19T11:23:42.943947+0000","last_clean":"2024-09-19T11:23:42.943947+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:23:42.943947+0000","last_undegraded":"2024-09-19T11:23:42.943947+0000","last_fullsized":"2024-09-19T11:23:42.943947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:32:59.314535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'15","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.809866+0000","last_change":"2024-09-19T11:23:09.261039+0000","last_active":"2024-09-19T11:25:16.809866+0000","last_peered":"2024-09-19T11:25:16.809866+0000","last_clean":"2024-09-19T11:25:16.809866+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:25:16.809866+0000","last_undegraded":"2024-09-19T11:25:16.809866+0000","last_fullsized":"2024-09-19T11:25:16.809866+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:42.183684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'285","reported_seq":359,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.071730+0000","last_change":"2024-09-19T11:23:11.263125+0000","last_active":"2024-09-19T11:26:19.071730+0000","last_peered":"2024-09-19T11:26:19.071730+0000","last_clean":"2024-09-19T11:26:19.071730+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:26:19.071730+0000","last_undegraded":"2024-09-19T11:26:19.071730+0000","last_fullsized":"2024-09-19T11:26:19.071730+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:10.328770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946621+0000","last_change":"2024-09-19T11:23:13.269759+0000","last_active":"2024-09-19T11:23:42.946621+0000","last_peered":"2024-09-19T11:23:42.946621+0000","last_clean":"2024-09-19T11:23:42.946621+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:23:42.946621+0000","last_undegraded":"2024-09-19T11:23:42.946621+0000","last_fullsized":"2024-09-19T11:23:42.946621+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:46.820239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'12","reported_seq":78,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.381726+0000","last_change":"2024-09-19T11:26:53.381726+0000","last_active":"2024-09-19T11:26:53.381726+0000","last_peered":"2024-09-19T11:26:53.381726+0000","last_clean":"2024-09-19T11:26:53.381726+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:53.381726+0000","last_undegraded":"2024-09-19T11:26:53.381726+0000","last_fullsized":"2024-09-19T11:26:53.381726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.381618+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:56.071102+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.185387+0000","last_change":"2024-09-19T11:26:28.185387+0000","last_active":"2024-09-19T11:26:28.185387+0000","last_peered":"2024-09-19T11:26:28.185387+0000","last_clean":"2024-09-19T11:26:28.185387+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:26:28.185387+0000","last_undegraded":"2024-09-19T11:26:28.185387+0000","last_fullsized":"2024-09-19T11:26:28.185387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.185291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:48.217740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.897455+0000","last_change":"2024-09-19T11:26:23.897455+0000","last_active":"2024-09-19T11:26:23.897455+0000","last_peered":"2024-09-19T11:26:23.897455+0000","last_clean":"2024-09-19T11:26:23.897455+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:26:23.897455+0000","last_undegraded":"2024-09-19T11:26:23.897455+0000","last_fullsized":"2024-09-19T11:26:23.897455+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.897373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:13.527667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946584+0000","last_change":"2024-09-19T11:23:07.245138+0000","last_active":"2024-09-19T11:23:42.946584+0000","last_peered":"2024-09-19T11:23:42.946584+0000","last_clean":"2024-09-19T11:23:42.946584+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:23:42.946584+0000","last_undegraded":"2024-09-19T11:23:42.946584+0000","last_fullsized":"2024-09-19T11:23:42.946584+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:33:31.680535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"31'11","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945354+0000","last_change":"2024-09-19T11:23:09.256698+0000","last_active":"2024-09-19T11:23:42.945354+0000","last_peered":"2024-09-19T11:23:42.945354+0000","last_clean":"2024-09-19T11:23:42.945354+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:23:42.945354+0000","last_undegraded":"2024-09-19T11:23:42.945354+0000","last_fullsized":"2024-09-19T11:23:42.945354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:21.852554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'285","reported_seq":433,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.076916+0000","last_change":"2024-09-19T11:23:11.258279+0000","last_active":"2024-09-19T11:26:19.076916+0000","last_peered":"2024-09-19T11:26:19.076916+0000","last_clean":"2024-09-19T11:26:19.076916+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:26:19.076916+0000","last_undegraded":"2024-09-19T11:26:19.076916+0000","last_fullsized":"2024-09-19T11:26:19.076916+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:24.889400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281484+0000","last_change":"2024-09-19T11:23:13.274470+0000","last_active":"2024-09-19T11:23:42.281484+0000","last_peered":"2024-09-19T11:23:42.281484+0000","last_clean":"2024-09-19T11:23:42.281484+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:23:42.281484+0000","last_undegraded":"2024-09-19T11:23:42.281484+0000","last_fullsized":"2024-09-19T11:23:42.281484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:33.500152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.850188+0000","last_change":"2024-09-19T11:26:39.850188+0000","last_active":"2024-09-19T11:26:39.850188+0000","last_peered":"2024-09-19T11:26:39.850188+0000","last_clean":"2024-09-19T11:26:39.850188+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:26:39.850188+0000","last_undegraded":"2024-09-19T11:26:39.850188+0000","last_fullsized":"2024-09-19T11:26:39.850188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.850077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:05.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.886518+0000","last_change":"2024-09-19T11:26:22.886518+0000","last_active":"2024-09-19T11:26:22.886518+0000","last_peered":"2024-09-19T11:26:22.886518+0000","last_clean":"2024-09-19T11:26:22.886518+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:26:22.886518+0000","last_undegraded":"2024-09-19T11:26:22.886518+0000","last_fullsized":"2024-09-19T11:26:22.886518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.886419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:07.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.146468+0000","last_change":"2024-09-19T11:26:26.146468+0000","last_active":"2024-09-19T11:26:26.146468+0000","last_peered":"2024-09-19T11:26:26.146468+0000","last_clean":"2024-09-19T11:26:26.146468+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:26:26.146468+0000","last_undegraded":"2024-09-19T11:26:26.146468+0000","last_fullsized":"2024-09-19T11:26:26.146468+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.146376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:37.768766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945462+0000","last_change":"2024-09-19T11:23:07.248324+0000","last_active":"2024-09-19T11:23:42.945462+0000","last_peered":"2024-09-19T11:23:42.945462+0000","last_clean":"2024-09-19T11:23:42.945462+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:23:42.945462+0000","last_undegraded":"2024-09-19T11:23:42.945462+0000","last_fullsized":"2024-09-19T11:23:42.945462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.767188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'99","reported_seq":192,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.848423+0000","last_change":"2024-09-19T11:26:58.848423+0000","last_active":"2024-09-19T11:26:58.848423+0000","last_peered":"2024-09-19T11:26:58.848423+0000","last_clean":"2024-09-19T11:26:58.848423+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:26:58.848423+0000","last_undegraded":"2024-09-19T11:26:58.848423+0000","last_fullsized":"2024-09-19T11:26:58.848423+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'99","last_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_deep_scrub":"33'99","last_deep_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.848330+0000","objects_scrubbed":8,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:48.091262+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280109+0000","last_change":"2024-09-19T11:23:11.263614+0000","last_active":"2024-09-19T11:23:42.280109+0000","last_peered":"2024-09-19T11:23:42.280109+0000","last_clean":"2024-09-19T11:23:42.280109+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:23:42.280109+0000","last_undegraded":"2024-09-19T11:23:42.280109+0000","last_fullsized":"2024-09-19T11:23:42.280109+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:55.842570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278855+0000","last_change":"2024-09-19T11:23:13.265949+0000","last_active":"2024-09-19T11:23:42.278855+0000","last_peered":"2024-09-19T11:23:42.278855+0000","last_clean":"2024-09-19T11:23:42.278855+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:23:42.278855+0000","last_undegraded":"2024-09-19T11:23:42.278855+0000","last_fullsized":"2024-09-19T11:23:42.278855+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:37.104486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278953+0000","last_change":"2024-09-19T11:23:13.271066+0000","last_active":"2024-09-19T11:23:42.278953+0000","last_peered":"2024-09-19T11:23:42.278953+0000","last_clean":"2024-09-19T11:23:42.278953+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:23:42.278953+0000","last_undegraded":"2024-09-19T11:23:42.278953+0000","last_fullsized":"2024-09-19T11:23:42.278953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:34.307629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.771683+0000","last_change":"2024-09-19T11:26:42.771683+0000","last_active":"2024-09-19T11:26:42.771683+0000","last_peered":"2024-09-19T11:26:42.771683+0000","last_clean":"2024-09-19T11:26:42.771683+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:26:42.771683+0000","last_undegraded":"2024-09-19T11:26:42.771683+0000","last_fullsized":"2024-09-19T11:26:42.771683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.771596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:48:05.322666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.396108+0000","last_change":"2024-09-19T11:26:38.396108+0000","last_active":"2024-09-19T11:26:38.396108+0000","last_peered":"2024-09-19T11:26:38.396108+0000","last_clean":"2024-09-19T11:26:38.396108+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:26:38.396108+0000","last_undegraded":"2024-09-19T11:26:38.396108+0000","last_fullsized":"2024-09-19T11:26:38.396108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.396006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:14.129195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.922151+0000","last_change":"2024-09-19T11:27:00.922151+0000","last_active":"2024-09-19T11:27:00.922151+0000","last_peered":"2024-09-19T11:27:00.922151+0000","last_clean":"2024-09-19T11:27:00.922151+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:27:00.922151+0000","last_undegraded":"2024-09-19T11:27:00.922151+0000","last_fullsized":"2024-09-19T11:27:00.922151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.922068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:51.324340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'35","reported_seq":83,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.220186+0000","last_change":"2024-09-19T11:23:09.259148+0000","last_active":"2024-09-19T11:26:19.220186+0000","last_peered":"2024-09-19T11:26:19.220186+0000","last_clean":"2024-09-19T11:26:19.220186+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:26:19.220186+0000","last_undegraded":"2024-09-19T11:26:19.220186+0000","last_fullsized":"2024-09-19T11:26:19.220186+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:29.000573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945264+0000","last_change":"2024-09-19T11:23:11.271128+0000","last_active":"2024-09-19T11:23:42.945264+0000","last_peered":"2024-09-19T11:23:42.945264+0000","last_clean":"2024-09-19T11:23:42.945264+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:23:42.945264+0000","last_undegraded":"2024-09-19T11:23:42.945264+0000","last_fullsized":"2024-09-19T11:23:42.945264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:35.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.427118+0000","last_change":"2024-09-19T11:23:13.273576+0000","last_active":"2024-09-19T11:25:09.427118+0000","last_peered":"2024-09-19T11:25:09.427118+0000","last_clean":"2024-09-19T11:25:09.427118+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:25:09.427118+0000","last_undegraded":"2024-09-19T11:25:09.427118+0000","last_fullsized":"2024-09-19T11:25:09.427118+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:43.417746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.036281+0000","last_change":"2024-09-19T11:26:41.036281+0000","last_active":"2024-09-19T11:26:41.036281+0000","last_peered":"2024-09-19T11:26:41.036281+0000","last_clean":"2024-09-19T11:26:41.036281+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:26:41.036281+0000","last_undegraded":"2024-09-19T11:26:41.036281+0000","last_fullsized":"2024-09-19T11:26:41.036281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.036169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:34.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.921998+0000","last_change":"2024-09-19T11:26:28.921998+0000","last_active":"2024-09-19T11:26:28.921998+0000","last_peered":"2024-09-19T11:26:28.921998+0000","last_clean":"2024-09-19T11:26:28.921998+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:26:28.921998+0000","last_undegraded":"2024-09-19T11:26:28.921998+0000","last_fullsized":"2024-09-19T11:26:28.921998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.921906+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:00.637589+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945177+0000","last_change":"2024-09-19T11:23:07.248908+0000","last_active":"2024-09-19T11:23:42.945177+0000","last_peered":"2024-09-19T11:23:42.945177+0000","last_clean":"2024-09-19T11:23:42.945177+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:23:42.945177+0000","last_undegraded":"2024-09-19T11:23:42.945177+0000","last_fullsized":"2024-09-19T11:23:42.945177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:57.835222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'5","reported_seq":37,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.535761+0000","last_change":"2024-09-19T11:23:09.244959+0000","last_active":"2024-09-19T11:26:11.535761+0000","last_peered":"2024-09-19T11:26:11.535761+0000","last_clean":"2024-09-19T11:26:11.535761+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:26:11.535761+0000","last_undegraded":"2024-09-19T11:26:11.535761+0000","last_fullsized":"2024-09-19T11:26:11.535761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:13.063964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277093+0000","last_change":"2024-09-19T11:23:11.268038+0000","last_active":"2024-09-19T11:23:42.277093+0000","last_peered":"2024-09-19T11:23:42.277093+0000","last_clean":"2024-09-19T11:23:42.277093+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:23:42.277093+0000","last_undegraded":"2024-09-19T11:23:42.277093+0000","last_fullsized":"2024-09-19T11:23:42.277093+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:40:40.232592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279462+0000","last_change":"2024-09-19T11:23:13.267238+0000","last_active":"2024-09-19T11:23:42.279462+0000","last_peered":"2024-09-19T11:23:42.279462+0000","last_clean":"2024-09-19T11:23:42.279462+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:23:42.279462+0000","last_undegraded":"2024-09-19T11:23:42.279462+0000","last_fullsized":"2024-09-19T11:23:42.279462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:36.252981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.001963+0000","last_change":"2024-09-19T11:26:41.001963+0000","last_active":"2024-09-19T11:26:41.001963+0000","last_peered":"2024-09-19T11:26:41.001963+0000","last_clean":"2024-09-19T11:26:41.001963+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:41.001963+0000","last_undegraded":"2024-09-19T11:26:41.001963+0000","last_fullsized":"2024-09-19T11:26:41.001963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.001858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:06.599582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.937976+0000","last_change":"2024-09-19T11:26:29.937976+0000","last_active":"2024-09-19T11:26:29.937976+0000","last_peered":"2024-09-19T11:26:29.937976+0000","last_clean":"2024-09-19T11:26:29.937976+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:29.937976+0000","last_undegraded":"2024-09-19T11:26:29.937976+0000","last_fullsized":"2024-09-19T11:26:29.937976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.937882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:14.058304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.813274+0000","last_change":"2024-09-19T11:26:51.813274+0000","last_active":"2024-09-19T11:26:51.813274+0000","last_peered":"2024-09-19T11:26:51.813274+0000","last_clean":"2024-09-19T11:26:51.813274+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:26:51.813274+0000","last_undegraded":"2024-09-19T11:26:51.813274+0000","last_fullsized":"2024-09-19T11:26:51.813274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.813187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:32.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'49","reported_seq":121,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.759116+0000","last_change":"2024-09-19T11:27:00.759116+0000","last_active":"2024-09-19T11:27:00.759116+0000","last_peered":"2024-09-19T11:27:00.759116+0000","last_clean":"2024-09-19T11:27:00.759116+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:27:00.759116+0000","last_undegraded":"2024-09-19T11:27:00.759116+0000","last_fullsized":"2024-09-19T11:27:00.759116+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'49","last_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_deep_scrub":"33'49","last_deep_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.759010+0000","objects_scrubbed":6,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:16.347162+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280357+0000","last_change":"2024-09-19T11:23:11.262690+0000","last_active":"2024-09-19T11:23:42.280357+0000","last_peered":"2024-09-19T11:23:42.280357+0000","last_clean":"2024-09-19T11:23:42.280357+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:23:42.280357+0000","last_undegraded":"2024-09-19T11:23:42.280357+0000","last_fullsized":"2024-09-19T11:23:42.280357+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:31:55.778119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.392197+0000","last_change":"2024-09-19T11:26:58.392197+0000","last_active":"2024-09-19T11:26:58.392197+0000","last_peered":"2024-09-19T11:26:58.392197+0000","last_clean":"2024-09-19T11:26:58.392197+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:26:58.392197+0000","last_undegraded":"2024-09-19T11:26:58.392197+0000","last_fullsized":"2024-09-19T11:26:58.392197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.392102+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:10.174526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.886507+0000","last_change":"2024-09-19T11:26:25.886507+0000","last_active":"2024-09-19T11:26:25.886507+0000","last_peered":"2024-09-19T11:26:25.886507+0000","last_clean":"2024-09-19T11:26:25.886507+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:26:25.886507+0000","last_undegraded":"2024-09-19T11:26:25.886507+0000","last_fullsized":"2024-09-19T11:26:25.886507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.886414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:38.573945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944123+0000","last_change":"2024-09-19T11:23:07.245862+0000","last_active":"2024-09-19T11:23:42.944123+0000","last_peered":"2024-09-19T11:23:42.944123+0000","last_clean":"2024-09-19T11:23:42.944123+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:23:42.944123+0000","last_undegraded":"2024-09-19T11:23:42.944123+0000","last_fullsized":"2024-09-19T11:23:42.944123+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:09.692396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.177030+0000","last_change":"2024-09-19T11:23:09.247041+0000","last_active":"2024-09-19T11:26:19.177030+0000","last_peered":"2024-09-19T11:26:19.177030+0000","last_clean":"2024-09-19T11:26:19.177030+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:26:19.177030+0000","last_undegraded":"2024-09-19T11:26:19.177030+0000","last_fullsized":"2024-09-19T11:26:19.177030+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:21.727511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.277107+0000","last_change":"2024-09-19T11:23:11.253755+0000","last_active":"2024-09-19T11:23:42.277107+0000","last_peered":"2024-09-19T11:23:42.277107+0000","last_clean":"2024-09-19T11:23:42.277107+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:23:42.277107+0000","last_undegraded":"2024-09-19T11:23:42.277107+0000","last_fullsized":"2024-09-19T11:23:42.277107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:24:51.209331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.486612+0000","last_change":"2024-09-19T11:27:01.486612+0000","last_active":"2024-09-19T11:27:01.486612+0000","last_peered":"2024-09-19T11:27:01.486612+0000","last_clean":"2024-09-19T11:27:01.486612+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:27:01.486612+0000","last_undegraded":"2024-09-19T11:27:01.486612+0000","last_fullsized":"2024-09-19T11:27:01.486612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.486507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:11.419750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'4","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.386729+0000","last_change":"2024-09-19T11:26:39.386729+0000","last_active":"2024-09-19T11:26:39.386729+0000","last_peered":"2024-09-19T11:26:39.386729+0000","last_clean":"2024-09-19T11:26:39.386729+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:26:39.386729+0000","last_undegraded":"2024-09-19T11:26:39.386729+0000","last_fullsized":"2024-09-19T11:26:39.386729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.386636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:26.742196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.840003+0000","last_change":"2024-09-19T11:26:55.840003+0000","last_active":"2024-09-19T11:26:55.840003+0000","last_peered":"2024-09-19T11:26:55.840003+0000","last_clean":"2024-09-19T11:26:55.840003+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:26:55.840003+0000","last_undegraded":"2024-09-19T11:26:55.840003+0000","last_fullsized":"2024-09-19T11:26:55.840003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.241449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'62","reported_seq":127,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.174069+0000","last_change":"2024-09-19T11:23:09.246587+0000","last_active":"2024-09-19T11:26:19.174069+0000","last_peered":"2024-09-19T11:26:19.174069+0000","last_clean":"2024-09-19T11:26:19.174069+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:26:19.174069+0000","last_undegraded":"2024-09-19T11:26:19.174069+0000","last_fullsized":"2024-09-19T11:26:19.174069+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:27:53.040162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":65,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278167+0000","last_change":"2024-09-19T11:23:11.260449+0000","last_active":"2024-09-19T11:23:42.278167+0000","last_peered":"2024-09-19T11:23:42.278167+0000","last_clean":"2024-09-19T11:23:42.278167+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:23:42.278167+0000","last_undegraded":"2024-09-19T11:23:42.278167+0000","last_fullsized":"2024-09-19T11:23:42.278167+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:30.983027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.419187+0000","last_change":"2024-09-19T11:23:13.268105+0000","last_active":"2024-09-19T11:25:11.419187+0000","last_peered":"2024-09-19T11:25:11.419187+0000","last_clean":"2024-09-19T11:25:11.419187+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:25:11.419187+0000","last_undegraded":"2024-09-19T11:25:11.419187+0000","last_fullsized":"2024-09-19T11:25:11.419187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:21.113571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":79,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.954113+0000","last_change":"2024-09-19T11:26:43.954113+0000","last_active":"2024-09-19T11:26:43.954113+0000","last_peered":"2024-09-19T11:26:43.954113+0000","last_clean":"2024-09-19T11:26:43.954113+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:43.954113+0000","last_undegraded":"2024-09-19T11:26:43.954113+0000","last_fullsized":"2024-09-19T11:26:43.954113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.954028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:24.870979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.929509+0000","last_change":"2024-09-19T11:26:26.929509+0000","last_active":"2024-09-19T11:26:26.929509+0000","last_peered":"2024-09-19T11:26:26.929509+0000","last_clean":"2024-09-19T11:26:26.929509+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:26:26.929509+0000","last_undegraded":"2024-09-19T11:26:26.929509+0000","last_fullsized":"2024-09-19T11:26:26.929509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.929422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:53.997533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279359+0000","last_change":"2024-09-19T11:23:07.243165+0000","last_active":"2024-09-19T11:23:42.279359+0000","last_peered":"2024-09-19T11:23:42.279359+0000","last_clean":"2024-09-19T11:23:42.279359+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:23:42.279359+0000","last_undegraded":"2024-09-19T11:23:42.279359+0000","last_fullsized":"2024-09-19T11:23:42.279359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:21.938032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.179930+0000","last_change":"2024-09-19T11:23:09.256642+0000","last_active":"2024-09-19T11:26:19.179930+0000","last_peered":"2024-09-19T11:26:19.179930+0000","last_clean":"2024-09-19T11:26:19.179930+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:26:19.179930+0000","last_undegraded":"2024-09-19T11:26:19.179930+0000","last_fullsized":"2024-09-19T11:26:19.179930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:42.797325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.282300+0000","last_change":"2024-09-19T11:23:11.266817+0000","last_active":"2024-09-19T11:23:42.282300+0000","last_peered":"2024-09-19T11:23:42.282300+0000","last_clean":"2024-09-19T11:23:42.282300+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:23:42.282300+0000","last_undegraded":"2024-09-19T11:23:42.282300+0000","last_fullsized":"2024-09-19T11:23:42.282300+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:41:28.972274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.474660+0000","last_change":"2024-09-19T11:27:00.474660+0000","last_active":"2024-09-19T11:27:00.474660+0000","last_peered":"2024-09-19T11:27:00.474660+0000","last_clean":"2024-09-19T11:27:00.474660+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:27:00.474660+0000","last_undegraded":"2024-09-19T11:27:00.474660+0000","last_fullsized":"2024-09-19T11:27:00.474660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.474573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:59.322724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.877728+0000","last_change":"2024-09-19T11:26:28.877728+0000","last_active":"2024-09-19T11:26:28.877728+0000","last_peered":"2024-09-19T11:26:28.877728+0000","last_clean":"2024-09-19T11:26:28.877728+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:26:28.877728+0000","last_undegraded":"2024-09-19T11:26:28.877728+0000","last_fullsized":"2024-09-19T11:26:28.877728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.877640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:09.086406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.801547+0000","last_change":"2024-09-19T11:26:57.801547+0000","last_active":"2024-09-19T11:26:57.801547+0000","last_peered":"2024-09-19T11:26:57.801547+0000","last_clean":"2024-09-19T11:26:57.801547+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:26:57.801547+0000","last_undegraded":"2024-09-19T11:26:57.801547+0000","last_fullsized":"2024-09-19T11:26:57.801547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.801457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:23.399181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'37","reported_seq":85,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.126834+0000","last_change":"2024-09-19T11:23:09.258190+0000","last_active":"2024-09-19T11:26:19.126834+0000","last_peered":"2024-09-19T11:26:19.126834+0000","last_clean":"2024-09-19T11:26:19.126834+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:26:19.126834+0000","last_undegraded":"2024-09-19T11:26:19.126834+0000","last_fullsized":"2024-09-19T11:26:19.126834+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:31.989630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.280329+0000","last_change":"2024-09-19T11:23:11.261698+0000","last_active":"2024-09-19T11:23:42.280329+0000","last_peered":"2024-09-19T11:23:42.280329+0000","last_clean":"2024-09-19T11:23:42.280329+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:23:42.280329+0000","last_undegraded":"2024-09-19T11:23:42.280329+0000","last_fullsized":"2024-09-19T11:23:42.280329+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:48.537146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.000020+0000","last_change":"2024-09-19T11:26:43.000020+0000","last_active":"2024-09-19T11:26:43.000020+0000","last_peered":"2024-09-19T11:26:43.000020+0000","last_clean":"2024-09-19T11:26:43.000020+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:26:43.000020+0000","last_undegraded":"2024-09-19T11:26:43.000020+0000","last_fullsized":"2024-09-19T11:26:43.000020+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.999915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:23.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.342895+0000","last_change":"2024-09-19T11:26:40.342895+0000","last_active":"2024-09-19T11:26:40.342895+0000","last_peered":"2024-09-19T11:26:40.342895+0000","last_clean":"2024-09-19T11:26:40.342895+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:26:40.342895+0000","last_undegraded":"2024-09-19T11:26:40.342895+0000","last_fullsized":"2024-09-19T11:26:40.342895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.342808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.126397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.286788+0000","last_change":"2024-09-19T11:23:42.286200+0000","last_active":"2024-09-19T11:23:42.286788+0000","last_peered":"2024-09-19T11:23:42.286788+0000","last_clean":"2024-09-19T11:23:42.286788+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:23:42.286788+0000","last_undegraded":"2024-09-19T11:23:42.286788+0000","last_fullsized":"2024-09-19T11:23:42.286788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.397616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946537+0000","last_change":"2024-09-19T11:23:09.257194+0000","last_active":"2024-09-19T11:23:42.946537+0000","last_peered":"2024-09-19T11:23:42.946537+0000","last_clean":"2024-09-19T11:23:42.946537+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:23:42.946537+0000","last_undegraded":"2024-09-19T11:23:42.946537+0000","last_fullsized":"2024-09-19T11:23:42.946537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:10.690183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'285","reported_seq":421,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.070038+0000","last_change":"2024-09-19T11:23:11.268055+0000","last_active":"2024-09-19T11:26:19.070038+0000","last_peered":"2024-09-19T11:26:19.070038+0000","last_clean":"2024-09-19T11:26:19.070038+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:26:19.070038+0000","last_undegraded":"2024-09-19T11:26:19.070038+0000","last_fullsized":"2024-09-19T11:26:19.070038+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:33.997140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943313+0000","last_change":"2024-09-19T11:23:13.272264+0000","last_active":"2024-09-19T11:23:42.943313+0000","last_peered":"2024-09-19T11:23:42.943313+0000","last_clean":"2024-09-19T11:23:42.943313+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:23:42.943313+0000","last_undegraded":"2024-09-19T11:23:42.943313+0000","last_fullsized":"2024-09-19T11:23:42.943313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:34.821219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.773400+0000","last_change":"2024-09-19T11:26:47.773400+0000","last_active":"2024-09-19T11:26:47.773400+0000","last_peered":"2024-09-19T11:26:47.773400+0000","last_clean":"2024-09-19T11:26:47.773400+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:47.773400+0000","last_undegraded":"2024-09-19T11:26:47.773400+0000","last_fullsized":"2024-09-19T11:26:47.773400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.773301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:48.002135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.907839+0000","last_change":"2024-09-19T11:26:27.907839+0000","last_active":"2024-09-19T11:26:27.907839+0000","last_peered":"2024-09-19T11:26:27.907839+0000","last_clean":"2024-09-19T11:26:27.907839+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:26:27.907839+0000","last_undegraded":"2024-09-19T11:26:27.907839+0000","last_fullsized":"2024-09-19T11:26:27.907839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.907726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:48.160182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.281352+0000","last_change":"2024-09-19T11:23:07.247890+0000","last_active":"2024-09-19T11:23:42.281352+0000","last_peered":"2024-09-19T11:23:42.281352+0000","last_clean":"2024-09-19T11:23:42.281352+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:23:42.281352+0000","last_undegraded":"2024-09-19T11:23:42.281352+0000","last_fullsized":"2024-09-19T11:23:42.281352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:24.412033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.151902+0000","last_change":"2024-09-19T11:23:09.246755+0000","last_active":"2024-09-19T11:26:19.151902+0000","last_peered":"2024-09-19T11:26:19.151902+0000","last_clean":"2024-09-19T11:26:19.151902+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:26:19.151902+0000","last_undegraded":"2024-09-19T11:26:19.151902+0000","last_fullsized":"2024-09-19T11:26:19.151902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.229678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278129+0000","last_change":"2024-09-19T11:23:11.266713+0000","last_active":"2024-09-19T11:23:42.278129+0000","last_peered":"2024-09-19T11:23:42.278129+0000","last_clean":"2024-09-19T11:23:42.278129+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:23:42.278129+0000","last_undegraded":"2024-09-19T11:23:42.278129+0000","last_fullsized":"2024-09-19T11:23:42.278129+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:38.162577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283516+0000","last_change":"2024-09-19T11:23:13.268771+0000","last_active":"2024-09-19T11:23:42.283516+0000","last_peered":"2024-09-19T11:23:42.283516+0000","last_clean":"2024-09-19T11:23:42.283516+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:23:42.283516+0000","last_undegraded":"2024-09-19T11:23:42.283516+0000","last_fullsized":"2024-09-19T11:23:42.283516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:19.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.724840+0000","last_change":"2024-09-19T11:26:46.724840+0000","last_active":"2024-09-19T11:26:46.724840+0000","last_peered":"2024-09-19T11:26:46.724840+0000","last_clean":"2024-09-19T11:26:46.724840+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:46.724840+0000","last_undegraded":"2024-09-19T11:26:46.724840+0000","last_fullsized":"2024-09-19T11:26:46.724840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.724740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:47.953525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.382940+0000","last_change":"2024-09-19T11:26:41.382940+0000","last_active":"2024-09-19T11:26:41.382940+0000","last_peered":"2024-09-19T11:26:41.382940+0000","last_clean":"2024-09-19T11:26:41.382940+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:26:41.382940+0000","last_undegraded":"2024-09-19T11:26:41.382940+0000","last_fullsized":"2024-09-19T11:26:41.382940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.382839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:55.560428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.798773+0000","last_change":"2024-09-19T11:26:56.798773+0000","last_active":"2024-09-19T11:26:56.798773+0000","last_peered":"2024-09-19T11:26:56.798773+0000","last_clean":"2024-09-19T11:26:56.798773+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:26:56.798773+0000","last_undegraded":"2024-09-19T11:26:56.798773+0000","last_fullsized":"2024-09-19T11:26:56.798773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.798674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:05.764652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.155681+0000","last_change":"2024-09-19T11:23:09.257850+0000","last_active":"2024-09-19T11:26:19.155681+0000","last_peered":"2024-09-19T11:26:19.155681+0000","last_clean":"2024-09-19T11:26:19.155681+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:26:19.155681+0000","last_undegraded":"2024-09-19T11:26:19.155681+0000","last_fullsized":"2024-09-19T11:26:19.155681+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:48.134834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943735+0000","last_change":"2024-09-19T11:23:11.271462+0000","last_active":"2024-09-19T11:23:42.943735+0000","last_peered":"2024-09-19T11:23:42.943735+0000","last_clean":"2024-09-19T11:23:42.943735+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:23:42.943735+0000","last_undegraded":"2024-09-19T11:23:42.943735+0000","last_fullsized":"2024-09-19T11:23:42.943735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.194246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.728999+0000","last_change":"2024-09-19T11:26:45.728999+0000","last_active":"2024-09-19T11:26:45.728999+0000","last_peered":"2024-09-19T11:26:45.728999+0000","last_clean":"2024-09-19T11:26:45.728999+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:45.728999+0000","last_undegraded":"2024-09-19T11:26:45.728999+0000","last_fullsized":"2024-09-19T11:26:45.728999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.728900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:26.277302+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.898841+0000","last_change":"2024-09-19T11:26:26.898841+0000","last_active":"2024-09-19T11:26:26.898841+0000","last_peered":"2024-09-19T11:26:26.898841+0000","last_clean":"2024-09-19T11:26:26.898841+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:26:26.898841+0000","last_undegraded":"2024-09-19T11:26:26.898841+0000","last_fullsized":"2024-09-19T11:26:26.898841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.898758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:04.348903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.929630+0000","last_change":"2024-09-19T11:26:59.929630+0000","last_active":"2024-09-19T11:26:59.929630+0000","last_peered":"2024-09-19T11:26:59.929630+0000","last_clean":"2024-09-19T11:26:59.929630+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:26:59.929630+0000","last_undegraded":"2024-09-19T11:26:59.929630+0000","last_fullsized":"2024-09-19T11:26:59.929630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.929544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:14.552058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'29","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.120252+0000","last_change":"2024-09-19T11:23:09.258685+0000","last_active":"2024-09-19T11:26:19.120252+0000","last_peered":"2024-09-19T11:26:19.120252+0000","last_clean":"2024-09-19T11:26:19.120252+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:26:19.120252+0000","last_undegraded":"2024-09-19T11:26:19.120252+0000","last_fullsized":"2024-09-19T11:26:19.120252+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:33.508526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944317+0000","last_change":"2024-09-19T11:23:11.267904+0000","last_active":"2024-09-19T11:23:42.944317+0000","last_peered":"2024-09-19T11:23:42.944317+0000","last_clean":"2024-09-19T11:23:42.944317+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:23:42.944317+0000","last_undegraded":"2024-09-19T11:23:42.944317+0000","last_fullsized":"2024-09-19T11:23:42.944317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:16.874318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.773207+0000","last_change":"2024-09-19T11:26:44.773207+0000","last_active":"2024-09-19T11:26:44.773207+0000","last_peered":"2024-09-19T11:26:44.773207+0000","last_clean":"2024-09-19T11:26:44.773207+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:26:44.773207+0000","last_undegraded":"2024-09-19T11:26:44.773207+0000","last_fullsized":"2024-09-19T11:26:44.773207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.773116+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:30.345346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.916292+0000","last_change":"2024-09-19T11:26:29.916292+0000","last_active":"2024-09-19T11:26:29.916292+0000","last_peered":"2024-09-19T11:26:29.916292+0000","last_clean":"2024-09-19T11:26:29.916292+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:26:29.916292+0000","last_undegraded":"2024-09-19T11:26:29.916292+0000","last_fullsized":"2024-09-19T11:26:29.916292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.916202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:50.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'2","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:08.365932+0000","last_change":"2024-09-19T11:23:07.247476+0000","last_active":"2024-09-19T11:25:08.365932+0000","last_peered":"2024-09-19T11:25:08.365932+0000","last_clean":"2024-09-19T11:25:08.365932+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:25:08.365932+0000","last_undegraded":"2024-09-19T11:25:08.365932+0000","last_fullsized":"2024-09-19T11:25:08.365932+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:06.222295+0000","last_clean_scrub_stamp":"2024-09-19T11:23:06.222295+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:03.848364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'37","reported_seq":86,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.129686+0000","last_change":"2024-09-19T11:23:09.255745+0000","last_active":"2024-09-19T11:26:19.129686+0000","last_peered":"2024-09-19T11:26:19.129686+0000","last_clean":"2024-09-19T11:26:19.129686+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:26:19.129686+0000","last_undegraded":"2024-09-19T11:26:19.129686+0000","last_fullsized":"2024-09-19T11:26:19.129686+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:35.538204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279071+0000","last_change":"2024-09-19T11:23:11.266117+0000","last_active":"2024-09-19T11:23:42.279071+0000","last_peered":"2024-09-19T11:23:42.279071+0000","last_clean":"2024-09-19T11:23:42.279071+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:23:42.279071+0000","last_undegraded":"2024-09-19T11:23:42.279071+0000","last_fullsized":"2024-09-19T11:23:42.279071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:22.839890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.120385+0000","last_change":"2024-09-19T11:26:45.120385+0000","last_active":"2024-09-19T11:26:45.120385+0000","last_peered":"2024-09-19T11:26:45.120385+0000","last_clean":"2024-09-19T11:26:45.120385+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:26:45.120385+0000","last_undegraded":"2024-09-19T11:26:45.120385+0000","last_fullsized":"2024-09-19T11:26:45.120385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.120277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:55.577999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.949825+0000","last_change":"2024-09-19T11:26:31.949825+0000","last_active":"2024-09-19T11:26:31.949825+0000","last_peered":"2024-09-19T11:26:31.949825+0000","last_clean":"2024-09-19T11:26:31.949825+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:26:31.949825+0000","last_undegraded":"2024-09-19T11:26:31.949825+0000","last_fullsized":"2024-09-19T11:26:31.949825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.949702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:38.587259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'18","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.879819+0000","last_change":"2024-09-19T11:26:46.879819+0000","last_active":"2024-09-19T11:26:46.879819+0000","last_peered":"2024-09-19T11:26:46.879819+0000","last_clean":"2024-09-19T11:26:46.879819+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:46.879819+0000","last_undegraded":"2024-09-19T11:26:46.879819+0000","last_fullsized":"2024-09-19T11:26:46.879819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'18","last_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_deep_scrub":"33'18","last_deep_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.879729+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:14.358902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.905294+0000","last_change":"2024-09-19T11:26:27.905294+0000","last_active":"2024-09-19T11:26:27.905294+0000","last_peered":"2024-09-19T11:26:27.905294+0000","last_clean":"2024-09-19T11:26:27.905294+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:26:27.905294+0000","last_undegraded":"2024-09-19T11:26:27.905294+0000","last_fullsized":"2024-09-19T11:26:27.905294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.905202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:43:38.666093+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:24:27.342000+0000","last_change":"2024-09-19T11:23:05.237372+0000","last_active":"2024-09-19T11:24:27.342000+0000","last_peered":"2024-09-19T11:24:27.342000+0000","last_clean":"2024-09-19T11:24:27.342000+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:24:27.342000+0000","last_undegraded":"2024-09-19T11:24:27.342000+0000","last_fullsized":"2024-09-19T11:24:27.342000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:27.975988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.388646+0000","last_change":"2024-09-19T11:26:42.388646+0000","last_active":"2024-09-19T11:26:42.388646+0000","last_peered":"2024-09-19T11:26:42.388646+0000","last_clean":"2024-09-19T11:26:42.388646+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:26:42.388646+0000","last_undegraded":"2024-09-19T11:26:42.388646+0000","last_fullsized":"2024-09-19T11:26:42.388646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.388560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:26.548057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.896965+0000","last_change":"2024-09-19T11:26:45.896965+0000","last_active":"2024-09-19T11:26:45.896965+0000","last_peered":"2024-09-19T11:26:45.896965+0000","last_clean":"2024-09-19T11:26:45.896965+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:26:45.896965+0000","last_undegraded":"2024-09-19T11:26:45.896965+0000","last_fullsized":"2024-09-19T11:26:45.896965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.896872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:37.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.953783+0000","last_change":"2024-09-19T11:26:30.953783+0000","last_active":"2024-09-19T11:26:30.953783+0000","last_peered":"2024-09-19T11:26:30.953783+0000","last_clean":"2024-09-19T11:26:30.953783+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:26:30.953783+0000","last_undegraded":"2024-09-19T11:26:30.953783+0000","last_fullsized":"2024-09-19T11:26:30.953783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.953693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:32.144901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.912991+0000","last_change":"2024-09-19T11:26:49.912991+0000","last_active":"2024-09-19T11:26:49.912991+0000","last_peered":"2024-09-19T11:26:49.912991+0000","last_clean":"2024-09-19T11:26:49.912991+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:49.912991+0000","last_undegraded":"2024-09-19T11:26:49.912991+0000","last_fullsized":"2024-09-19T11:26:49.912991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.912905+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:07.844062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.952079+0000","last_change":"2024-09-19T11:26:29.952079+0000","last_active":"2024-09-19T11:26:29.952079+0000","last_peered":"2024-09-19T11:26:29.952079+0000","last_clean":"2024-09-19T11:26:29.952079+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:29.952079+0000","last_undegraded":"2024-09-19T11:26:29.952079+0000","last_fullsized":"2024-09-19T11:26:29.952079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.951980+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:30.180814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.103981+0000","last_change":"2024-09-19T11:26:44.103981+0000","last_active":"2024-09-19T11:26:44.103981+0000","last_peered":"2024-09-19T11:26:44.103981+0000","last_clean":"2024-09-19T11:26:44.103981+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:26:44.103981+0000","last_undegraded":"2024-09-19T11:26:44.103981+0000","last_fullsized":"2024-09-19T11:26:44.103981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.103889+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:14.482038+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.867275+0000","last_change":"2024-09-19T11:26:30.867275+0000","last_active":"2024-09-19T11:26:30.867275+0000","last_peered":"2024-09-19T11:26:30.867275+0000","last_clean":"2024-09-19T11:26:30.867275+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:26:30.867275+0000","last_undegraded":"2024-09-19T11:26:30.867275+0000","last_fullsized":"2024-09-19T11:26:30.867275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.867184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:38.055814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.932807+0000","last_change":"2024-09-19T11:26:44.932807+0000","last_active":"2024-09-19T11:26:44.932807+0000","last_peered":"2024-09-19T11:26:44.932807+0000","last_clean":"2024-09-19T11:26:44.932807+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:26:44.932807+0000","last_undegraded":"2024-09-19T11:26:44.932807+0000","last_fullsized":"2024-09-19T11:26:44.932807+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.932715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:07.921677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.977981+0000","last_change":"2024-09-19T11:26:32.977981+0000","last_active":"2024-09-19T11:26:32.977981+0000","last_peered":"2024-09-19T11:26:32.977981+0000","last_clean":"2024-09-19T11:26:32.977981+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:32.977981+0000","last_undegraded":"2024-09-19T11:26:32.977981+0000","last_fullsized":"2024-09-19T11:26:32.977981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.977891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:36.076149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":88,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.063925+0000","last_change":"2024-09-19T11:26:43.063925+0000","last_active":"2024-09-19T11:26:43.063925+0000","last_peered":"2024-09-19T11:26:43.063925+0000","last_clean":"2024-09-19T11:26:43.063925+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:43.063925+0000","last_undegraded":"2024-09-19T11:26:43.063925+0000","last_fullsized":"2024-09-19T11:26:43.063925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.063836+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:23.821074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.990845+0000","last_change":"2024-09-19T11:26:31.990845+0000","last_active":"2024-09-19T11:26:31.990845+0000","last_peered":"2024-09-19T11:26:31.990845+0000","last_clean":"2024-09-19T11:26:31.990845+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:26:31.990845+0000","last_undegraded":"2024-09-19T11:26:31.990845+0000","last_fullsized":"2024-09-19T11:26:31.990845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.990752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:33.522127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.044177+0000","last_change":"2024-09-19T11:26:50.044177+0000","last_active":"2024-09-19T11:26:50.044177+0000","last_peered":"2024-09-19T11:26:50.044177+0000","last_clean":"2024-09-19T11:26:50.044177+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:26:50.044177+0000","last_undegraded":"2024-09-19T11:26:50.044177+0000","last_fullsized":"2024-09-19T11:26:50.044177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.044090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:50.741427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.359747+0000","last_change":"2024-09-19T11:26:43.359747+0000","last_active":"2024-09-19T11:26:43.359747+0000","last_peered":"2024-09-19T11:26:43.359747+0000","last_clean":"2024-09-19T11:26:43.359747+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:26:43.359747+0000","last_undegraded":"2024-09-19T11:26:43.359747+0000","last_fullsized":"2024-09-19T11:26:43.359747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.359650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.024659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.368100+0000","last_change":"2024-09-19T11:23:05.243797+0000","last_active":"2024-09-19T11:26:12.368100+0000","last_peered":"2024-09-19T11:26:12.368100+0000","last_clean":"2024-09-19T11:26:12.368100+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:26:12.368100+0000","last_undegraded":"2024-09-19T11:26:12.368100+0000","last_fullsized":"2024-09-19T11:26:12.368100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:33.524342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.017041+0000","last_change":"2024-09-19T11:26:33.017041+0000","last_active":"2024-09-19T11:26:33.017041+0000","last_peered":"2024-09-19T11:26:33.017041+0000","last_clean":"2024-09-19T11:26:33.017041+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:26:33.017041+0000","last_undegraded":"2024-09-19T11:26:33.017041+0000","last_fullsized":"2024-09-19T11:26:33.017041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.016950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:06.549550+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.883512+0000","last_change":"2024-09-19T11:26:47.883512+0000","last_active":"2024-09-19T11:26:47.883512+0000","last_peered":"2024-09-19T11:26:47.883512+0000","last_clean":"2024-09-19T11:26:47.883512+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:26:47.883512+0000","last_undegraded":"2024-09-19T11:26:47.883512+0000","last_fullsized":"2024-09-19T11:26:47.883512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.883414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:39.095013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.853602+0000","last_change":"2024-09-19T11:26:31.853602+0000","last_active":"2024-09-19T11:26:31.853602+0000","last_peered":"2024-09-19T11:26:31.853602+0000","last_clean":"2024-09-19T11:26:31.853602+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:26:31.853602+0000","last_undegraded":"2024-09-19T11:26:31.853602+0000","last_fullsized":"2024-09-19T11:26:31.853602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.853479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:16:07.826853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.073137+0000","last_change":"2024-09-19T11:26:49.073137+0000","last_active":"2024-09-19T11:26:49.073137+0000","last_peered":"2024-09-19T11:26:49.073137+0000","last_clean":"2024-09-19T11:26:49.073137+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:26:49.073137+0000","last_undegraded":"2024-09-19T11:26:49.073137+0000","last_fullsized":"2024-09-19T11:26:49.073137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.073043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:26.833881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.886892+0000","last_change":"2024-09-19T11:26:32.886892+0000","last_active":"2024-09-19T11:26:32.886892+0000","last_peered":"2024-09-19T11:26:32.886892+0000","last_clean":"2024-09-19T11:26:32.886892+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:26:32.886892+0000","last_undegraded":"2024-09-19T11:26:32.886892+0000","last_fullsized":"2024-09-19T11:26:32.886892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.886787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:19.114834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.038260+0000","last_change":"2024-09-19T11:26:48.038260+0000","last_active":"2024-09-19T11:26:48.038260+0000","last_peered":"2024-09-19T11:26:48.038260+0000","last_clean":"2024-09-19T11:26:48.038260+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:26:48.038260+0000","last_undegraded":"2024-09-19T11:26:48.038260+0000","last_fullsized":"2024-09-19T11:26:48.038260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.038149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:16.505179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.342671+0000","last_change":"2024-09-19T11:26:44.342671+0000","last_active":"2024-09-19T11:26:44.342671+0000","last_peered":"2024-09-19T11:26:44.342671+0000","last_clean":"2024-09-19T11:26:44.342671+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:26:44.342671+0000","last_undegraded":"2024-09-19T11:26:44.342671+0000","last_fullsized":"2024-09-19T11:26:44.342671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.342574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:38.309444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.082623+0000","last_change":"2024-09-19T11:26:47.082623+0000","last_active":"2024-09-19T11:26:47.082623+0000","last_peered":"2024-09-19T11:26:47.082623+0000","last_clean":"2024-09-19T11:26:47.082623+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:47.082623+0000","last_undegraded":"2024-09-19T11:26:47.082623+0000","last_fullsized":"2024-09-19T11:26:47.082623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.082498+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:22.862275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.395278+0000","last_change":"2024-09-19T11:26:46.395278+0000","last_active":"2024-09-19T11:26:46.395278+0000","last_peered":"2024-09-19T11:26:46.395278+0000","last_clean":"2024-09-19T11:26:46.395278+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:26:46.395278+0000","last_undegraded":"2024-09-19T11:26:46.395278+0000","last_fullsized":"2024-09-19T11:26:46.395278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.395181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:36.015668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.123790+0000","last_change":"2024-09-19T11:23:05.250503+0000","last_active":"2024-09-19T11:26:12.123790+0000","last_peered":"2024-09-19T11:26:12.123790+0000","last_clean":"2024-09-19T11:26:12.123790+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:26:12.123790+0000","last_undegraded":"2024-09-19T11:26:12.123790+0000","last_fullsized":"2024-09-19T11:26:12.123790+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:04.215609+0000","last_clean_scrub_stamp":"2024-09-19T11:23:04.215609+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:46.547197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.979119+0000","last_change":"2024-09-19T11:26:33.979119+0000","last_active":"2024-09-19T11:26:33.979119+0000","last_peered":"2024-09-19T11:26:33.979119+0000","last_clean":"2024-09-19T11:26:33.979119+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:26:33.979119+0000","last_undegraded":"2024-09-19T11:26:33.979119+0000","last_fullsized":"2024-09-19T11:26:33.979119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.979000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:24.446915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":86,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.075580+0000","last_change":"2024-09-19T11:26:46.075580+0000","last_active":"2024-09-19T11:26:46.075580+0000","last_peered":"2024-09-19T11:26:46.075580+0000","last_clean":"2024-09-19T11:26:46.075580+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:46.075580+0000","last_undegraded":"2024-09-19T11:26:46.075580+0000","last_fullsized":"2024-09-19T11:26:46.075580+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'26","last_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_deep_scrub":"33'26","last_deep_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.075475+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:51.242922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.388148+0000","last_change":"2024-09-19T11:26:45.388148+0000","last_active":"2024-09-19T11:26:45.388148+0000","last_peered":"2024-09-19T11:26:45.388148+0000","last_clean":"2024-09-19T11:26:45.388148+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:26:45.388148+0000","last_undegraded":"2024-09-19T11:26:45.388148+0000","last_fullsized":"2024-09-19T11:26:45.388148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.388045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:47.891066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.428183+0000","last_change":"2024-09-19T11:26:47.428183+0000","last_active":"2024-09-19T11:26:47.428183+0000","last_peered":"2024-09-19T11:26:47.428183+0000","last_clean":"2024-09-19T11:26:47.428183+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:26:47.428183+0000","last_undegraded":"2024-09-19T11:26:47.428183+0000","last_fullsized":"2024-09-19T11:26:47.428183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.428079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:59:02.198551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.933041+0000","last_change":"2024-09-19T11:26:48.933041+0000","last_active":"2024-09-19T11:26:48.933041+0000","last_peered":"2024-09-19T11:26:48.933041+0000","last_clean":"2024-09-19T11:26:48.933041+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:48.933041+0000","last_undegraded":"2024-09-19T11:26:48.933041+0000","last_fullsized":"2024-09-19T11:26:48.933041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.932953+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:16.178277+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.990707+0000","last_change":"2024-09-19T11:26:33.990707+0000","last_active":"2024-09-19T11:26:33.990707+0000","last_peered":"2024-09-19T11:26:33.990707+0000","last_clean":"2024-09-19T11:26:33.990707+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:26:33.990707+0000","last_undegraded":"2024-09-19T11:26:33.990707+0000","last_fullsized":"2024-09-19T11:26:33.990707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.990617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:35.040214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.051717+0000","last_change":"2024-09-19T11:26:54.051717+0000","last_active":"2024-09-19T11:26:54.051717+0000","last_peered":"2024-09-19T11:26:54.051717+0000","last_clean":"2024-09-19T11:26:54.051717+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:26:54.051717+0000","last_undegraded":"2024-09-19T11:26:54.051717+0000","last_fullsized":"2024-09-19T11:26:54.051717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.051625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:27.912591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1042,"num_omap_keys":4,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1397,"ondisk_log_size":1397,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":479,"ondisk_log_size":479,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673014,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":150572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673014,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":737565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705718,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":739071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705718,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":148560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:27:04.599 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.599 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.599 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.600 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.600 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.600 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.600 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.600 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.600 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.600 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.601 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.602 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.603 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.604 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.605 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.606 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.607 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.608 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.609 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.610 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.610 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.610 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.610 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.610 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.610 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.610 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.611 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.612 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.613 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-09-19T11:23:06.222295+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=6, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.613 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.613 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.613 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.613 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.613 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.614 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-09-19T11:23:04.215609+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:04.614 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T11:27:24.615 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:27:24.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.729+0000 7f6bc35cd640 1 Processor -- start 2024-09-19T11:27:24.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.729+0000 7f6bc35cd640 1 -- start start 2024-09-19T11:27:24.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc35cd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6bbc12cba0 0x7f6bbc1315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:24.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc35cd640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bbc09a290 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc35cd640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7f6bbc071860 con 0x7f6bbc12cba0 2024-09-19T11:27:24.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc0b68640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7f6bbc12cba0 0x7f6bbc1315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:24.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc1369640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7f6bbc12c0b0 0x7f6bbc12c490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:58416/0 (socket says 172.21.15.22:58416) 2024-09-19T11:27:24.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc1369640 1 -- 172.21.15.22:0/2099986360 learned_addr learned my addr 172.21.15.22:0/2099986360 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:27:24.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc0b68640 1 -- 172.21.15.22:0/2099986360 >> v1:172.21.15.22:6789/0 conn(0x7f6bbc12c0b0 legacy=0x7f6bbc12c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:24.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.730+0000 7f6bc0b68640 1 -- 172.21.15.22:0/2099986360 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc099e20 con 0x7f6bbc12cba0 2024-09-19T11:27:24.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bc0b68640 1 --2- 172.21.15.22:0/2099986360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12cba0 0x7f6bbc1315e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009750 tx=0x7f6bac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b0fb7ea3dc29e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bb3fff640 1 -- 172.21.15.22:0/2099986360 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bac037070 con 0x7f6bbc12cba0 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bb3fff640 1 -- 172.21.15.22:0/2099986360 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac02ad80 con 0x7f6bbc12cba0 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bb3fff640 1 -- 172.21.15.22:0/2099986360 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bac032590 con 0x7f6bbc12cba0 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bc35cd640 1 -- 172.21.15.22:0/2099986360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12cba0 msgr2=0x7f6bbc1315e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bc35cd640 1 --2- 172.21.15.22:0/2099986360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12cba0 0x7f6bbc1315e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009750 tx=0x7f6bac02a140 comp rx=0 tx=0).stop 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bc35cd640 1 -- 172.21.15.22:0/2099986360 shutdown_connections 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bc35cd640 1 --2- 172.21.15.22:0/2099986360 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12cba0 0x7f6bbc1315e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.731+0000 7f6bc35cd640 1 -- 172.21.15.22:0/2099986360 >> 172.21.15.22:0/2099986360 conn(0x7f6bbc09abc0 msgr2=0x7f6bbc098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.732+0000 7f6bc35cd640 1 -- 172.21.15.22:0/2099986360 shutdown_connections 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.732+0000 7f6bc35cd640 1 -- 172.21.15.22:0/2099986360 wait complete. 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.732+0000 7f6bc35cd640 1 Processor -- start 2024-09-19T11:27:24.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.732+0000 7f6bc35cd640 1 -- start start 2024-09-19T11:27:24.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bc35cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12c0b0 0x7f6bbc12fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:24.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bc35cd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc070c40 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bc1369640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12c0b0 0x7f6bbc12fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bc1369640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12c0b0 0x7f6bbc12fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59378/0 (socket says 172.21.15.22:59378) 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bc1369640 1 -- 172.21.15.22:0/3906047714 learned_addr learned my addr 172.21.15.22:0/3906047714 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bc1369640 1 -- 172.21.15.22:0/3906047714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc08caf0 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bc1369640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12c0b0 0x7f6bbc12fbe0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c480 tx=0x7f6bb800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bb1ffb640 1 -- 172.21.15.22:0/3906047714 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb8017070 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.733+0000 7f6bb1ffb640 1 -- 172.21.15.22:0/3906047714 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb8005c10 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.734+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc08bfe0 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.734+0000 7f6bb1ffb640 1 -- 172.21.15.22:0/3906047714 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bb8007070 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.734+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc097ac0 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.734+0000 7f6bb1ffb640 1 -- 172.21.15.22:0/3906047714 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7f6bb801a070 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.735+0000 7f6bb1ffb640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6ba003e710 0x7f6ba0040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.735+0000 7f6bb1ffb640 1 -- 172.21.15.22:0/3906047714 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7f6bb8052a30 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.735+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc12c490 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.735+0000 7f6bc0b68640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6ba003e710 0x7f6ba0040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:24.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.736+0000 7f6bc0b68640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6ba003e710 0x7f6ba0040bd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6bac034000 tx=0x7f6bac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:24.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.740+0000 7f6bb1ffb640 1 -- 172.21.15.22:0/3906047714 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7f6bb801ac60 con 0x7f6bbc12c0b0 2024-09-19T11:27:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.926+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6bbc073d40 con 0x7f6ba003e710 2024-09-19T11:27:24.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.936+0000 7f6bb1ffb640 1 -- 172.21.15.22:0/3906047714 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665945 (secure 0 0 0) 0x7f6bac0326f0 con 0x7f6ba003e710 2024-09-19T11:27:24.945 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:27:24.951 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:27:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6ba003e710 msgr2=0x7f6ba0040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6ba003e710 0x7f6ba0040bd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6bac034000 tx=0x7f6bac02e000 comp rx=0 tx=0).stop 2024-09-19T11:27:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12c0b0 msgr2=0x7f6bbc12fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12c0b0 0x7f6bbc12fbe0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c480 tx=0x7f6bb800c940 comp rx=0 tx=0).stop 2024-09-19T11:27:24.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 shutdown_connections 2024-09-19T11:27:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7f6ba003e710 0x7f6ba0040bd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 --2- 172.21.15.22:0/3906047714 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6bbc12c0b0 0x7f6bbc12fbe0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 >> 172.21.15.22:0/3906047714 conn(0x7f6bbc09abc0 msgr2=0x7f6bbc129190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:27:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 shutdown_connections 2024-09-19T11:27:24.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:24.942+0000 7f6bc35cd640 1 -- 172.21.15.22:0/3906047714 wait complete. 2024-09-19T11:27:24.956 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-19T11:27:24.868155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605410,"num_objects":222,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":0,"num_read":5189,"num_read_kb":4731,"num_write":2121,"num_write_kb":60587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1899,"num_omap_keys":9,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4734,"ondisk_log_size":4734,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":115136,"kb_used_data":2880,"kb_used_omap":6,"kb_used_meta":112185,"kb_avail":377372224,"statfs":{"total":386547056640,"available":386429157376,"internally_reserved":0,"allocated":2949120,"data_stored":1775768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":114878230},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":563,"num_omap_keys":3,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004991"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.932419+0000","last_change":"2024-09-19T11:26:37.932354+0000","last_active":"2024-09-19T11:26:37.932419+0000","last_peered":"2024-09-19T11:26:37.932419+0000","last_clean":"2024-09-19T11:26:37.932419+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:26:37.932419+0000","last_undegraded":"2024-09-19T11:26:37.932419+0000","last_fullsized":"2024-09-19T11:26:37.932419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.932266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:41.727377+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.069528+0000","last_change":"2024-09-19T11:26:55.069528+0000","last_active":"2024-09-19T11:26:55.069528+0000","last_peered":"2024-09-19T11:26:55.069528+0000","last_clean":"2024-09-19T11:26:55.069528+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:55.069528+0000","last_undegraded":"2024-09-19T11:26:55.069528+0000","last_fullsized":"2024-09-19T11:26:55.069528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.069440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:55.494351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.990965+0000","last_change":"2024-09-19T11:26:36.990885+0000","last_active":"2024-09-19T11:26:36.990965+0000","last_peered":"2024-09-19T11:26:36.990965+0000","last_clean":"2024-09-19T11:26:36.990965+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:26:36.990965+0000","last_undegraded":"2024-09-19T11:26:36.990965+0000","last_fullsized":"2024-09-19T11:26:36.990965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.990795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:53.995221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.480743+0000","last_change":"2024-09-19T11:27:05.480743+0000","last_active":"2024-09-19T11:27:05.480743+0000","last_peered":"2024-09-19T11:27:05.480743+0000","last_clean":"2024-09-19T11:27:05.480743+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:27:05.480743+0000","last_undegraded":"2024-09-19T11:27:05.480743+0000","last_fullsized":"2024-09-19T11:27:05.480743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:05.480656+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:05.480656+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.480656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:16.117082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.390858+0000","last_change":"2024-09-19T11:26:51.390768+0000","last_active":"2024-09-19T11:26:51.390858+0000","last_peered":"2024-09-19T11:26:51.390858+0000","last_clean":"2024-09-19T11:26:51.390858+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:26:51.390858+0000","last_undegraded":"2024-09-19T11:26:51.390858+0000","last_fullsized":"2024-09-19T11:26:51.390858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.390678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:06.584463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'12","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.945164+0000","last_change":"2024-09-19T11:26:50.945164+0000","last_active":"2024-09-19T11:26:50.945164+0000","last_peered":"2024-09-19T11:26:50.945164+0000","last_clean":"2024-09-19T11:26:50.945164+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:50.945164+0000","last_undegraded":"2024-09-19T11:26:50.945164+0000","last_fullsized":"2024-09-19T11:26:50.945164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.945069+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:26:56.726703+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.958237+0000","last_change":"2024-09-19T11:26:36.958139+0000","last_active":"2024-09-19T11:26:36.958237+0000","last_peered":"2024-09-19T11:26:36.958237+0000","last_clean":"2024-09-19T11:26:36.958237+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:26:36.958237+0000","last_undegraded":"2024-09-19T11:26:36.958237+0000","last_fullsized":"2024-09-19T11:26:36.958237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.958025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:34.544025+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.497840+0000","last_change":"2024-09-19T11:27:07.497840+0000","last_active":"2024-09-19T11:27:07.497840+0000","last_peered":"2024-09-19T11:27:07.497840+0000","last_clean":"2024-09-19T11:27:07.497840+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:27:07.497840+0000","last_undegraded":"2024-09-19T11:27:07.497840+0000","last_fullsized":"2024-09-19T11:27:07.497840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.497726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.497726+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.497726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:24.485123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.373571+0000","last_change":"2024-09-19T11:26:50.373571+0000","last_active":"2024-09-19T11:26:50.373571+0000","last_peered":"2024-09-19T11:26:50.373571+0000","last_clean":"2024-09-19T11:26:50.373571+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:26:50.373571+0000","last_undegraded":"2024-09-19T11:26:50.373571+0000","last_fullsized":"2024-09-19T11:26:50.373571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.373464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:22:03.662429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'12","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.036262+0000","last_change":"2024-09-19T11:26:56.036130+0000","last_active":"2024-09-19T11:26:56.036262+0000","last_peered":"2024-09-19T11:26:56.036262+0000","last_clean":"2024-09-19T11:26:56.036262+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:56.036262+0000","last_undegraded":"2024-09-19T11:26:56.036262+0000","last_fullsized":"2024-09-19T11:26:56.036262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.036029+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.756679+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.972860+0000","last_change":"2024-09-19T11:26:35.972860+0000","last_active":"2024-09-19T11:26:35.972860+0000","last_peered":"2024-09-19T11:26:35.972860+0000","last_clean":"2024-09-19T11:26:35.972860+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:26:35.972860+0000","last_undegraded":"2024-09-19T11:26:35.972860+0000","last_fullsized":"2024-09-19T11:26:35.972860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.972760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:17.815936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.524392+0000","last_change":"2024-09-19T11:27:08.524312+0000","last_active":"2024-09-19T11:27:08.524392+0000","last_peered":"2024-09-19T11:27:08.524392+0000","last_clean":"2024-09-19T11:27:08.524392+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:27:08.524392+0000","last_undegraded":"2024-09-19T11:27:08.524392+0000","last_fullsized":"2024-09-19T11:27:08.524392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:27:08.524226+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:27:08.524226+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.524226+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:57.580270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.431257+0000","last_change":"2024-09-19T11:26:48.431257+0000","last_active":"2024-09-19T11:26:48.431257+0000","last_peered":"2024-09-19T11:26:48.431257+0000","last_clean":"2024-09-19T11:26:48.431257+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:48.431257+0000","last_undegraded":"2024-09-19T11:26:48.431257+0000","last_fullsized":"2024-09-19T11:26:48.431257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.431171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:57.441652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.745217+0000","last_change":"2024-09-19T11:26:49.745135+0000","last_active":"2024-09-19T11:26:49.745217+0000","last_peered":"2024-09-19T11:26:49.745217+0000","last_clean":"2024-09-19T11:26:49.745217+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:49.745217+0000","last_undegraded":"2024-09-19T11:26:49.745217+0000","last_fullsized":"2024-09-19T11:26:49.745217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.745054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:51.276429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.989858+0000","last_change":"2024-09-19T11:26:35.989858+0000","last_active":"2024-09-19T11:26:35.989858+0000","last_peered":"2024-09-19T11:26:35.989858+0000","last_clean":"2024-09-19T11:26:35.989858+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:26:35.989858+0000","last_undegraded":"2024-09-19T11:26:35.989858+0000","last_fullsized":"2024-09-19T11:26:35.989858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.989770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.409598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.783741+0000","last_change":"2024-09-19T11:26:50.783644+0000","last_active":"2024-09-19T11:26:50.783741+0000","last_peered":"2024-09-19T11:26:50.783741+0000","last_clean":"2024-09-19T11:26:50.783741+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:26:50.783741+0000","last_undegraded":"2024-09-19T11:26:50.783741+0000","last_fullsized":"2024-09-19T11:26:50.783741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.783546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:56.110032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.464525+0000","last_change":"2024-09-19T11:27:06.464525+0000","last_active":"2024-09-19T11:27:06.464525+0000","last_peered":"2024-09-19T11:27:06.464525+0000","last_clean":"2024-09-19T11:27:06.464525+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:27:06.464525+0000","last_undegraded":"2024-09-19T11:27:06.464525+0000","last_fullsized":"2024-09-19T11:27:06.464525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:06.464436+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:06.464436+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.464436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:12.768140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.946461+0000","last_change":"2024-09-19T11:26:34.946461+0000","last_active":"2024-09-19T11:26:34.946461+0000","last_peered":"2024-09-19T11:26:34.946461+0000","last_clean":"2024-09-19T11:26:34.946461+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:26:34.946461+0000","last_undegraded":"2024-09-19T11:26:34.946461+0000","last_fullsized":"2024-09-19T11:26:34.946461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.946370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:48.027671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.017232+0000","last_change":"2024-09-19T11:26:51.017232+0000","last_active":"2024-09-19T11:26:51.017232+0000","last_peered":"2024-09-19T11:26:51.017232+0000","last_clean":"2024-09-19T11:26:51.017232+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:26:51.017232+0000","last_undegraded":"2024-09-19T11:26:51.017232+0000","last_fullsized":"2024-09-19T11:26:51.017232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.017146+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.668961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.952151+0000","last_change":"2024-09-19T11:26:34.952151+0000","last_active":"2024-09-19T11:26:34.952151+0000","last_peered":"2024-09-19T11:26:34.952151+0000","last_clean":"2024-09-19T11:26:34.952151+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:26:34.952151+0000","last_undegraded":"2024-09-19T11:26:34.952151+0000","last_fullsized":"2024-09-19T11:26:34.952151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.952058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:03.872887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.018264+0000","last_change":"2024-09-19T11:26:52.018264+0000","last_active":"2024-09-19T11:26:52.018264+0000","last_peered":"2024-09-19T11:26:52.018264+0000","last_clean":"2024-09-19T11:26:52.018264+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:26:52.018264+0000","last_undegraded":"2024-09-19T11:26:52.018264+0000","last_fullsized":"2024-09-19T11:26:52.018264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.018180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:31.898878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.916754+0000","last_change":"2024-09-19T11:26:33.916754+0000","last_active":"2024-09-19T11:26:33.916754+0000","last_peered":"2024-09-19T11:26:33.916754+0000","last_clean":"2024-09-19T11:26:33.916754+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:33.916754+0000","last_undegraded":"2024-09-19T11:26:33.916754+0000","last_fullsized":"2024-09-19T11:26:33.916754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.916663+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:35.684322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":68,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.898475+0000","last_change":"2024-09-19T11:26:51.898475+0000","last_active":"2024-09-19T11:26:51.898475+0000","last_peered":"2024-09-19T11:26:51.898475+0000","last_clean":"2024-09-19T11:26:51.898475+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:51.898475+0000","last_undegraded":"2024-09-19T11:26:51.898475+0000","last_fullsized":"2024-09-19T11:26:51.898475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.898386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:37.296273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.987445+0000","last_change":"2024-09-19T11:26:37.987445+0000","last_active":"2024-09-19T11:26:37.987445+0000","last_peered":"2024-09-19T11:26:37.987445+0000","last_clean":"2024-09-19T11:26:37.987445+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:26:37.987445+0000","last_undegraded":"2024-09-19T11:26:37.987445+0000","last_fullsized":"2024-09-19T11:26:37.987445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.987357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:27.173996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.779288+0000","last_change":"2024-09-19T11:26:48.779288+0000","last_active":"2024-09-19T11:26:48.779288+0000","last_peered":"2024-09-19T11:26:48.779288+0000","last_clean":"2024-09-19T11:26:48.779288+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:48.779288+0000","last_undegraded":"2024-09-19T11:26:48.779288+0000","last_fullsized":"2024-09-19T11:26:48.779288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.779185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:49.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.001114+0000","last_change":"2024-09-19T11:26:35.001114+0000","last_active":"2024-09-19T11:26:35.001114+0000","last_peered":"2024-09-19T11:26:35.001114+0000","last_clean":"2024-09-19T11:26:35.001114+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:26:35.001114+0000","last_undegraded":"2024-09-19T11:26:35.001114+0000","last_fullsized":"2024-09-19T11:26:35.001114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.001027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:09.576343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.033350+0000","last_change":"2024-09-19T11:26:53.033350+0000","last_active":"2024-09-19T11:26:53.033350+0000","last_peered":"2024-09-19T11:26:53.033350+0000","last_clean":"2024-09-19T11:26:53.033350+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:53.033350+0000","last_undegraded":"2024-09-19T11:26:53.033350+0000","last_fullsized":"2024-09-19T11:26:53.033350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.033252+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:11.509923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.392963+0000","last_change":"2024-09-19T11:26:49.392963+0000","last_active":"2024-09-19T11:26:49.392963+0000","last_peered":"2024-09-19T11:26:49.392963+0000","last_clean":"2024-09-19T11:26:49.392963+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:26:49.392963+0000","last_undegraded":"2024-09-19T11:26:49.392963+0000","last_fullsized":"2024-09-19T11:26:49.392963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.392873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:20.014824+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'37","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.305388+0000","last_change":"2024-09-19T11:23:13.275524+0000","last_active":"2024-09-19T11:26:12.305388+0000","last_peered":"2024-09-19T11:26:12.305388+0000","last_clean":"2024-09-19T11:26:12.305388+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:26:12.305388+0000","last_undegraded":"2024-09-19T11:26:12.305388+0000","last_fullsized":"2024-09-19T11:26:12.305388+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:57.670701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.766852+0000","last_change":"2024-09-19T11:26:43.766852+0000","last_active":"2024-09-19T11:26:43.766852+0000","last_peered":"2024-09-19T11:26:43.766852+0000","last_clean":"2024-09-19T11:26:43.766852+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:43.766852+0000","last_undegraded":"2024-09-19T11:26:43.766852+0000","last_fullsized":"2024-09-19T11:26:43.766852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.766755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:14.579342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.365713+0000","last_change":"2024-09-19T11:26:37.365713+0000","last_active":"2024-09-19T11:26:37.365713+0000","last_peered":"2024-09-19T11:26:37.365713+0000","last_clean":"2024-09-19T11:26:37.365713+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:26:37.365713+0000","last_undegraded":"2024-09-19T11:26:37.365713+0000","last_fullsized":"2024-09-19T11:26:37.365713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.365625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:20.032640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.572512+0000","last_change":"2024-09-19T11:27:15.572512+0000","last_active":"2024-09-19T11:27:15.572512+0000","last_peered":"2024-09-19T11:27:15.572512+0000","last_clean":"2024-09-19T11:27:15.572512+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:27:15.572512+0000","last_undegraded":"2024-09-19T11:27:15.572512+0000","last_fullsized":"2024-09-19T11:27:15.572512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.572426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.572426+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.572426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:44.280324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'34","reported_seq":81,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.204815+0000","last_change":"2024-09-19T11:23:09.256765+0000","last_active":"2024-09-19T11:26:19.204815+0000","last_peered":"2024-09-19T11:26:19.204815+0000","last_clean":"2024-09-19T11:26:19.204815+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:26:19.204815+0000","last_undegraded":"2024-09-19T11:26:19.204815+0000","last_fullsized":"2024-09-19T11:26:19.204815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:39:32.451376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.903454+0000","last_change":"2024-09-19T11:27:14.903454+0000","last_active":"2024-09-19T11:27:14.903454+0000","last_peered":"2024-09-19T11:27:14.903454+0000","last_clean":"2024-09-19T11:27:14.903454+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:27:14.903454+0000","last_undegraded":"2024-09-19T11:27:14.903454+0000","last_fullsized":"2024-09-19T11:27:14.903454+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.903359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.903359+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.903359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:25.564676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:11.707113+0000","last_change":"2024-09-19T11:23:13.274325+0000","last_active":"2024-09-19T11:26:11.707113+0000","last_peered":"2024-09-19T11:26:11.707113+0000","last_clean":"2024-09-19T11:26:11.707113+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:26:11.707113+0000","last_undegraded":"2024-09-19T11:26:11.707113+0000","last_fullsized":"2024-09-19T11:26:11.707113+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:53:22.517816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.402917+0000","last_change":"2024-09-19T11:26:57.402917+0000","last_active":"2024-09-19T11:26:57.402917+0000","last_peered":"2024-09-19T11:26:57.402917+0000","last_clean":"2024-09-19T11:26:57.402917+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:57.402917+0000","last_undegraded":"2024-09-19T11:26:57.402917+0000","last_fullsized":"2024-09-19T11:26:57.402917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.402830+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:15.259544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.907786+0000","last_change":"2024-09-19T11:26:30.907786+0000","last_active":"2024-09-19T11:26:30.907786+0000","last_peered":"2024-09-19T11:26:30.907786+0000","last_clean":"2024-09-19T11:26:30.907786+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:26:30.907786+0000","last_undegraded":"2024-09-19T11:26:30.907786+0000","last_fullsized":"2024-09-19T11:26:30.907786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.907688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:01.389670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.767856+0000","last_change":"2024-09-19T11:26:59.767856+0000","last_active":"2024-09-19T11:26:59.767856+0000","last_peered":"2024-09-19T11:26:59.767856+0000","last_clean":"2024-09-19T11:26:59.767856+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:26:59.767856+0000","last_undegraded":"2024-09-19T11:26:59.767856+0000","last_fullsized":"2024-09-19T11:26:59.767856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.767773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:00:22.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'10","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.925116+0000","last_change":"2024-09-19T11:27:09.925116+0000","last_active":"2024-09-19T11:27:09.925116+0000","last_peered":"2024-09-19T11:27:09.925116+0000","last_clean":"2024-09-19T11:27:09.925116+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:27:09.925116+0000","last_undegraded":"2024-09-19T11:27:09.925116+0000","last_fullsized":"2024-09-19T11:27:09.925116+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:27:09.925027+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:27:09.925027+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.925027+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:01.267066+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.901960+0000","last_change":"2024-09-19T11:27:13.901960+0000","last_active":"2024-09-19T11:27:13.901960+0000","last_peered":"2024-09-19T11:27:13.901960+0000","last_clean":"2024-09-19T11:27:13.901960+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:27:13.901960+0000","last_undegraded":"2024-09-19T11:27:13.901960+0000","last_fullsized":"2024-09-19T11:27:13.901960+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:13.901857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:13.901857+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.901857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:59.640755+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.278313+0000","last_change":"2024-09-19T11:23:13.266866+0000","last_active":"2024-09-19T11:23:42.278313+0000","last_peered":"2024-09-19T11:23:42.278313+0000","last_clean":"2024-09-19T11:23:42.278313+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:23:42.278313+0000","last_undegraded":"2024-09-19T11:23:42.278313+0000","last_fullsized":"2024-09-19T11:23:42.278313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:33:33.618068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.434595+0000","last_change":"2024-09-19T11:26:59.434595+0000","last_active":"2024-09-19T11:26:59.434595+0000","last_peered":"2024-09-19T11:26:59.434595+0000","last_clean":"2024-09-19T11:26:59.434595+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:26:59.434595+0000","last_undegraded":"2024-09-19T11:26:59.434595+0000","last_fullsized":"2024-09-19T11:26:59.434595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.434505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:29.091390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.378428+0000","last_change":"2024-09-19T11:26:35.378428+0000","last_active":"2024-09-19T11:26:35.378428+0000","last_peered":"2024-09-19T11:26:35.378428+0000","last_clean":"2024-09-19T11:26:35.378428+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:26:35.378428+0000","last_undegraded":"2024-09-19T11:26:35.378428+0000","last_fullsized":"2024-09-19T11:26:35.378428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.378338+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:26.014662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.815608+0000","last_change":"2024-09-19T11:26:54.815608+0000","last_active":"2024-09-19T11:26:54.815608+0000","last_peered":"2024-09-19T11:26:54.815608+0000","last_clean":"2024-09-19T11:26:54.815608+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:26:54.815608+0000","last_undegraded":"2024-09-19T11:26:54.815608+0000","last_fullsized":"2024-09-19T11:26:54.815608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.815526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:36.935748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"33'15","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.846706+0000","last_change":"2024-09-19T11:27:04.846706+0000","last_active":"2024-09-19T11:27:04.846706+0000","last_peered":"2024-09-19T11:27:04.846706+0000","last_clean":"2024-09-19T11:27:04.846706+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:27:04.846706+0000","last_undegraded":"2024-09-19T11:27:04.846706+0000","last_fullsized":"2024-09-19T11:27:04.846706+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'15","last_scrub_stamp":"2024-09-19T11:27:04.846607+0000","last_deep_scrub":"33'15","last_deep_scrub_stamp":"2024-09-19T11:27:04.846607+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.846607+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:56.058206+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'285","reported_seq":389,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.073009+0000","last_change":"2024-09-19T11:23:11.271056+0000","last_active":"2024-09-19T11:26:19.073009+0000","last_peered":"2024-09-19T11:26:19.073009+0000","last_clean":"2024-09-19T11:26:19.073009+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:26:19.073009+0000","last_undegraded":"2024-09-19T11:26:19.073009+0000","last_fullsized":"2024-09-19T11:26:19.073009+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:49:07.822761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.970806+0000","last_change":"2024-09-19T11:27:17.970806+0000","last_active":"2024-09-19T11:27:17.970806+0000","last_peered":"2024-09-19T11:27:17.970806+0000","last_clean":"2024-09-19T11:27:17.970806+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:27:17.970806+0000","last_undegraded":"2024-09-19T11:27:17.970806+0000","last_fullsized":"2024-09-19T11:27:17.970806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:27:17.970666+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:27:17.970666+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.970666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:52.045636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.025004+0000","last_change":"2024-09-19T11:26:42.025004+0000","last_active":"2024-09-19T11:26:42.025004+0000","last_peered":"2024-09-19T11:26:42.025004+0000","last_clean":"2024-09-19T11:26:42.025004+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:26:42.025004+0000","last_undegraded":"2024-09-19T11:26:42.025004+0000","last_fullsized":"2024-09-19T11:26:42.025004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.024919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:31.267851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.870975+0000","last_change":"2024-09-19T11:26:24.870975+0000","last_active":"2024-09-19T11:26:24.870975+0000","last_peered":"2024-09-19T11:26:24.870975+0000","last_clean":"2024-09-19T11:26:24.870975+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:26:24.870975+0000","last_undegraded":"2024-09-19T11:26:24.870975+0000","last_fullsized":"2024-09-19T11:26:24.870975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.870884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:14.567909+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.528989+0000","last_change":"2024-09-19T11:27:17.528989+0000","last_active":"2024-09-19T11:27:17.528989+0000","last_peered":"2024-09-19T11:27:17.528989+0000","last_clean":"2024-09-19T11:27:17.528989+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:27:17.528989+0000","last_undegraded":"2024-09-19T11:27:17.528989+0000","last_fullsized":"2024-09-19T11:27:17.528989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:17.528899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:17.528899+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.528899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:52:06.431179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'38","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.841593+0000","last_change":"2024-09-19T11:27:05.841593+0000","last_active":"2024-09-19T11:27:05.841593+0000","last_peered":"2024-09-19T11:27:05.841593+0000","last_clean":"2024-09-19T11:27:05.841593+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:27:05.841593+0000","last_undegraded":"2024-09-19T11:27:05.841593+0000","last_fullsized":"2024-09-19T11:27:05.841593+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:05.841496+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:05.841496+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.841496+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:33.086820+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'286","reported_seq":414,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.828626+0000","last_change":"2024-09-19T11:27:13.828626+0000","last_active":"2024-09-19T11:27:13.828626+0000","last_peered":"2024-09-19T11:27:13.828626+0000","last_clean":"2024-09-19T11:27:13.828626+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:27:13.828626+0000","last_undegraded":"2024-09-19T11:27:13.828626+0000","last_fullsized":"2024-09-19T11:27:13.828626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:13.828521+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:13.828521+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.828521+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:34.119736+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.381320+0000","last_change":"2024-09-19T11:26:55.381320+0000","last_active":"2024-09-19T11:26:55.381320+0000","last_peered":"2024-09-19T11:26:55.381320+0000","last_clean":"2024-09-19T11:26:55.381320+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:26:55.381320+0000","last_undegraded":"2024-09-19T11:26:55.381320+0000","last_fullsized":"2024-09-19T11:26:55.381320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.381228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:24:06.217118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.956170+0000","last_change":"2024-09-19T11:26:27.956170+0000","last_active":"2024-09-19T11:26:27.956170+0000","last_peered":"2024-09-19T11:26:27.956170+0000","last_clean":"2024-09-19T11:26:27.956170+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:26:27.956170+0000","last_undegraded":"2024-09-19T11:26:27.956170+0000","last_fullsized":"2024-09-19T11:26:27.956170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.956081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:59.956862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.969498+0000","last_change":"2024-09-19T11:26:57.969498+0000","last_active":"2024-09-19T11:26:57.969498+0000","last_peered":"2024-09-19T11:26:57.969498+0000","last_clean":"2024-09-19T11:26:57.969498+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:26:57.969498+0000","last_undegraded":"2024-09-19T11:26:57.969498+0000","last_fullsized":"2024-09-19T11:26:57.969498+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.969412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.712881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'52","reported_seq":112,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.849236+0000","last_change":"2024-09-19T11:27:01.849236+0000","last_active":"2024-09-19T11:27:01.849236+0000","last_peered":"2024-09-19T11:27:01.849236+0000","last_clean":"2024-09-19T11:27:01.849236+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:27:01.849236+0000","last_undegraded":"2024-09-19T11:27:01.849236+0000","last_fullsized":"2024-09-19T11:27:01.849236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'52","last_scrub_stamp":"2024-09-19T11:27:01.849143+0000","last_deep_scrub":"33'52","last_deep_scrub_stamp":"2024-09-19T11:27:01.849143+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.849143+0000","objects_scrubbed":9,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:24.838105+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.808200+0000","last_change":"2024-09-19T11:27:12.808200+0000","last_active":"2024-09-19T11:27:12.808200+0000","last_peered":"2024-09-19T11:27:12.808200+0000","last_clean":"2024-09-19T11:27:12.808200+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:27:12.808200+0000","last_undegraded":"2024-09-19T11:27:12.808200+0000","last_fullsized":"2024-09-19T11:27:12.808200+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:12.808111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:12.808111+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.808111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:53.887293+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.942123+0000","last_change":"2024-09-19T11:27:14.942063+0000","last_active":"2024-09-19T11:27:14.942123+0000","last_peered":"2024-09-19T11:27:14.942123+0000","last_clean":"2024-09-19T11:27:14.942123+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:27:14.942123+0000","last_undegraded":"2024-09-19T11:27:14.942123+0000","last_fullsized":"2024-09-19T11:27:14.942123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:14.941960+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:14.941960+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.941960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:20:48.525210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.014072+0000","last_change":"2024-09-19T11:26:39.014072+0000","last_active":"2024-09-19T11:26:39.014072+0000","last_peered":"2024-09-19T11:26:39.014072+0000","last_clean":"2024-09-19T11:26:39.014072+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:26:39.014072+0000","last_undegraded":"2024-09-19T11:26:39.014072+0000","last_fullsized":"2024-09-19T11:26:39.014072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.013977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:33.627023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.962098+0000","last_change":"2024-09-19T11:26:26.962098+0000","last_active":"2024-09-19T11:26:26.962098+0000","last_peered":"2024-09-19T11:26:26.962098+0000","last_clean":"2024-09-19T11:26:26.962098+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:26:26.962098+0000","last_undegraded":"2024-09-19T11:26:26.962098+0000","last_fullsized":"2024-09-19T11:26:26.962098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.962016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:19.643957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.852269+0000","last_change":"2024-09-19T11:26:53.852269+0000","last_active":"2024-09-19T11:26:53.852269+0000","last_peered":"2024-09-19T11:26:53.852269+0000","last_clean":"2024-09-19T11:26:53.852269+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:26:53.852269+0000","last_undegraded":"2024-09-19T11:26:53.852269+0000","last_fullsized":"2024-09-19T11:26:53.852269+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.852186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:52.995416+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'33","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.158685+0000","last_change":"2024-09-19T11:23:09.262668+0000","last_active":"2024-09-19T11:26:19.158685+0000","last_peered":"2024-09-19T11:26:19.158685+0000","last_clean":"2024-09-19T11:26:19.158685+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:26:19.158685+0000","last_undegraded":"2024-09-19T11:26:19.158685+0000","last_fullsized":"2024-09-19T11:26:19.158685+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:42.306676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.801023+0000","last_change":"2024-09-19T11:27:11.801023+0000","last_active":"2024-09-19T11:27:11.801023+0000","last_peered":"2024-09-19T11:27:11.801023+0000","last_clean":"2024-09-19T11:27:11.801023+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:27:11.801023+0000","last_undegraded":"2024-09-19T11:27:11.801023+0000","last_fullsized":"2024-09-19T11:27:11.801023+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.800931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.800931+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.800931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:52:07.495727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'6","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.973033+0000","last_change":"2024-09-19T11:27:17.973033+0000","last_active":"2024-09-19T11:27:17.973033+0000","last_peered":"2024-09-19T11:27:17.973033+0000","last_clean":"2024-09-19T11:27:17.973033+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:27:17.973033+0000","last_undegraded":"2024-09-19T11:27:17.973033+0000","last_fullsized":"2024-09-19T11:27:17.973033+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:27:17.972934+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:27:17.972934+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.972934+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:31.636218+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.889558+0000","last_change":"2024-09-19T11:26:38.889489+0000","last_active":"2024-09-19T11:26:38.889558+0000","last_peered":"2024-09-19T11:26:38.889558+0000","last_clean":"2024-09-19T11:26:38.889558+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:26:38.889558+0000","last_undegraded":"2024-09-19T11:26:38.889558+0000","last_fullsized":"2024-09-19T11:26:38.889558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.889400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:14.595110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.412815+0000","last_change":"2024-09-19T11:26:36.412815+0000","last_active":"2024-09-19T11:26:36.412815+0000","last_peered":"2024-09-19T11:26:36.412815+0000","last_clean":"2024-09-19T11:26:36.412815+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:36.412815+0000","last_undegraded":"2024-09-19T11:26:36.412815+0000","last_fullsized":"2024-09-19T11:26:36.412815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.412703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:59.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.839126+0000","last_change":"2024-09-19T11:26:55.839126+0000","last_active":"2024-09-19T11:26:55.839126+0000","last_peered":"2024-09-19T11:26:55.839126+0000","last_clean":"2024-09-19T11:26:55.839126+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:26:55.839126+0000","last_undegraded":"2024-09-19T11:26:55.839126+0000","last_fullsized":"2024-09-19T11:26:55.839126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:08.800957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'46","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:21.581442+0000","last_change":"2024-09-19T11:27:21.581442+0000","last_active":"2024-09-19T11:27:21.581442+0000","last_peered":"2024-09-19T11:27:21.581442+0000","last_clean":"2024-09-19T11:27:21.581442+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:27:21.581442+0000","last_undegraded":"2024-09-19T11:27:21.581442+0000","last_fullsized":"2024-09-19T11:27:21.581442+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:21.581324+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:21.581324+0000","last_clean_scrub_stamp":"2024-09-19T11:27:21.581324+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:02.933515+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'286","reported_seq":361,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.777433+0000","last_change":"2024-09-19T11:27:10.777433+0000","last_active":"2024-09-19T11:27:10.777433+0000","last_peered":"2024-09-19T11:27:10.777433+0000","last_clean":"2024-09-19T11:27:10.777433+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:27:10.777433+0000","last_undegraded":"2024-09-19T11:27:10.777433+0000","last_fullsized":"2024-09-19T11:27:10.777433+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:10.777329+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:10.777329+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.777329+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:10.736169+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.854808+0000","last_change":"2024-09-19T11:27:17.854808+0000","last_active":"2024-09-19T11:27:17.854808+0000","last_peered":"2024-09-19T11:27:17.854808+0000","last_clean":"2024-09-19T11:27:17.854808+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:27:17.854808+0000","last_undegraded":"2024-09-19T11:27:17.854808+0000","last_fullsized":"2024-09-19T11:27:17.854808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:17.854703+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:17.854703+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.854703+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:23.432636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.810354+0000","last_change":"2024-09-19T11:26:40.810354+0000","last_active":"2024-09-19T11:26:40.810354+0000","last_peered":"2024-09-19T11:26:40.810354+0000","last_clean":"2024-09-19T11:26:40.810354+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:40.810354+0000","last_undegraded":"2024-09-19T11:26:40.810354+0000","last_fullsized":"2024-09-19T11:26:40.810354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.810272+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.328550+0000","last_change":"2024-09-19T11:26:34.328550+0000","last_active":"2024-09-19T11:26:34.328550+0000","last_peered":"2024-09-19T11:26:34.328550+0000","last_clean":"2024-09-19T11:26:34.328550+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:26:34.328550+0000","last_undegraded":"2024-09-19T11:26:34.328550+0000","last_fullsized":"2024-09-19T11:26:34.328550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.328463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:54.619678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.868748+0000","last_change":"2024-09-19T11:26:54.868748+0000","last_active":"2024-09-19T11:26:54.868748+0000","last_peered":"2024-09-19T11:26:54.868748+0000","last_clean":"2024-09-19T11:26:54.868748+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:26:54.868748+0000","last_undegraded":"2024-09-19T11:26:54.868748+0000","last_fullsized":"2024-09-19T11:26:54.868748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.868649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:17.358870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'61","reported_seq":126,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.167477+0000","last_change":"2024-09-19T11:23:09.259576+0000","last_active":"2024-09-19T11:26:19.167477+0000","last_peered":"2024-09-19T11:26:19.167477+0000","last_clean":"2024-09-19T11:26:19.167477+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:26:19.167477+0000","last_undegraded":"2024-09-19T11:26:19.167477+0000","last_fullsized":"2024-09-19T11:26:19.167477+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:55.556001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.744324+0000","last_change":"2024-09-19T11:27:08.744324+0000","last_active":"2024-09-19T11:27:08.744324+0000","last_peered":"2024-09-19T11:27:08.744324+0000","last_clean":"2024-09-19T11:27:08.744324+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:27:08.744324+0000","last_undegraded":"2024-09-19T11:27:08.744324+0000","last_fullsized":"2024-09-19T11:27:08.744324+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:08.744222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:08.744222+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.744222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:52.725769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'21","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.840174+0000","last_change":"2024-09-19T11:27:18.840174+0000","last_active":"2024-09-19T11:27:18.840174+0000","last_peered":"2024-09-19T11:27:18.840174+0000","last_clean":"2024-09-19T11:27:18.840174+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:27:18.840174+0000","last_undegraded":"2024-09-19T11:27:18.840174+0000","last_fullsized":"2024-09-19T11:27:18.840174+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'21","last_scrub_stamp":"2024-09-19T11:27:18.840078+0000","last_deep_scrub":"33'21","last_deep_scrub_stamp":"2024-09-19T11:27:18.840078+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.840078+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:42.094194+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":92,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.048324+0000","last_change":"2024-09-19T11:26:40.048324+0000","last_active":"2024-09-19T11:26:40.048324+0000","last_peered":"2024-09-19T11:26:40.048324+0000","last_clean":"2024-09-19T11:26:40.048324+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:26:40.048324+0000","last_undegraded":"2024-09-19T11:26:40.048324+0000","last_fullsized":"2024-09-19T11:26:40.048324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.048219+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:04:18.644192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.937533+0000","last_change":"2024-09-19T11:26:28.937533+0000","last_active":"2024-09-19T11:26:28.937533+0000","last_peered":"2024-09-19T11:26:28.937533+0000","last_clean":"2024-09-19T11:26:28.937533+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:28.937533+0000","last_undegraded":"2024-09-19T11:26:28.937533+0000","last_fullsized":"2024-09-19T11:26:28.937533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.937444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:51.513741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.828665+0000","last_change":"2024-09-19T11:26:52.828665+0000","last_active":"2024-09-19T11:26:52.828665+0000","last_peered":"2024-09-19T11:26:52.828665+0000","last_clean":"2024-09-19T11:26:52.828665+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:26:52.828665+0000","last_undegraded":"2024-09-19T11:26:52.828665+0000","last_fullsized":"2024-09-19T11:26:52.828665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.828574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'41","reported_seq":97,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.915209+0000","last_change":"2024-09-19T11:27:07.915209+0000","last_active":"2024-09-19T11:27:07.915209+0000","last_peered":"2024-09-19T11:27:07.915209+0000","last_clean":"2024-09-19T11:27:07.915209+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:27:07.915209+0000","last_undegraded":"2024-09-19T11:27:07.915209+0000","last_fullsized":"2024-09-19T11:27:07.915209+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'41","last_scrub_stamp":"2024-09-19T11:27:07.915099+0000","last_deep_scrub":"33'41","last_deep_scrub_stamp":"2024-09-19T11:27:07.915099+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.915099+0000","objects_scrubbed":5,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:28.937882+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946607+0000","last_change":"2024-09-19T11:23:11.270709+0000","last_active":"2024-09-19T11:23:42.946607+0000","last_peered":"2024-09-19T11:23:42.946607+0000","last_clean":"2024-09-19T11:23:42.946607+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:23:42.946607+0000","last_undegraded":"2024-09-19T11:23:42.946607+0000","last_fullsized":"2024-09-19T11:23:42.946607+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:39.157590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'3","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.473352+0000","last_change":"2024-09-19T11:23:13.272542+0000","last_active":"2024-09-19T11:26:12.473352+0000","last_peered":"2024-09-19T11:26:12.473352+0000","last_clean":"2024-09-19T11:26:12.473352+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:26:12.473352+0000","last_undegraded":"2024-09-19T11:26:12.473352+0000","last_fullsized":"2024-09-19T11:26:12.473352+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":70,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.393655+0000","last_change":"2024-09-19T11:26:56.393655+0000","last_active":"2024-09-19T11:26:56.393655+0000","last_peered":"2024-09-19T11:26:56.393655+0000","last_clean":"2024-09-19T11:26:56.393655+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:26:56.393655+0000","last_undegraded":"2024-09-19T11:26:56.393655+0000","last_fullsized":"2024-09-19T11:26:56.393655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.393557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:06.327682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.308889+0000","last_change":"2024-09-19T11:26:33.308889+0000","last_active":"2024-09-19T11:26:33.308889+0000","last_peered":"2024-09-19T11:26:33.308889+0000","last_clean":"2024-09-19T11:26:33.308889+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:26:33.308889+0000","last_undegraded":"2024-09-19T11:26:33.308889+0000","last_fullsized":"2024-09-19T11:26:33.308889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.308801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:19.525046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.829806+0000","last_change":"2024-09-19T11:26:57.829806+0000","last_active":"2024-09-19T11:26:57.829806+0000","last_peered":"2024-09-19T11:26:57.829806+0000","last_clean":"2024-09-19T11:26:57.829806+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:26:57.829806+0000","last_undegraded":"2024-09-19T11:26:57.829806+0000","last_fullsized":"2024-09-19T11:26:57.829806+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.829722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:47.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'46","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.818874+0000","last_change":"2024-09-19T11:27:02.818874+0000","last_active":"2024-09-19T11:27:02.818874+0000","last_peered":"2024-09-19T11:27:02.818874+0000","last_clean":"2024-09-19T11:27:02.818874+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:27:02.818874+0000","last_undegraded":"2024-09-19T11:27:02.818874+0000","last_fullsized":"2024-09-19T11:27:02.818874+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:02.818754+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:02.818754+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.818754+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:54.439165+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'286","reported_seq":416,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.815074+0000","last_change":"2024-09-19T11:27:06.815074+0000","last_active":"2024-09-19T11:27:06.815074+0000","last_peered":"2024-09-19T11:27:06.815074+0000","last_clean":"2024-09-19T11:27:06.815074+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:27:06.815074+0000","last_undegraded":"2024-09-19T11:27:06.815074+0000","last_fullsized":"2024-09-19T11:27:06.815074+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:06.814991+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:06.814991+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.814991+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:24.746148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'14","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:12.475120+0000","last_change":"2024-09-19T11:23:13.265074+0000","last_active":"2024-09-19T11:26:12.475120+0000","last_peered":"2024-09-19T11:26:12.475120+0000","last_clean":"2024-09-19T11:26:12.475120+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:26:12.475120+0000","last_undegraded":"2024-09-19T11:26:12.475120+0000","last_fullsized":"2024-09-19T11:26:12.475120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:47.118156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.810833+0000","last_change":"2024-09-19T11:26:41.810833+0000","last_active":"2024-09-19T11:26:41.810833+0000","last_peered":"2024-09-19T11:26:41.810833+0000","last_clean":"2024-09-19T11:26:41.810833+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:41.810833+0000","last_undegraded":"2024-09-19T11:26:41.810833+0000","last_fullsized":"2024-09-19T11:26:41.810833+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.810721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:31.441015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.268981+0000","last_change":"2024-09-19T11:26:31.268981+0000","last_active":"2024-09-19T11:26:31.268981+0000","last_peered":"2024-09-19T11:26:31.268981+0000","last_clean":"2024-09-19T11:26:31.268981+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:26:31.268981+0000","last_undegraded":"2024-09-19T11:26:31.268981+0000","last_fullsized":"2024-09-19T11:26:31.268981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.268893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:31.227733+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'5","reported_seq":428,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.945300+0000","last_change":"2024-09-19T11:26:58.945300+0000","last_active":"2024-09-19T11:26:58.945300+0000","last_peered":"2024-09-19T11:26:58.945300+0000","last_clean":"2024-09-19T11:26:58.945300+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:26:58.945300+0000","last_undegraded":"2024-09-19T11:26:58.945300+0000","last_fullsized":"2024-09-19T11:26:58.945300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.945214+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:11.607957+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"31'10","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946255+0000","last_change":"2024-09-19T11:23:09.255928+0000","last_active":"2024-09-19T11:23:42.946255+0000","last_peered":"2024-09-19T11:23:42.946255+0000","last_clean":"2024-09-19T11:23:42.946255+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:23:42.946255+0000","last_undegraded":"2024-09-19T11:23:42.946255+0000","last_fullsized":"2024-09-19T11:23:42.946255+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:04.689951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'286","reported_seq":478,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.877361+0000","last_change":"2024-09-19T11:27:12.877361+0000","last_active":"2024-09-19T11:27:12.877361+0000","last_peered":"2024-09-19T11:27:12.877361+0000","last_clean":"2024-09-19T11:27:12.877361+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:27:12.877361+0000","last_undegraded":"2024-09-19T11:27:12.877361+0000","last_fullsized":"2024-09-19T11:27:12.877361+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:12.877271+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:12.877271+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.877271+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:29.229183+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.902199+0000","last_change":"2024-09-19T11:27:13.902118+0000","last_active":"2024-09-19T11:27:13.902199+0000","last_peered":"2024-09-19T11:27:13.902199+0000","last_clean":"2024-09-19T11:27:13.902199+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:27:13.902199+0000","last_undegraded":"2024-09-19T11:27:13.902199+0000","last_fullsized":"2024-09-19T11:27:13.902199+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:27:13.902022+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:27:13.902022+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.902022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:10.843373+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":109,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.028189+0000","last_change":"2024-09-19T11:26:42.028189+0000","last_active":"2024-09-19T11:26:42.028189+0000","last_peered":"2024-09-19T11:26:42.028189+0000","last_clean":"2024-09-19T11:26:42.028189+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:42.028189+0000","last_undegraded":"2024-09-19T11:26:42.028189+0000","last_fullsized":"2024-09-19T11:26:42.028189+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'23","last_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_deep_scrub":"33'23","last_deep_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.028101+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.215685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.225653+0000","last_change":"2024-09-19T11:26:29.225653+0000","last_active":"2024-09-19T11:26:29.225653+0000","last_peered":"2024-09-19T11:26:29.225653+0000","last_clean":"2024-09-19T11:26:29.225653+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:26:29.225653+0000","last_undegraded":"2024-09-19T11:26:29.225653+0000","last_fullsized":"2024-09-19T11:26:29.225653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.225549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:47.310948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'7","reported_seq":430,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.565040+0000","last_change":"2024-09-19T11:27:16.565040+0000","last_active":"2024-09-19T11:27:16.565040+0000","last_peered":"2024-09-19T11:27:16.565040+0000","last_clean":"2024-09-19T11:27:16.565040+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:27:16.565040+0000","last_undegraded":"2024-09-19T11:27:16.565040+0000","last_fullsized":"2024-09-19T11:27:16.565040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:27:16.564928+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:27:16.564928+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.564928+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:52:13.697680+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":586,"num_read_kb":586,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'49","reported_seq":108,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.884406+0000","last_change":"2024-09-19T11:27:06.884406+0000","last_active":"2024-09-19T11:27:06.884406+0000","last_peered":"2024-09-19T11:27:06.884406+0000","last_clean":"2024-09-19T11:27:06.884406+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:27:06.884406+0000","last_undegraded":"2024-09-19T11:27:06.884406+0000","last_fullsized":"2024-09-19T11:27:06.884406+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'49","last_scrub_stamp":"2024-09-19T11:27:06.884296+0000","last_deep_scrub":"33'49","last_deep_scrub_stamp":"2024-09-19T11:27:06.884296+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.884296+0000","objects_scrubbed":8,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:16.451590+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.819603+0000","last_change":"2024-09-19T11:27:08.819603+0000","last_active":"2024-09-19T11:27:08.819603+0000","last_peered":"2024-09-19T11:27:08.819603+0000","last_clean":"2024-09-19T11:27:08.819603+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:27:08.819603+0000","last_undegraded":"2024-09-19T11:27:08.819603+0000","last_fullsized":"2024-09-19T11:27:08.819603+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:08.819506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:08.819506+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.819506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:54.217393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'7","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.938455+0000","last_change":"2024-09-19T11:27:16.938455+0000","last_active":"2024-09-19T11:27:16.938455+0000","last_peered":"2024-09-19T11:27:16.938455+0000","last_clean":"2024-09-19T11:27:16.938455+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:27:16.938455+0000","last_undegraded":"2024-09-19T11:27:16.938455+0000","last_fullsized":"2024-09-19T11:27:16.938455+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:27:16.938366+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:27:16.938366+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.938366+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:08.561118+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'8","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.369733+0000","last_change":"2024-09-19T11:26:52.369670+0000","last_active":"2024-09-19T11:26:52.369733+0000","last_peered":"2024-09-19T11:26:52.369733+0000","last_clean":"2024-09-19T11:26:52.369733+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:52.369733+0000","last_undegraded":"2024-09-19T11:26:52.369733+0000","last_fullsized":"2024-09-19T11:26:52.369733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.369580+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:43.275474+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"33'74","reported_seq":157,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.856490+0000","last_change":"2024-09-19T11:26:23.856409+0000","last_active":"2024-09-19T11:26:23.856490+0000","last_peered":"2024-09-19T11:26:23.856490+0000","last_clean":"2024-09-19T11:26:23.856490+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:26:23.856490+0000","last_undegraded":"2024-09-19T11:26:23.856490+0000","last_fullsized":"2024-09-19T11:26:23.856490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'74","last_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_deep_scrub":"33'74","last_deep_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.856290+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:02.254545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'2","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.143039+0000","last_change":"2024-09-19T11:26:27.142962+0000","last_active":"2024-09-19T11:26:27.143039+0000","last_peered":"2024-09-19T11:26:27.143039+0000","last_clean":"2024-09-19T11:26:27.143039+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:26:27.143039+0000","last_undegraded":"2024-09-19T11:26:27.143039+0000","last_fullsized":"2024-09-19T11:26:27.143039+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.142873+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.914383+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.274051+0000","last_change":"2024-09-19T11:26:30.273990+0000","last_active":"2024-09-19T11:26:30.274051+0000","last_peered":"2024-09-19T11:26:30.274051+0000","last_clean":"2024-09-19T11:26:30.274051+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:26:30.274051+0000","last_undegraded":"2024-09-19T11:26:30.274051+0000","last_fullsized":"2024-09-19T11:26:30.274051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.273902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:45.300652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.985765+0000","last_change":"2024-09-19T11:26:56.985644+0000","last_active":"2024-09-19T11:26:56.985765+0000","last_peered":"2024-09-19T11:26:56.985765+0000","last_clean":"2024-09-19T11:26:56.985765+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:26:56.985765+0000","last_undegraded":"2024-09-19T11:26:56.985765+0000","last_fullsized":"2024-09-19T11:26:56.985765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.985544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:47.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'90","reported_seq":180,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.543099+0000","last_change":"2024-09-19T11:27:20.543099+0000","last_active":"2024-09-19T11:27:20.543099+0000","last_peered":"2024-09-19T11:27:20.543099+0000","last_clean":"2024-09-19T11:27:20.543099+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:27:20.543099+0000","last_undegraded":"2024-09-19T11:27:20.543099+0000","last_fullsized":"2024-09-19T11:27:20.543099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'90","last_scrub_stamp":"2024-09-19T11:27:20.542999+0000","last_deep_scrub":"33'90","last_deep_scrub_stamp":"2024-09-19T11:27:20.542999+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.542999+0000","objects_scrubbed":5,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:14.545112+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.736634+0000","last_change":"2024-09-19T11:27:07.736634+0000","last_active":"2024-09-19T11:27:07.736634+0000","last_peered":"2024-09-19T11:27:07.736634+0000","last_clean":"2024-09-19T11:27:07.736634+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:27:07.736634+0000","last_undegraded":"2024-09-19T11:27:07.736634+0000","last_fullsized":"2024-09-19T11:27:07.736634+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.736538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.736538+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.736538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:27.768987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":171,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.953879+0000","last_change":"2024-09-19T11:27:15.953879+0000","last_active":"2024-09-19T11:27:15.953879+0000","last_peered":"2024-09-19T11:27:15.953879+0000","last_clean":"2024-09-19T11:27:15.953879+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:27:15.953879+0000","last_undegraded":"2024-09-19T11:27:15.953879+0000","last_fullsized":"2024-09-19T11:27:15.953879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'81","last_scrub_stamp":"2024-09-19T11:27:15.953765+0000","last_deep_scrub":"33'81","last_deep_scrub_stamp":"2024-09-19T11:27:15.953765+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.953765+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:43.808777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.415515+0000","last_change":"2024-09-19T11:26:54.415515+0000","last_active":"2024-09-19T11:26:54.415515+0000","last_peered":"2024-09-19T11:26:54.415515+0000","last_clean":"2024-09-19T11:26:54.415515+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:26:54.415515+0000","last_undegraded":"2024-09-19T11:26:54.415515+0000","last_fullsized":"2024-09-19T11:26:54.415515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.415426+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:50.714030+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.079877+0000","last_change":"2024-09-19T11:26:24.079800+0000","last_active":"2024-09-19T11:26:24.079877+0000","last_peered":"2024-09-19T11:26:24.079877+0000","last_clean":"2024-09-19T11:26:24.079877+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:26:24.079877+0000","last_undegraded":"2024-09-19T11:26:24.079877+0000","last_fullsized":"2024-09-19T11:26:24.079877+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.079699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:48:18.789176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.923570+0000","last_change":"2024-09-19T11:26:25.923493+0000","last_active":"2024-09-19T11:26:25.923570+0000","last_peered":"2024-09-19T11:26:25.923570+0000","last_clean":"2024-09-19T11:26:25.923570+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:26:25.923570+0000","last_undegraded":"2024-09-19T11:26:25.923570+0000","last_fullsized":"2024-09-19T11:26:25.923570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.923407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:56.735995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.613049+0000","last_change":"2024-09-19T11:27:13.613049+0000","last_active":"2024-09-19T11:27:13.613049+0000","last_peered":"2024-09-19T11:27:13.613049+0000","last_clean":"2024-09-19T11:27:13.613049+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:27:13.613049+0000","last_undegraded":"2024-09-19T11:27:13.613049+0000","last_fullsized":"2024-09-19T11:27:13.613049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:13.612955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:13.612955+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.612955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:02.028356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'294","reported_seq":520,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.901143+0000","last_change":"2024-09-19T11:27:05.901072+0000","last_active":"2024-09-19T11:27:05.901143+0000","last_peered":"2024-09-19T11:27:05.901143+0000","last_clean":"2024-09-19T11:27:05.901143+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:27:05.901143+0000","last_undegraded":"2024-09-19T11:27:05.901143+0000","last_fullsized":"2024-09-19T11:27:05.901143+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'294","last_scrub_stamp":"2024-09-19T11:27:05.900953+0000","last_deep_scrub":"33'294","last_deep_scrub_stamp":"2024-09-19T11:27:05.900953+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.900953+0000","objects_scrubbed":10,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:53.378733+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:16.810146+0000","last_change":"2024-09-19T11:23:11.270807+0000","last_active":"2024-09-19T11:25:16.810146+0000","last_peered":"2024-09-19T11:25:16.810146+0000","last_clean":"2024-09-19T11:25:16.810146+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:25:16.810146+0000","last_undegraded":"2024-09-19T11:25:16.810146+0000","last_fullsized":"2024-09-19T11:25:16.810146+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.945189+0000","last_change":"2024-09-19T11:27:19.945189+0000","last_active":"2024-09-19T11:27:19.945189+0000","last_peered":"2024-09-19T11:27:19.945189+0000","last_clean":"2024-09-19T11:27:19.945189+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:27:19.945189+0000","last_undegraded":"2024-09-19T11:27:19.945189+0000","last_fullsized":"2024-09-19T11:27:19.945189+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:27:19.945103+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:27:19.945103+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.945103+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:34.266283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.967834+0000","last_change":"2024-09-19T11:26:39.967834+0000","last_active":"2024-09-19T11:26:39.967834+0000","last_peered":"2024-09-19T11:26:39.967834+0000","last_clean":"2024-09-19T11:26:39.967834+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:39.967834+0000","last_undegraded":"2024-09-19T11:26:39.967834+0000","last_fullsized":"2024-09-19T11:26:39.967834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.967748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:48.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.952580+0000","last_change":"2024-09-19T11:26:23.952524+0000","last_active":"2024-09-19T11:26:23.952580+0000","last_peered":"2024-09-19T11:26:23.952580+0000","last_clean":"2024-09-19T11:26:23.952580+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:26:23.952580+0000","last_undegraded":"2024-09-19T11:26:23.952580+0000","last_fullsized":"2024-09-19T11:26:23.952580+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.952436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:33.569870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.922891+0000","last_change":"2024-09-19T11:26:24.922822+0000","last_active":"2024-09-19T11:26:24.922891+0000","last_peered":"2024-09-19T11:26:24.922891+0000","last_clean":"2024-09-19T11:26:24.922891+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:26:24.922891+0000","last_undegraded":"2024-09-19T11:26:24.922891+0000","last_fullsized":"2024-09-19T11:26:24.922891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.922736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:47.473806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"33'2","reported_seq":40,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.833236+0000","last_change":"2024-09-19T11:26:53.833236+0000","last_active":"2024-09-19T11:26:53.833236+0000","last_peered":"2024-09-19T11:26:53.833236+0000","last_clean":"2024-09-19T11:26:53.833236+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:26:53.833236+0000","last_undegraded":"2024-09-19T11:26:53.833236+0000","last_fullsized":"2024-09-19T11:26:53.833236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.833148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:46.323076+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'24","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.835137+0000","last_change":"2024-09-19T11:26:59.835062+0000","last_active":"2024-09-19T11:26:59.835137+0000","last_peered":"2024-09-19T11:26:59.835137+0000","last_clean":"2024-09-19T11:26:59.835137+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:59.835137+0000","last_undegraded":"2024-09-19T11:26:59.835137+0000","last_fullsized":"2024-09-19T11:26:59.835137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'24","last_scrub_stamp":"2024-09-19T11:26:59.834946+0000","last_deep_scrub":"33'24","last_deep_scrub_stamp":"2024-09-19T11:26:59.834946+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.834946+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:40.523084+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.753997+0000","last_change":"2024-09-19T11:27:06.753941+0000","last_active":"2024-09-19T11:27:06.753997+0000","last_peered":"2024-09-19T11:27:06.753997+0000","last_clean":"2024-09-19T11:27:06.753997+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:27:06.753997+0000","last_undegraded":"2024-09-19T11:27:06.753997+0000","last_fullsized":"2024-09-19T11:27:06.753997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:06.753841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:06.753841+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.753841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:57.677312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943722+0000","last_change":"2024-09-19T11:23:13.273373+0000","last_active":"2024-09-19T11:23:42.943722+0000","last_peered":"2024-09-19T11:23:42.943722+0000","last_clean":"2024-09-19T11:23:42.943722+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:23:42.943722+0000","last_undegraded":"2024-09-19T11:23:42.943722+0000","last_fullsized":"2024-09-19T11:23:42.943722+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:11:13.436131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":68,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.977214+0000","last_change":"2024-09-19T11:26:37.977150+0000","last_active":"2024-09-19T11:26:37.977214+0000","last_peered":"2024-09-19T11:26:37.977214+0000","last_clean":"2024-09-19T11:26:37.977214+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:37.977214+0000","last_undegraded":"2024-09-19T11:26:37.977214+0000","last_fullsized":"2024-09-19T11:26:37.977214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.977048+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:20.097270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"33'3","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.109984+0000","last_change":"2024-09-19T11:26:25.109916+0000","last_active":"2024-09-19T11:26:25.109984+0000","last_peered":"2024-09-19T11:26:25.109984+0000","last_clean":"2024-09-19T11:26:25.109984+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:26:25.109984+0000","last_undegraded":"2024-09-19T11:26:25.109984+0000","last_fullsized":"2024-09-19T11:26:25.109984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.109824+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:48.020763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.938101+0000","last_change":"2024-09-19T11:26:25.938045+0000","last_active":"2024-09-19T11:26:25.938101+0000","last_peered":"2024-09-19T11:26:25.938101+0000","last_clean":"2024-09-19T11:26:25.938101+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:25.938101+0000","last_undegraded":"2024-09-19T11:26:25.938101+0000","last_fullsized":"2024-09-19T11:26:25.938101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.937963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:58:57.848583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.801326+0000","last_change":"2024-09-19T11:26:56.801257+0000","last_active":"2024-09-19T11:26:56.801326+0000","last_peered":"2024-09-19T11:26:56.801326+0000","last_clean":"2024-09-19T11:26:56.801326+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:26:56.801326+0000","last_undegraded":"2024-09-19T11:26:56.801326+0000","last_fullsized":"2024-09-19T11:26:56.801326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.801170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:01.667240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'20","reported_seq":62,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.527663+0000","last_change":"2024-09-19T11:27:19.527663+0000","last_active":"2024-09-19T11:27:19.527663+0000","last_peered":"2024-09-19T11:27:19.527663+0000","last_clean":"2024-09-19T11:27:19.527663+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:27:19.527663+0000","last_undegraded":"2024-09-19T11:27:19.527663+0000","last_fullsized":"2024-09-19T11:27:19.527663+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'20","last_scrub_stamp":"2024-09-19T11:27:19.527564+0000","last_deep_scrub":"33'20","last_deep_scrub_stamp":"2024-09-19T11:27:19.527564+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.527564+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:21.167261+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.947826+0000","last_change":"2024-09-19T11:23:11.271073+0000","last_active":"2024-09-19T11:23:42.947826+0000","last_peered":"2024-09-19T11:23:42.947826+0000","last_clean":"2024-09-19T11:23:42.947826+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:23:42.947826+0000","last_undegraded":"2024-09-19T11:23:42.947826+0000","last_fullsized":"2024-09-19T11:23:42.947826+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:52.079937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.934889+0000","last_change":"2024-09-19T11:27:18.934889+0000","last_active":"2024-09-19T11:27:18.934889+0000","last_peered":"2024-09-19T11:27:18.934889+0000","last_clean":"2024-09-19T11:27:18.934889+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:27:18.934889+0000","last_undegraded":"2024-09-19T11:27:18.934889+0000","last_fullsized":"2024-09-19T11:27:18.934889+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:18.934732+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:18.934732+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.934732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:23.688618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.916131+0000","last_change":"2024-09-19T11:26:35.916064+0000","last_active":"2024-09-19T11:26:35.916131+0000","last_peered":"2024-09-19T11:26:35.916131+0000","last_clean":"2024-09-19T11:26:35.916131+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:35.916131+0000","last_undegraded":"2024-09-19T11:26:35.916131+0000","last_fullsized":"2024-09-19T11:26:35.916131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.915972+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:33.729135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.933807+0000","last_change":"2024-09-19T11:26:24.933741+0000","last_active":"2024-09-19T11:26:24.933807+0000","last_peered":"2024-09-19T11:26:24.933807+0000","last_clean":"2024-09-19T11:26:24.933807+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:26:24.933807+0000","last_undegraded":"2024-09-19T11:26:24.933807+0000","last_fullsized":"2024-09-19T11:26:24.933807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.933645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.620054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.910643+0000","last_change":"2024-09-19T11:26:22.910571+0000","last_active":"2024-09-19T11:26:22.910643+0000","last_peered":"2024-09-19T11:26:22.910643+0000","last_clean":"2024-09-19T11:26:22.910643+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:26:22.910643+0000","last_undegraded":"2024-09-19T11:26:22.910643+0000","last_fullsized":"2024-09-19T11:26:22.910643+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.910497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:20.068073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.866238+0000","last_change":"2024-09-19T11:26:52.866167+0000","last_active":"2024-09-19T11:26:52.866238+0000","last_peered":"2024-09-19T11:26:52.866238+0000","last_clean":"2024-09-19T11:26:52.866238+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:26:52.866238+0000","last_undegraded":"2024-09-19T11:26:52.866238+0000","last_fullsized":"2024-09-19T11:26:52.866238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.866082+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:47.631328+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'77","reported_seq":154,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.785719+0000","last_change":"2024-09-19T11:26:58.785636+0000","last_active":"2024-09-19T11:26:58.785719+0000","last_peered":"2024-09-19T11:26:58.785719+0000","last_clean":"2024-09-19T11:26:58.785719+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:26:58.785719+0000","last_undegraded":"2024-09-19T11:26:58.785719+0000","last_fullsized":"2024-09-19T11:26:58.785719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'77","last_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_deep_scrub":"33'77","last_deep_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.785526+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:40.190489+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.829172+0000","last_change":"2024-09-19T11:27:07.829111+0000","last_active":"2024-09-19T11:27:07.829172+0000","last_peered":"2024-09-19T11:27:07.829172+0000","last_clean":"2024-09-19T11:27:07.829172+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:27:07.829172+0000","last_undegraded":"2024-09-19T11:27:07.829172+0000","last_fullsized":"2024-09-19T11:27:07.829172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.829015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.829015+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.829015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:13.610649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.843376+0000","last_change":"2024-09-19T11:27:19.843376+0000","last_active":"2024-09-19T11:27:19.843376+0000","last_peered":"2024-09-19T11:27:19.843376+0000","last_clean":"2024-09-19T11:27:19.843376+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:27:19.843376+0000","last_undegraded":"2024-09-19T11:27:19.843376+0000","last_fullsized":"2024-09-19T11:27:19.843376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:27:19.843296+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:27:19.843296+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.843296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:43.970345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.952834+0000","last_change":"2024-09-19T11:26:36.952752+0000","last_active":"2024-09-19T11:26:36.952834+0000","last_peered":"2024-09-19T11:26:36.952834+0000","last_clean":"2024-09-19T11:26:36.952834+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:36.952834+0000","last_undegraded":"2024-09-19T11:26:36.952834+0000","last_fullsized":"2024-09-19T11:26:36.952834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.952660+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.496448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.969573+0000","last_change":"2024-09-19T11:26:38.969511+0000","last_active":"2024-09-19T11:26:38.969573+0000","last_peered":"2024-09-19T11:26:38.969573+0000","last_clean":"2024-09-19T11:26:38.969573+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:26:38.969573+0000","last_undegraded":"2024-09-19T11:26:38.969573+0000","last_fullsized":"2024-09-19T11:26:38.969573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.969423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:49.370950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.293541+0000","last_change":"2024-09-19T11:26:32.293468+0000","last_active":"2024-09-19T11:26:32.293541+0000","last_peered":"2024-09-19T11:26:32.293541+0000","last_clean":"2024-09-19T11:26:32.293541+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:26:32.293541+0000","last_undegraded":"2024-09-19T11:26:32.293541+0000","last_fullsized":"2024-09-19T11:26:32.293541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.293378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:48.996006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.538549+0000","last_change":"2024-09-19T11:27:10.538492+0000","last_active":"2024-09-19T11:27:10.538549+0000","last_peered":"2024-09-19T11:27:10.538549+0000","last_clean":"2024-09-19T11:27:10.538549+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:27:10.538549+0000","last_undegraded":"2024-09-19T11:27:10.538549+0000","last_fullsized":"2024-09-19T11:27:10.538549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:10.538407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:10.538407+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.538407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:33.331328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'22","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.936170+0000","last_change":"2024-09-19T11:27:04.936063+0000","last_active":"2024-09-19T11:27:04.936170+0000","last_peered":"2024-09-19T11:27:04.936170+0000","last_clean":"2024-09-19T11:27:04.936170+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:27:04.936170+0000","last_undegraded":"2024-09-19T11:27:04.936170+0000","last_fullsized":"2024-09-19T11:27:04.936170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'22","last_scrub_stamp":"2024-09-19T11:27:04.935933+0000","last_deep_scrub":"33'22","last_deep_scrub_stamp":"2024-09-19T11:27:04.935933+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.935933+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.148853+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'286","reported_seq":369,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.915549+0000","last_change":"2024-09-19T11:27:10.915484+0000","last_active":"2024-09-19T11:27:10.915549+0000","last_peered":"2024-09-19T11:27:10.915549+0000","last_clean":"2024-09-19T11:27:10.915549+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:27:10.915549+0000","last_undegraded":"2024-09-19T11:27:10.915549+0000","last_fullsized":"2024-09-19T11:27:10.915549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:10.915397+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:10.915397+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.915397+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:07.096881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946621+0000","last_change":"2024-09-19T11:23:13.269759+0000","last_active":"2024-09-19T11:23:42.946621+0000","last_peered":"2024-09-19T11:23:42.946621+0000","last_clean":"2024-09-19T11:23:42.946621+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:23:42.946621+0000","last_undegraded":"2024-09-19T11:23:42.946621+0000","last_fullsized":"2024-09-19T11:23:42.946621+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:46.820239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'12","reported_seq":79,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.381797+0000","last_change":"2024-09-19T11:26:53.381726+0000","last_active":"2024-09-19T11:26:53.381797+0000","last_peered":"2024-09-19T11:26:53.381797+0000","last_clean":"2024-09-19T11:26:53.381797+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:53.381797+0000","last_undegraded":"2024-09-19T11:26:53.381797+0000","last_fullsized":"2024-09-19T11:26:53.381797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.381618+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:56.071102+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.185387+0000","last_change":"2024-09-19T11:26:28.185387+0000","last_active":"2024-09-19T11:26:28.185387+0000","last_peered":"2024-09-19T11:26:28.185387+0000","last_clean":"2024-09-19T11:26:28.185387+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:26:28.185387+0000","last_undegraded":"2024-09-19T11:26:28.185387+0000","last_fullsized":"2024-09-19T11:26:28.185387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.185291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:48.217740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.897506+0000","last_change":"2024-09-19T11:26:23.897455+0000","last_active":"2024-09-19T11:26:23.897506+0000","last_peered":"2024-09-19T11:26:23.897506+0000","last_clean":"2024-09-19T11:26:23.897506+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:26:23.897506+0000","last_undegraded":"2024-09-19T11:26:23.897506+0000","last_fullsized":"2024-09-19T11:26:23.897506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.897373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:13.527667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.503825+0000","last_change":"2024-09-19T11:27:09.503759+0000","last_active":"2024-09-19T11:27:09.503825+0000","last_peered":"2024-09-19T11:27:09.503825+0000","last_clean":"2024-09-19T11:27:09.503825+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:27:09.503825+0000","last_undegraded":"2024-09-19T11:27:09.503825+0000","last_fullsized":"2024-09-19T11:27:09.503825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:09.503672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:09.503672+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.503672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:53.319658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"33'16","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.512281+0000","last_change":"2024-09-19T11:27:18.512211+0000","last_active":"2024-09-19T11:27:18.512281+0000","last_peered":"2024-09-19T11:27:18.512281+0000","last_clean":"2024-09-19T11:27:18.512281+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:27:18.512281+0000","last_undegraded":"2024-09-19T11:27:18.512281+0000","last_fullsized":"2024-09-19T11:27:18.512281+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:27:18.512110+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:27:18.512110+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.512110+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:13.848124+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'286","reported_seq":443,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.791846+0000","last_change":"2024-09-19T11:27:09.791720+0000","last_active":"2024-09-19T11:27:09.791846+0000","last_peered":"2024-09-19T11:27:09.791846+0000","last_clean":"2024-09-19T11:27:09.791846+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:27:09.791846+0000","last_undegraded":"2024-09-19T11:27:09.791846+0000","last_fullsized":"2024-09-19T11:27:09.791846+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:09.791625+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:09.791625+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.791625+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:24.818374+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.916756+0000","last_change":"2024-09-19T11:27:20.916756+0000","last_active":"2024-09-19T11:27:20.916756+0000","last_peered":"2024-09-19T11:27:20.916756+0000","last_clean":"2024-09-19T11:27:20.916756+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:27:20.916756+0000","last_undegraded":"2024-09-19T11:27:20.916756+0000","last_fullsized":"2024-09-19T11:27:20.916756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-09-19T11:27:20.916668+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-09-19T11:27:20.916668+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.916668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:56.260631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.850294+0000","last_change":"2024-09-19T11:26:39.850188+0000","last_active":"2024-09-19T11:26:39.850294+0000","last_peered":"2024-09-19T11:26:39.850294+0000","last_clean":"2024-09-19T11:26:39.850294+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:26:39.850294+0000","last_undegraded":"2024-09-19T11:26:39.850294+0000","last_fullsized":"2024-09-19T11:26:39.850294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.850077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:05.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.886518+0000","last_change":"2024-09-19T11:26:22.886518+0000","last_active":"2024-09-19T11:26:22.886518+0000","last_peered":"2024-09-19T11:26:22.886518+0000","last_clean":"2024-09-19T11:26:22.886518+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:26:22.886518+0000","last_undegraded":"2024-09-19T11:26:22.886518+0000","last_fullsized":"2024-09-19T11:26:22.886518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.886419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:07.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.146522+0000","last_change":"2024-09-19T11:26:26.146468+0000","last_active":"2024-09-19T11:26:26.146522+0000","last_peered":"2024-09-19T11:26:26.146522+0000","last_clean":"2024-09-19T11:26:26.146522+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:26:26.146522+0000","last_undegraded":"2024-09-19T11:26:26.146522+0000","last_fullsized":"2024-09-19T11:26:26.146522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.146376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:37.768766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.572672+0000","last_change":"2024-09-19T11:27:11.572672+0000","last_active":"2024-09-19T11:27:11.572672+0000","last_peered":"2024-09-19T11:27:11.572672+0000","last_clean":"2024-09-19T11:27:11.572672+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:27:11.572672+0000","last_undegraded":"2024-09-19T11:27:11.572672+0000","last_fullsized":"2024-09-19T11:27:11.572672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.572582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.572582+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.572582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:13.389084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'99","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.848493+0000","last_change":"2024-09-19T11:26:58.848423+0000","last_active":"2024-09-19T11:26:58.848493+0000","last_peered":"2024-09-19T11:26:58.848493+0000","last_clean":"2024-09-19T11:26:58.848493+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:26:58.848493+0000","last_undegraded":"2024-09-19T11:26:58.848493+0000","last_fullsized":"2024-09-19T11:26:58.848493+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'99","last_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_deep_scrub":"33'99","last_deep_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.848330+0000","objects_scrubbed":8,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:48.091262+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.742808+0000","last_change":"2024-09-19T11:27:05.742747+0000","last_active":"2024-09-19T11:27:05.742808+0000","last_peered":"2024-09-19T11:27:05.742808+0000","last_clean":"2024-09-19T11:27:05.742808+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:27:05.742808+0000","last_undegraded":"2024-09-19T11:27:05.742808+0000","last_fullsized":"2024-09-19T11:27:05.742808+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:05.742667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:05.742667+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.742667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.395498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.964584+0000","last_change":"2024-09-19T11:27:16.964584+0000","last_active":"2024-09-19T11:27:16.964584+0000","last_peered":"2024-09-19T11:27:16.964584+0000","last_clean":"2024-09-19T11:27:16.964584+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:27:16.964584+0000","last_undegraded":"2024-09-19T11:27:16.964584+0000","last_fullsized":"2024-09-19T11:27:16.964584+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:16.964498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:16.964498+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.964498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:58.369461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.936328+0000","last_change":"2024-09-19T11:27:18.936226+0000","last_active":"2024-09-19T11:27:18.936328+0000","last_peered":"2024-09-19T11:27:18.936328+0000","last_clean":"2024-09-19T11:27:18.936328+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:27:18.936328+0000","last_undegraded":"2024-09-19T11:27:18.936328+0000","last_fullsized":"2024-09-19T11:27:18.936328+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:18.936125+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:18.936125+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.936125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:50.662583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.771683+0000","last_change":"2024-09-19T11:26:42.771683+0000","last_active":"2024-09-19T11:26:42.771683+0000","last_peered":"2024-09-19T11:26:42.771683+0000","last_clean":"2024-09-19T11:26:42.771683+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:26:42.771683+0000","last_undegraded":"2024-09-19T11:26:42.771683+0000","last_fullsized":"2024-09-19T11:26:42.771683+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.771596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:48:05.322666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.396108+0000","last_change":"2024-09-19T11:26:38.396108+0000","last_active":"2024-09-19T11:26:38.396108+0000","last_peered":"2024-09-19T11:26:38.396108+0000","last_clean":"2024-09-19T11:26:38.396108+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:26:38.396108+0000","last_undegraded":"2024-09-19T11:26:38.396108+0000","last_fullsized":"2024-09-19T11:26:38.396108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.396006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:14.129195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.922151+0000","last_change":"2024-09-19T11:27:00.922151+0000","last_active":"2024-09-19T11:27:00.922151+0000","last_peered":"2024-09-19T11:27:00.922151+0000","last_clean":"2024-09-19T11:27:00.922151+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:27:00.922151+0000","last_undegraded":"2024-09-19T11:27:00.922151+0000","last_fullsized":"2024-09-19T11:27:00.922151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.922068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:51.324340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'35","reported_seq":83,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.220186+0000","last_change":"2024-09-19T11:23:09.259148+0000","last_active":"2024-09-19T11:26:19.220186+0000","last_peered":"2024-09-19T11:26:19.220186+0000","last_clean":"2024-09-19T11:26:19.220186+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:26:19.220186+0000","last_undegraded":"2024-09-19T11:26:19.220186+0000","last_fullsized":"2024-09-19T11:26:19.220186+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:25:29.000573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.945264+0000","last_change":"2024-09-19T11:23:11.271128+0000","last_active":"2024-09-19T11:23:42.945264+0000","last_peered":"2024-09-19T11:23:42.945264+0000","last_clean":"2024-09-19T11:23:42.945264+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:23:42.945264+0000","last_undegraded":"2024-09-19T11:23:42.945264+0000","last_fullsized":"2024-09-19T11:23:42.945264+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:35.425421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:15.806724+0000","last_change":"2024-09-19T11:23:13.273576+0000","last_active":"2024-09-19T11:25:15.806724+0000","last_peered":"2024-09-19T11:25:15.806724+0000","last_clean":"2024-09-19T11:25:15.806724+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:25:15.806724+0000","last_undegraded":"2024-09-19T11:25:15.806724+0000","last_fullsized":"2024-09-19T11:25:15.806724+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.036281+0000","last_change":"2024-09-19T11:26:41.036281+0000","last_active":"2024-09-19T11:26:41.036281+0000","last_peered":"2024-09-19T11:26:41.036281+0000","last_clean":"2024-09-19T11:26:41.036281+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:26:41.036281+0000","last_undegraded":"2024-09-19T11:26:41.036281+0000","last_fullsized":"2024-09-19T11:26:41.036281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.036169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:34.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.921998+0000","last_change":"2024-09-19T11:26:28.921998+0000","last_active":"2024-09-19T11:26:28.921998+0000","last_peered":"2024-09-19T11:26:28.921998+0000","last_clean":"2024-09-19T11:26:28.921998+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:26:28.921998+0000","last_undegraded":"2024-09-19T11:26:28.921998+0000","last_fullsized":"2024-09-19T11:26:28.921998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.921906+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:00.637589+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.583167+0000","last_change":"2024-09-19T11:27:14.583167+0000","last_active":"2024-09-19T11:27:14.583167+0000","last_peered":"2024-09-19T11:27:14.583167+0000","last_clean":"2024-09-19T11:27:14.583167+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:27:14.583167+0000","last_undegraded":"2024-09-19T11:27:14.583167+0000","last_fullsized":"2024-09-19T11:27:14.583167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.583070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.583070+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.583070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:47.425828+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'8","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.727321+0000","last_change":"2024-09-19T11:27:01.727321+0000","last_active":"2024-09-19T11:27:01.727321+0000","last_peered":"2024-09-19T11:27:01.727321+0000","last_clean":"2024-09-19T11:27:01.727321+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:27:01.727321+0000","last_undegraded":"2024-09-19T11:27:01.727321+0000","last_fullsized":"2024-09-19T11:27:01.727321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:01.727234+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:01.727234+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.727234+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:28.373856+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.813741+0000","last_change":"2024-09-19T11:27:09.813741+0000","last_active":"2024-09-19T11:27:09.813741+0000","last_peered":"2024-09-19T11:27:09.813741+0000","last_clean":"2024-09-19T11:27:09.813741+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:27:09.813741+0000","last_undegraded":"2024-09-19T11:27:09.813741+0000","last_fullsized":"2024-09-19T11:27:09.813741+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:09.813641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:09.813641+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.813641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:52:43.359127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.279462+0000","last_change":"2024-09-19T11:23:13.267238+0000","last_active":"2024-09-19T11:23:42.279462+0000","last_peered":"2024-09-19T11:23:42.279462+0000","last_clean":"2024-09-19T11:23:42.279462+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:23:42.279462+0000","last_undegraded":"2024-09-19T11:23:42.279462+0000","last_fullsized":"2024-09-19T11:23:42.279462+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:36.252981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.001963+0000","last_change":"2024-09-19T11:26:41.001963+0000","last_active":"2024-09-19T11:26:41.001963+0000","last_peered":"2024-09-19T11:26:41.001963+0000","last_clean":"2024-09-19T11:26:41.001963+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:41.001963+0000","last_undegraded":"2024-09-19T11:26:41.001963+0000","last_fullsized":"2024-09-19T11:26:41.001963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.001858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:06.599582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.937976+0000","last_change":"2024-09-19T11:26:29.937976+0000","last_active":"2024-09-19T11:26:29.937976+0000","last_peered":"2024-09-19T11:26:29.937976+0000","last_clean":"2024-09-19T11:26:29.937976+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:29.937976+0000","last_undegraded":"2024-09-19T11:26:29.937976+0000","last_fullsized":"2024-09-19T11:26:29.937976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.937882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:14.058304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.813274+0000","last_change":"2024-09-19T11:26:51.813274+0000","last_active":"2024-09-19T11:26:51.813274+0000","last_peered":"2024-09-19T11:26:51.813274+0000","last_clean":"2024-09-19T11:26:51.813274+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:26:51.813274+0000","last_undegraded":"2024-09-19T11:26:51.813274+0000","last_fullsized":"2024-09-19T11:26:51.813274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.813187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:32.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'49","reported_seq":121,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.759116+0000","last_change":"2024-09-19T11:27:00.759116+0000","last_active":"2024-09-19T11:27:00.759116+0000","last_peered":"2024-09-19T11:27:00.759116+0000","last_clean":"2024-09-19T11:27:00.759116+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:27:00.759116+0000","last_undegraded":"2024-09-19T11:27:00.759116+0000","last_fullsized":"2024-09-19T11:27:00.759116+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'49","last_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_deep_scrub":"33'49","last_deep_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.759010+0000","objects_scrubbed":6,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:16.347162+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.892506+0000","last_change":"2024-09-19T11:27:11.892506+0000","last_active":"2024-09-19T11:27:11.892506+0000","last_peered":"2024-09-19T11:27:11.892506+0000","last_clean":"2024-09-19T11:27:11.892506+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:27:11.892506+0000","last_undegraded":"2024-09-19T11:27:11.892506+0000","last_fullsized":"2024-09-19T11:27:11.892506+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.892416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.892416+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.892416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:56:17.528967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.392197+0000","last_change":"2024-09-19T11:26:58.392197+0000","last_active":"2024-09-19T11:26:58.392197+0000","last_peered":"2024-09-19T11:26:58.392197+0000","last_clean":"2024-09-19T11:26:58.392197+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:26:58.392197+0000","last_undegraded":"2024-09-19T11:26:58.392197+0000","last_fullsized":"2024-09-19T11:26:58.392197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.392102+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:10.174526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.886507+0000","last_change":"2024-09-19T11:26:25.886507+0000","last_active":"2024-09-19T11:26:25.886507+0000","last_peered":"2024-09-19T11:26:25.886507+0000","last_clean":"2024-09-19T11:26:25.886507+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:26:25.886507+0000","last_undegraded":"2024-09-19T11:26:25.886507+0000","last_fullsized":"2024-09-19T11:26:25.886507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.886414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:38.573945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.584986+0000","last_change":"2024-09-19T11:27:12.584986+0000","last_active":"2024-09-19T11:27:12.584986+0000","last_peered":"2024-09-19T11:27:12.584986+0000","last_clean":"2024-09-19T11:27:12.584986+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:27:12.584986+0000","last_undegraded":"2024-09-19T11:27:12.584986+0000","last_fullsized":"2024-09-19T11:27:12.584986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:12.584897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:12.584897+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.584897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:55.169943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'38","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.810233+0000","last_change":"2024-09-19T11:27:03.810233+0000","last_active":"2024-09-19T11:27:03.810233+0000","last_peered":"2024-09-19T11:27:03.810233+0000","last_clean":"2024-09-19T11:27:03.810233+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:27:03.810233+0000","last_undegraded":"2024-09-19T11:27:03.810233+0000","last_fullsized":"2024-09-19T11:27:03.810233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:03.810143+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:03.810143+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.810143+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:31.289316+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.822230+0000","last_change":"2024-09-19T11:27:11.822230+0000","last_active":"2024-09-19T11:27:11.822230+0000","last_peered":"2024-09-19T11:27:11.822230+0000","last_clean":"2024-09-19T11:27:11.822230+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:27:11.822230+0000","last_undegraded":"2024-09-19T11:27:11.822230+0000","last_fullsized":"2024-09-19T11:27:11.822230+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.822138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.822138+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.822138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:55.083098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.486672+0000","last_change":"2024-09-19T11:27:01.486612+0000","last_active":"2024-09-19T11:27:01.486672+0000","last_peered":"2024-09-19T11:27:01.486672+0000","last_clean":"2024-09-19T11:27:01.486672+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:27:01.486672+0000","last_undegraded":"2024-09-19T11:27:01.486672+0000","last_fullsized":"2024-09-19T11:27:01.486672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.486507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:11.419750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'4","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.386729+0000","last_change":"2024-09-19T11:26:39.386729+0000","last_active":"2024-09-19T11:26:39.386729+0000","last_peered":"2024-09-19T11:26:39.386729+0000","last_clean":"2024-09-19T11:26:39.386729+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:26:39.386729+0000","last_undegraded":"2024-09-19T11:26:39.386729+0000","last_fullsized":"2024-09-19T11:26:39.386729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.386636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:26.742196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.840003+0000","last_change":"2024-09-19T11:26:55.840003+0000","last_active":"2024-09-19T11:26:55.840003+0000","last_peered":"2024-09-19T11:26:55.840003+0000","last_clean":"2024-09-19T11:26:55.840003+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:26:55.840003+0000","last_undegraded":"2024-09-19T11:26:55.840003+0000","last_fullsized":"2024-09-19T11:26:55.840003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.241449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'67","reported_seq":140,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.770359+0000","last_change":"2024-09-19T11:27:03.770359+0000","last_active":"2024-09-19T11:27:03.770359+0000","last_peered":"2024-09-19T11:27:03.770359+0000","last_clean":"2024-09-19T11:27:03.770359+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:27:03.770359+0000","last_undegraded":"2024-09-19T11:27:03.770359+0000","last_fullsized":"2024-09-19T11:27:03.770359+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'67","last_scrub_stamp":"2024-09-19T11:27:03.770268+0000","last_deep_scrub":"33'67","last_deep_scrub_stamp":"2024-09-19T11:27:03.770268+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.770268+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.209518+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":65,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.859547+0000","last_change":"2024-09-19T11:27:10.859547+0000","last_active":"2024-09-19T11:27:10.859547+0000","last_peered":"2024-09-19T11:27:10.859547+0000","last_clean":"2024-09-19T11:27:10.859547+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:27:10.859547+0000","last_undegraded":"2024-09-19T11:27:10.859547+0000","last_fullsized":"2024-09-19T11:27:10.859547+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:10.859446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:10.859446+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.859446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:44.106754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.419187+0000","last_change":"2024-09-19T11:23:13.268105+0000","last_active":"2024-09-19T11:25:11.419187+0000","last_peered":"2024-09-19T11:25:11.419187+0000","last_clean":"2024-09-19T11:25:11.419187+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:25:11.419187+0000","last_undegraded":"2024-09-19T11:25:11.419187+0000","last_fullsized":"2024-09-19T11:25:11.419187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:14:21.113571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.954173+0000","last_change":"2024-09-19T11:26:43.954113+0000","last_active":"2024-09-19T11:26:43.954173+0000","last_peered":"2024-09-19T11:26:43.954173+0000","last_clean":"2024-09-19T11:26:43.954173+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:43.954173+0000","last_undegraded":"2024-09-19T11:26:43.954173+0000","last_fullsized":"2024-09-19T11:26:43.954173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.954028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:24.870979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.929509+0000","last_change":"2024-09-19T11:26:26.929509+0000","last_active":"2024-09-19T11:26:26.929509+0000","last_peered":"2024-09-19T11:26:26.929509+0000","last_clean":"2024-09-19T11:26:26.929509+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:26:26.929509+0000","last_undegraded":"2024-09-19T11:26:26.929509+0000","last_fullsized":"2024-09-19T11:26:26.929509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.929422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:53.997533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.861748+0000","last_change":"2024-09-19T11:27:00.861682+0000","last_active":"2024-09-19T11:27:00.861748+0000","last_peered":"2024-09-19T11:27:00.861748+0000","last_clean":"2024-09-19T11:27:00.861748+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:27:00.861748+0000","last_undegraded":"2024-09-19T11:27:00.861748+0000","last_fullsized":"2024-09-19T11:27:00.861748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.861599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.861599+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.861599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:41.778551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'39","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.179930+0000","last_change":"2024-09-19T11:23:09.256642+0000","last_active":"2024-09-19T11:26:19.179930+0000","last_peered":"2024-09-19T11:26:19.179930+0000","last_clean":"2024-09-19T11:26:19.179930+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:26:19.179930+0000","last_undegraded":"2024-09-19T11:26:19.179930+0000","last_fullsized":"2024-09-19T11:26:19.179930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:42.797325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.953402+0000","last_change":"2024-09-19T11:27:15.953402+0000","last_active":"2024-09-19T11:27:15.953402+0000","last_peered":"2024-09-19T11:27:15.953402+0000","last_clean":"2024-09-19T11:27:15.953402+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:27:15.953402+0000","last_undegraded":"2024-09-19T11:27:15.953402+0000","last_fullsized":"2024-09-19T11:27:15.953402+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.953297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.953297+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.953297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:28.014945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.474660+0000","last_change":"2024-09-19T11:27:00.474660+0000","last_active":"2024-09-19T11:27:00.474660+0000","last_peered":"2024-09-19T11:27:00.474660+0000","last_clean":"2024-09-19T11:27:00.474660+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:27:00.474660+0000","last_undegraded":"2024-09-19T11:27:00.474660+0000","last_fullsized":"2024-09-19T11:27:00.474660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.474573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:59.322724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.877728+0000","last_change":"2024-09-19T11:26:28.877728+0000","last_active":"2024-09-19T11:26:28.877728+0000","last_peered":"2024-09-19T11:26:28.877728+0000","last_clean":"2024-09-19T11:26:28.877728+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:26:28.877728+0000","last_undegraded":"2024-09-19T11:26:28.877728+0000","last_fullsized":"2024-09-19T11:26:28.877728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.877640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:09.086406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.801547+0000","last_change":"2024-09-19T11:26:57.801547+0000","last_active":"2024-09-19T11:26:57.801547+0000","last_peered":"2024-09-19T11:26:57.801547+0000","last_clean":"2024-09-19T11:26:57.801547+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:26:57.801547+0000","last_undegraded":"2024-09-19T11:26:57.801547+0000","last_fullsized":"2024-09-19T11:26:57.801547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.801457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:23.399181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'42","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.773714+0000","last_change":"2024-09-19T11:27:04.773660+0000","last_active":"2024-09-19T11:27:04.773714+0000","last_peered":"2024-09-19T11:27:04.773714+0000","last_clean":"2024-09-19T11:27:04.773714+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:27:04.773714+0000","last_undegraded":"2024-09-19T11:27:04.773714+0000","last_fullsized":"2024-09-19T11:27:04.773714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'42","last_scrub_stamp":"2024-09-19T11:27:04.773575+0000","last_deep_scrub":"33'42","last_deep_scrub_stamp":"2024-09-19T11:27:04.773575+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.773575+0000","objects_scrubbed":5,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:44.880940+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.829408+0000","last_change":"2024-09-19T11:27:16.829408+0000","last_active":"2024-09-19T11:27:16.829408+0000","last_peered":"2024-09-19T11:27:16.829408+0000","last_clean":"2024-09-19T11:27:16.829408+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:27:16.829408+0000","last_undegraded":"2024-09-19T11:27:16.829408+0000","last_fullsized":"2024-09-19T11:27:16.829408+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:16.829319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:16.829319+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.829319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:59.496334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.000020+0000","last_change":"2024-09-19T11:26:43.000020+0000","last_active":"2024-09-19T11:26:43.000020+0000","last_peered":"2024-09-19T11:26:43.000020+0000","last_clean":"2024-09-19T11:26:43.000020+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:26:43.000020+0000","last_undegraded":"2024-09-19T11:26:43.000020+0000","last_fullsized":"2024-09-19T11:26:43.000020+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.999915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:23.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.342895+0000","last_change":"2024-09-19T11:26:40.342895+0000","last_active":"2024-09-19T11:26:40.342895+0000","last_peered":"2024-09-19T11:26:40.342895+0000","last_clean":"2024-09-19T11:26:40.342895+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:26:40.342895+0000","last_undegraded":"2024-09-19T11:26:40.342895+0000","last_fullsized":"2024-09-19T11:26:40.342895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.342808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.126397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.896501+0000","last_change":"2024-09-19T11:27:03.896501+0000","last_active":"2024-09-19T11:27:03.896501+0000","last_peered":"2024-09-19T11:27:03.896501+0000","last_clean":"2024-09-19T11:27:03.896501+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:27:03.896501+0000","last_undegraded":"2024-09-19T11:27:03.896501+0000","last_fullsized":"2024-09-19T11:27:03.896501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:03.896413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:03.896413+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.896413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:17.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"31'10","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.809129+0000","last_change":"2024-09-19T11:23:09.257194+0000","last_active":"2024-09-19T11:25:11.809129+0000","last_peered":"2024-09-19T11:25:11.809129+0000","last_clean":"2024-09-19T11:25:11.809129+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:25:11.809129+0000","last_undegraded":"2024-09-19T11:25:11.809129+0000","last_fullsized":"2024-09-19T11:25:11.809129+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'286","reported_seq":431,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.871742+0000","last_change":"2024-09-19T11:27:12.871624+0000","last_active":"2024-09-19T11:27:12.871742+0000","last_peered":"2024-09-19T11:27:12.871742+0000","last_clean":"2024-09-19T11:27:12.871742+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:27:12.871742+0000","last_undegraded":"2024-09-19T11:27:12.871742+0000","last_fullsized":"2024-09-19T11:27:12.871742+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:12.871525+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:12.871525+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.871525+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:18.160862+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943313+0000","last_change":"2024-09-19T11:23:13.272264+0000","last_active":"2024-09-19T11:23:42.943313+0000","last_peered":"2024-09-19T11:23:42.943313+0000","last_clean":"2024-09-19T11:23:42.943313+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:23:42.943313+0000","last_undegraded":"2024-09-19T11:23:42.943313+0000","last_fullsized":"2024-09-19T11:23:42.943313+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:34.821219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.773400+0000","last_change":"2024-09-19T11:26:47.773400+0000","last_active":"2024-09-19T11:26:47.773400+0000","last_peered":"2024-09-19T11:26:47.773400+0000","last_clean":"2024-09-19T11:26:47.773400+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:47.773400+0000","last_undegraded":"2024-09-19T11:26:47.773400+0000","last_fullsized":"2024-09-19T11:26:47.773400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.773301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:48.002135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.907839+0000","last_change":"2024-09-19T11:26:27.907839+0000","last_active":"2024-09-19T11:26:27.907839+0000","last_peered":"2024-09-19T11:26:27.907839+0000","last_clean":"2024-09-19T11:26:27.907839+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:26:27.907839+0000","last_undegraded":"2024-09-19T11:26:27.907839+0000","last_fullsized":"2024-09-19T11:26:27.907839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.907726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:48.160182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.884954+0000","last_change":"2024-09-19T11:27:02.884954+0000","last_active":"2024-09-19T11:27:02.884954+0000","last_peered":"2024-09-19T11:27:02.884954+0000","last_clean":"2024-09-19T11:27:02.884954+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:27:02.884954+0000","last_undegraded":"2024-09-19T11:27:02.884954+0000","last_fullsized":"2024-09-19T11:27:02.884954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:02.884869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:02.884869+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.884869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:00.122518+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'46","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.959960+0000","last_change":"2024-09-19T11:27:08.959960+0000","last_active":"2024-09-19T11:27:08.959960+0000","last_peered":"2024-09-19T11:27:08.959960+0000","last_clean":"2024-09-19T11:27:08.959960+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:27:08.959960+0000","last_undegraded":"2024-09-19T11:27:08.959960+0000","last_fullsized":"2024-09-19T11:27:08.959960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:08.959862+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:08.959862+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.959862+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:37.904449+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.836563+0000","last_change":"2024-09-19T11:27:14.836563+0000","last_active":"2024-09-19T11:27:14.836563+0000","last_peered":"2024-09-19T11:27:14.836563+0000","last_clean":"2024-09-19T11:27:14.836563+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:27:14.836563+0000","last_undegraded":"2024-09-19T11:27:14.836563+0000","last_fullsized":"2024-09-19T11:27:14.836563+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.836468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.836468+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.836468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:05.472792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.283516+0000","last_change":"2024-09-19T11:23:13.268771+0000","last_active":"2024-09-19T11:23:42.283516+0000","last_peered":"2024-09-19T11:23:42.283516+0000","last_clean":"2024-09-19T11:23:42.283516+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:23:42.283516+0000","last_undegraded":"2024-09-19T11:23:42.283516+0000","last_fullsized":"2024-09-19T11:23:42.283516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:19.996233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.724840+0000","last_change":"2024-09-19T11:26:46.724840+0000","last_active":"2024-09-19T11:26:46.724840+0000","last_peered":"2024-09-19T11:26:46.724840+0000","last_clean":"2024-09-19T11:26:46.724840+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:46.724840+0000","last_undegraded":"2024-09-19T11:26:46.724840+0000","last_fullsized":"2024-09-19T11:26:46.724840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.724740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:47.953525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.382940+0000","last_change":"2024-09-19T11:26:41.382940+0000","last_active":"2024-09-19T11:26:41.382940+0000","last_peered":"2024-09-19T11:26:41.382940+0000","last_clean":"2024-09-19T11:26:41.382940+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:26:41.382940+0000","last_undegraded":"2024-09-19T11:26:41.382940+0000","last_fullsized":"2024-09-19T11:26:41.382940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.382839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:55.560428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.798773+0000","last_change":"2024-09-19T11:26:56.798773+0000","last_active":"2024-09-19T11:26:56.798773+0000","last_peered":"2024-09-19T11:26:56.798773+0000","last_clean":"2024-09-19T11:26:56.798773+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:26:56.798773+0000","last_undegraded":"2024-09-19T11:26:56.798773+0000","last_fullsized":"2024-09-19T11:26:56.798773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.798674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:05.764652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'38","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.766754+0000","last_change":"2024-09-19T11:27:02.766754+0000","last_active":"2024-09-19T11:27:02.766754+0000","last_peered":"2024-09-19T11:27:02.766754+0000","last_clean":"2024-09-19T11:27:02.766754+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:27:02.766754+0000","last_undegraded":"2024-09-19T11:27:02.766754+0000","last_fullsized":"2024-09-19T11:27:02.766754+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:02.766665+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:02.766665+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.766665+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:23.822170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943735+0000","last_change":"2024-09-19T11:23:11.271462+0000","last_active":"2024-09-19T11:23:42.943735+0000","last_peered":"2024-09-19T11:23:42.943735+0000","last_clean":"2024-09-19T11:23:42.943735+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:23:42.943735+0000","last_undegraded":"2024-09-19T11:23:42.943735+0000","last_fullsized":"2024-09-19T11:23:42.943735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:15.194246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.728999+0000","last_change":"2024-09-19T11:26:45.728999+0000","last_active":"2024-09-19T11:26:45.728999+0000","last_peered":"2024-09-19T11:26:45.728999+0000","last_clean":"2024-09-19T11:26:45.728999+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:45.728999+0000","last_undegraded":"2024-09-19T11:26:45.728999+0000","last_fullsized":"2024-09-19T11:26:45.728999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.728900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:26.277302+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.898920+0000","last_change":"2024-09-19T11:26:26.898841+0000","last_active":"2024-09-19T11:26:26.898920+0000","last_peered":"2024-09-19T11:26:26.898920+0000","last_clean":"2024-09-19T11:26:26.898920+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:26:26.898920+0000","last_undegraded":"2024-09-19T11:26:26.898920+0000","last_fullsized":"2024-09-19T11:26:26.898920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.898758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:04.348903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.929630+0000","last_change":"2024-09-19T11:26:59.929630+0000","last_active":"2024-09-19T11:26:59.929630+0000","last_peered":"2024-09-19T11:26:59.929630+0000","last_clean":"2024-09-19T11:26:59.929630+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:26:59.929630+0000","last_undegraded":"2024-09-19T11:26:59.929630+0000","last_fullsized":"2024-09-19T11:26:59.929630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.929544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:14.552058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'29","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.120252+0000","last_change":"2024-09-19T11:23:09.258685+0000","last_active":"2024-09-19T11:26:19.120252+0000","last_peered":"2024-09-19T11:26:19.120252+0000","last_clean":"2024-09-19T11:26:19.120252+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:26:19.120252+0000","last_undegraded":"2024-09-19T11:26:19.120252+0000","last_fullsized":"2024-09-19T11:26:19.120252+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:33.508526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.944317+0000","last_change":"2024-09-19T11:23:11.267904+0000","last_active":"2024-09-19T11:23:42.944317+0000","last_peered":"2024-09-19T11:23:42.944317+0000","last_clean":"2024-09-19T11:23:42.944317+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:23:42.944317+0000","last_undegraded":"2024-09-19T11:23:42.944317+0000","last_fullsized":"2024-09-19T11:23:42.944317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:10.236761+0000","last_clean_scrub_stamp":"2024-09-19T11:23:10.236761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:16.874318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.773207+0000","last_change":"2024-09-19T11:26:44.773207+0000","last_active":"2024-09-19T11:26:44.773207+0000","last_peered":"2024-09-19T11:26:44.773207+0000","last_clean":"2024-09-19T11:26:44.773207+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:26:44.773207+0000","last_undegraded":"2024-09-19T11:26:44.773207+0000","last_fullsized":"2024-09-19T11:26:44.773207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.773116+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:30.345346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.916369+0000","last_change":"2024-09-19T11:26:29.916292+0000","last_active":"2024-09-19T11:26:29.916369+0000","last_peered":"2024-09-19T11:26:29.916369+0000","last_clean":"2024-09-19T11:26:29.916369+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:26:29.916369+0000","last_undegraded":"2024-09-19T11:26:29.916369+0000","last_fullsized":"2024-09-19T11:26:29.916369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.916202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:50.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'3","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.924054+0000","last_change":"2024-09-19T11:27:01.924054+0000","last_active":"2024-09-19T11:27:01.924054+0000","last_peered":"2024-09-19T11:27:01.924054+0000","last_clean":"2024-09-19T11:27:01.924054+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:27:01.924054+0000","last_undegraded":"2024-09-19T11:27:01.924054+0000","last_fullsized":"2024-09-19T11:27:01.924054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:01.923971+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:01.923971+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.923971+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:26:44.012364+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'37","reported_seq":86,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:19.129686+0000","last_change":"2024-09-19T11:23:09.255745+0000","last_active":"2024-09-19T11:26:19.129686+0000","last_peered":"2024-09-19T11:26:19.129686+0000","last_clean":"2024-09-19T11:26:19.129686+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:26:19.129686+0000","last_undegraded":"2024-09-19T11:26:19.129686+0000","last_fullsized":"2024-09-19T11:26:19.129686+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:08.229251+0000","last_clean_scrub_stamp":"2024-09-19T11:23:08.229251+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:35.538204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.837684+0000","last_change":"2024-09-19T11:27:15.837684+0000","last_active":"2024-09-19T11:27:15.837684+0000","last_peered":"2024-09-19T11:27:15.837684+0000","last_clean":"2024-09-19T11:27:15.837684+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:27:15.837684+0000","last_undegraded":"2024-09-19T11:27:15.837684+0000","last_fullsized":"2024-09-19T11:27:15.837684+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.837602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.837602+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.837602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:39.151539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.120385+0000","last_change":"2024-09-19T11:26:45.120385+0000","last_active":"2024-09-19T11:26:45.120385+0000","last_peered":"2024-09-19T11:26:45.120385+0000","last_clean":"2024-09-19T11:26:45.120385+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:26:45.120385+0000","last_undegraded":"2024-09-19T11:26:45.120385+0000","last_fullsized":"2024-09-19T11:26:45.120385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.120277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:55.577999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.949943+0000","last_change":"2024-09-19T11:26:31.949825+0000","last_active":"2024-09-19T11:26:31.949943+0000","last_peered":"2024-09-19T11:26:31.949943+0000","last_clean":"2024-09-19T11:26:31.949943+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:26:31.949943+0000","last_undegraded":"2024-09-19T11:26:31.949943+0000","last_fullsized":"2024-09-19T11:26:31.949943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.949702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:38.587259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'18","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.879819+0000","last_change":"2024-09-19T11:26:46.879819+0000","last_active":"2024-09-19T11:26:46.879819+0000","last_peered":"2024-09-19T11:26:46.879819+0000","last_clean":"2024-09-19T11:26:46.879819+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:46.879819+0000","last_undegraded":"2024-09-19T11:26:46.879819+0000","last_fullsized":"2024-09-19T11:26:46.879819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'18","last_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_deep_scrub":"33'18","last_deep_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.879729+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:14.358902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.905354+0000","last_change":"2024-09-19T11:26:27.905294+0000","last_active":"2024-09-19T11:26:27.905354+0000","last_peered":"2024-09-19T11:26:27.905354+0000","last_clean":"2024-09-19T11:26:27.905354+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:26:27.905354+0000","last_undegraded":"2024-09-19T11:26:27.905354+0000","last_fullsized":"2024-09-19T11:26:27.905354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.905202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:43:38.666093+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.503595+0000","last_change":"2024-09-19T11:27:04.503595+0000","last_active":"2024-09-19T11:27:04.503595+0000","last_peered":"2024-09-19T11:27:04.503595+0000","last_clean":"2024-09-19T11:27:04.503595+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:27:04.503595+0000","last_undegraded":"2024-09-19T11:27:04.503595+0000","last_fullsized":"2024-09-19T11:27:04.503595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:04.503510+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:04.503510+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.503510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:01:27.895549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.388722+0000","last_change":"2024-09-19T11:26:42.388646+0000","last_active":"2024-09-19T11:26:42.388722+0000","last_peered":"2024-09-19T11:26:42.388722+0000","last_clean":"2024-09-19T11:26:42.388722+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:26:42.388722+0000","last_undegraded":"2024-09-19T11:26:42.388722+0000","last_fullsized":"2024-09-19T11:26:42.388722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.388560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:26.548057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.896965+0000","last_change":"2024-09-19T11:26:45.896965+0000","last_active":"2024-09-19T11:26:45.896965+0000","last_peered":"2024-09-19T11:26:45.896965+0000","last_clean":"2024-09-19T11:26:45.896965+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:26:45.896965+0000","last_undegraded":"2024-09-19T11:26:45.896965+0000","last_fullsized":"2024-09-19T11:26:45.896965+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.896872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:37.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.953844+0000","last_change":"2024-09-19T11:26:30.953783+0000","last_active":"2024-09-19T11:26:30.953844+0000","last_peered":"2024-09-19T11:26:30.953844+0000","last_clean":"2024-09-19T11:26:30.953844+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:26:30.953844+0000","last_undegraded":"2024-09-19T11:26:30.953844+0000","last_fullsized":"2024-09-19T11:26:30.953844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.953693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:32.144901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.913054+0000","last_change":"2024-09-19T11:26:49.912991+0000","last_active":"2024-09-19T11:26:49.913054+0000","last_peered":"2024-09-19T11:26:49.913054+0000","last_clean":"2024-09-19T11:26:49.913054+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:49.913054+0000","last_undegraded":"2024-09-19T11:26:49.913054+0000","last_fullsized":"2024-09-19T11:26:49.913054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.912905+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:07.844062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.952079+0000","last_change":"2024-09-19T11:26:29.952079+0000","last_active":"2024-09-19T11:26:29.952079+0000","last_peered":"2024-09-19T11:26:29.952079+0000","last_clean":"2024-09-19T11:26:29.952079+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:29.952079+0000","last_undegraded":"2024-09-19T11:26:29.952079+0000","last_fullsized":"2024-09-19T11:26:29.952079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.951980+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:30.180814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.104057+0000","last_change":"2024-09-19T11:26:44.103981+0000","last_active":"2024-09-19T11:26:44.104057+0000","last_peered":"2024-09-19T11:26:44.104057+0000","last_clean":"2024-09-19T11:26:44.104057+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:26:44.104057+0000","last_undegraded":"2024-09-19T11:26:44.104057+0000","last_fullsized":"2024-09-19T11:26:44.104057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.103889+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:14.482038+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.867275+0000","last_change":"2024-09-19T11:26:30.867275+0000","last_active":"2024-09-19T11:26:30.867275+0000","last_peered":"2024-09-19T11:26:30.867275+0000","last_clean":"2024-09-19T11:26:30.867275+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:26:30.867275+0000","last_undegraded":"2024-09-19T11:26:30.867275+0000","last_fullsized":"2024-09-19T11:26:30.867275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.867184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:38.055814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.932867+0000","last_change":"2024-09-19T11:26:44.932807+0000","last_active":"2024-09-19T11:26:44.932867+0000","last_peered":"2024-09-19T11:26:44.932867+0000","last_clean":"2024-09-19T11:26:44.932867+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:26:44.932867+0000","last_undegraded":"2024-09-19T11:26:44.932867+0000","last_fullsized":"2024-09-19T11:26:44.932867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.932715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:07.921677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.978046+0000","last_change":"2024-09-19T11:26:32.977981+0000","last_active":"2024-09-19T11:26:32.978046+0000","last_peered":"2024-09-19T11:26:32.978046+0000","last_clean":"2024-09-19T11:26:32.978046+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:32.978046+0000","last_undegraded":"2024-09-19T11:26:32.978046+0000","last_fullsized":"2024-09-19T11:26:32.978046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.977891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:36.076149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.063998+0000","last_change":"2024-09-19T11:26:43.063925+0000","last_active":"2024-09-19T11:26:43.063998+0000","last_peered":"2024-09-19T11:26:43.063998+0000","last_clean":"2024-09-19T11:26:43.063998+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:43.063998+0000","last_undegraded":"2024-09-19T11:26:43.063998+0000","last_fullsized":"2024-09-19T11:26:43.063998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.063836+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:23.821074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.990845+0000","last_change":"2024-09-19T11:26:31.990845+0000","last_active":"2024-09-19T11:26:31.990845+0000","last_peered":"2024-09-19T11:26:31.990845+0000","last_clean":"2024-09-19T11:26:31.990845+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:26:31.990845+0000","last_undegraded":"2024-09-19T11:26:31.990845+0000","last_fullsized":"2024-09-19T11:26:31.990845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.990752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:33.522127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.044177+0000","last_change":"2024-09-19T11:26:50.044177+0000","last_active":"2024-09-19T11:26:50.044177+0000","last_peered":"2024-09-19T11:26:50.044177+0000","last_clean":"2024-09-19T11:26:50.044177+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:26:50.044177+0000","last_undegraded":"2024-09-19T11:26:50.044177+0000","last_fullsized":"2024-09-19T11:26:50.044177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.044090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:50.741427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.359747+0000","last_change":"2024-09-19T11:26:43.359747+0000","last_active":"2024-09-19T11:26:43.359747+0000","last_peered":"2024-09-19T11:26:43.359747+0000","last_clean":"2024-09-19T11:26:43.359747+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:26:43.359747+0000","last_undegraded":"2024-09-19T11:26:43.359747+0000","last_fullsized":"2024-09-19T11:26:43.359747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.359650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.024659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.528597+0000","last_change":"2024-09-19T11:27:03.528597+0000","last_active":"2024-09-19T11:27:03.528597+0000","last_peered":"2024-09-19T11:27:03.528597+0000","last_clean":"2024-09-19T11:27:03.528597+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:27:03.528597+0000","last_undegraded":"2024-09-19T11:27:03.528597+0000","last_fullsized":"2024-09-19T11:27:03.528597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:27:03.528502+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:27:03.528502+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.528502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:34.145386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.017041+0000","last_change":"2024-09-19T11:26:33.017041+0000","last_active":"2024-09-19T11:26:33.017041+0000","last_peered":"2024-09-19T11:26:33.017041+0000","last_clean":"2024-09-19T11:26:33.017041+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:26:33.017041+0000","last_undegraded":"2024-09-19T11:26:33.017041+0000","last_fullsized":"2024-09-19T11:26:33.017041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.016950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:06.549550+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.883512+0000","last_change":"2024-09-19T11:26:47.883512+0000","last_active":"2024-09-19T11:26:47.883512+0000","last_peered":"2024-09-19T11:26:47.883512+0000","last_clean":"2024-09-19T11:26:47.883512+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:26:47.883512+0000","last_undegraded":"2024-09-19T11:26:47.883512+0000","last_fullsized":"2024-09-19T11:26:47.883512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.883414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:39.095013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.853602+0000","last_change":"2024-09-19T11:26:31.853602+0000","last_active":"2024-09-19T11:26:31.853602+0000","last_peered":"2024-09-19T11:26:31.853602+0000","last_clean":"2024-09-19T11:26:31.853602+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:26:31.853602+0000","last_undegraded":"2024-09-19T11:26:31.853602+0000","last_fullsized":"2024-09-19T11:26:31.853602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.853479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:16:07.826853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.073137+0000","last_change":"2024-09-19T11:26:49.073137+0000","last_active":"2024-09-19T11:26:49.073137+0000","last_peered":"2024-09-19T11:26:49.073137+0000","last_clean":"2024-09-19T11:26:49.073137+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:26:49.073137+0000","last_undegraded":"2024-09-19T11:26:49.073137+0000","last_fullsized":"2024-09-19T11:26:49.073137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.073043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:26.833881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.886892+0000","last_change":"2024-09-19T11:26:32.886892+0000","last_active":"2024-09-19T11:26:32.886892+0000","last_peered":"2024-09-19T11:26:32.886892+0000","last_clean":"2024-09-19T11:26:32.886892+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:26:32.886892+0000","last_undegraded":"2024-09-19T11:26:32.886892+0000","last_fullsized":"2024-09-19T11:26:32.886892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.886787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:19.114834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.038260+0000","last_change":"2024-09-19T11:26:48.038260+0000","last_active":"2024-09-19T11:26:48.038260+0000","last_peered":"2024-09-19T11:26:48.038260+0000","last_clean":"2024-09-19T11:26:48.038260+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:26:48.038260+0000","last_undegraded":"2024-09-19T11:26:48.038260+0000","last_fullsized":"2024-09-19T11:26:48.038260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.038149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:16.505179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.342671+0000","last_change":"2024-09-19T11:26:44.342671+0000","last_active":"2024-09-19T11:26:44.342671+0000","last_peered":"2024-09-19T11:26:44.342671+0000","last_clean":"2024-09-19T11:26:44.342671+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:26:44.342671+0000","last_undegraded":"2024-09-19T11:26:44.342671+0000","last_fullsized":"2024-09-19T11:26:44.342671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.342574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:38.309444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.082623+0000","last_change":"2024-09-19T11:26:47.082623+0000","last_active":"2024-09-19T11:26:47.082623+0000","last_peered":"2024-09-19T11:26:47.082623+0000","last_clean":"2024-09-19T11:26:47.082623+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:47.082623+0000","last_undegraded":"2024-09-19T11:26:47.082623+0000","last_fullsized":"2024-09-19T11:26:47.082623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.082498+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:22.862275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.395278+0000","last_change":"2024-09-19T11:26:46.395278+0000","last_active":"2024-09-19T11:26:46.395278+0000","last_peered":"2024-09-19T11:26:46.395278+0000","last_clean":"2024-09-19T11:26:46.395278+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:26:46.395278+0000","last_undegraded":"2024-09-19T11:26:46.395278+0000","last_fullsized":"2024-09-19T11:26:46.395278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.395181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:36.015668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":68,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.516337+0000","last_change":"2024-09-19T11:27:02.516337+0000","last_active":"2024-09-19T11:27:02.516337+0000","last_peered":"2024-09-19T11:27:02.516337+0000","last_clean":"2024-09-19T11:27:02.516337+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:27:02.516337+0000","last_undegraded":"2024-09-19T11:27:02.516337+0000","last_fullsized":"2024-09-19T11:27:02.516337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:02.516249+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:02.516249+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.516249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:11.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.979119+0000","last_change":"2024-09-19T11:26:33.979119+0000","last_active":"2024-09-19T11:26:33.979119+0000","last_peered":"2024-09-19T11:26:33.979119+0000","last_clean":"2024-09-19T11:26:33.979119+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:26:33.979119+0000","last_undegraded":"2024-09-19T11:26:33.979119+0000","last_fullsized":"2024-09-19T11:26:33.979119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.979000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:24.446915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":86,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.075580+0000","last_change":"2024-09-19T11:26:46.075580+0000","last_active":"2024-09-19T11:26:46.075580+0000","last_peered":"2024-09-19T11:26:46.075580+0000","last_clean":"2024-09-19T11:26:46.075580+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:46.075580+0000","last_undegraded":"2024-09-19T11:26:46.075580+0000","last_fullsized":"2024-09-19T11:26:46.075580+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'26","last_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_deep_scrub":"33'26","last_deep_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.075475+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:51.242922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.388148+0000","last_change":"2024-09-19T11:26:45.388148+0000","last_active":"2024-09-19T11:26:45.388148+0000","last_peered":"2024-09-19T11:26:45.388148+0000","last_clean":"2024-09-19T11:26:45.388148+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:26:45.388148+0000","last_undegraded":"2024-09-19T11:26:45.388148+0000","last_fullsized":"2024-09-19T11:26:45.388148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.388045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:47.891066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.428183+0000","last_change":"2024-09-19T11:26:47.428183+0000","last_active":"2024-09-19T11:26:47.428183+0000","last_peered":"2024-09-19T11:26:47.428183+0000","last_clean":"2024-09-19T11:26:47.428183+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:26:47.428183+0000","last_undegraded":"2024-09-19T11:26:47.428183+0000","last_fullsized":"2024-09-19T11:26:47.428183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.428079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:59:02.198551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.933041+0000","last_change":"2024-09-19T11:26:48.933041+0000","last_active":"2024-09-19T11:26:48.933041+0000","last_peered":"2024-09-19T11:26:48.933041+0000","last_clean":"2024-09-19T11:26:48.933041+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:48.933041+0000","last_undegraded":"2024-09-19T11:26:48.933041+0000","last_fullsized":"2024-09-19T11:26:48.933041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.932953+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:16.178277+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.990707+0000","last_change":"2024-09-19T11:26:33.990707+0000","last_active":"2024-09-19T11:26:33.990707+0000","last_peered":"2024-09-19T11:26:33.990707+0000","last_clean":"2024-09-19T11:26:33.990707+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:26:33.990707+0000","last_undegraded":"2024-09-19T11:26:33.990707+0000","last_fullsized":"2024-09-19T11:26:33.990707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.990617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:35.040214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.051717+0000","last_change":"2024-09-19T11:26:54.051717+0000","last_active":"2024-09-19T11:26:54.051717+0000","last_peered":"2024-09-19T11:26:54.051717+0000","last_clean":"2024-09-19T11:26:54.051717+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:26:54.051717+0000","last_undegraded":"2024-09-19T11:26:54.051717+0000","last_fullsized":"2024-09-19T11:26:54.051717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.051625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:27.912591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2287,"ondisk_log_size":2287,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1730,"num_omap_keys":8,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1515,"ondisk_log_size":1515,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":479,"ondisk_log_size":479,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673018,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":479232,"data_stored":150572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673018,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":737565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705723,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":739071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705722,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":148560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:27:24.967 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.967 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.967 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.967 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.967 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.967 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.968 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.968 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.968 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.968 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.968 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.969 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.969 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.969 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.969 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.970 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.970 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.970 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.970 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.970 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.971 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.971 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.971 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.971 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.971 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.971 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.972 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.972 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.972 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.972 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.972 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-09-19T11:23:10.236761+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=10, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.972 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.972 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-09-19T11:23:08.229251+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.973 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:24.973 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T11:27:44.974 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:27:45.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.085+0000 7fea7b717640 1 Processor -- start 2024-09-19T11:27:45.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea7b717640 1 -- start start 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea7b717640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea7406f020 0x7fea74134890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea7b717640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea740708a0 con 0x7fea7406e530 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea7b717640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fea740704d0 con 0x7fea7406f020 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea78cb2640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fea7406f020 0x7fea74134890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea794b3640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fea7406e530 0x7fea7406e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:32828/0 (socket says 172.21.15.22:32828) 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea794b3640 1 -- 172.21.15.22:0/863779577 learned_addr learned my addr 172.21.15.22:0/863779577 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.086+0000 7fea78cb2640 1 --2- 172.21.15.22:0/863779577 >> v2:172.21.15.22:3300/0 conn(0x7fea7406f020 0x7fea74134890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58026/0 (socket says 172.21.15.22:58026) 2024-09-19T11:27:45.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.087+0000 7fea78cb2640 1 -- 172.21.15.22:0/863779577 >> v1:172.21.15.22:6789/0 conn(0x7fea7406e530 legacy=0x7fea7406e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.087+0000 7fea78cb2640 1 -- 172.21.15.22:0/863779577 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7406f430 con 0x7fea7406f020 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.087+0000 7fea6bfff640 1 -- 172.21.15.22:0/863779577 <== mon.0 v1:172.21.15.22:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089189892 0 0) 0x7fea740708a0 con 0x7fea7406e530 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.087+0000 7fea78cb2640 1 --2- 172.21.15.22:0/863779577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406f020 0x7fea74134890 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fea64009720 tx=0x7fea6402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba318c832d3cb4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.087+0000 7fea6bfff640 1 -- 172.21.15.22:0/863779577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea64037070 con 0x7fea7406f020 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.087+0000 7fea6bfff640 1 -- 172.21.15.22:0/863779577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea6402ad90 con 0x7fea7406f020 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.087+0000 7fea6bfff640 1 -- 172.21.15.22:0/863779577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea64032590 con 0x7fea7406f020 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.088+0000 7fea7b717640 1 -- 172.21.15.22:0/863779577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406f020 msgr2=0x7fea74134890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.088+0000 7fea7b717640 1 --2- 172.21.15.22:0/863779577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406f020 0x7fea74134890 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fea64009720 tx=0x7fea6402a110 comp rx=0 tx=0).stop 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.088+0000 7fea7b717640 1 -- 172.21.15.22:0/863779577 shutdown_connections 2024-09-19T11:27:45.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.088+0000 7fea7b717640 1 --2- 172.21.15.22:0/863779577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406f020 0x7fea74134890 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.088+0000 7fea7b717640 1 -- 172.21.15.22:0/863779577 >> 172.21.15.22:0/863779577 conn(0x7fea740a22c0 msgr2=0x7fea740a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.088+0000 7fea7b717640 1 -- 172.21.15.22:0/863779577 shutdown_connections 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.088+0000 7fea7b717640 1 -- 172.21.15.22:0/863779577 wait complete. 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.089+0000 7fea7b717640 1 Processor -- start 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.089+0000 7fea7b717640 1 -- start start 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.089+0000 7fea7b717640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406e530 0x7fea74130610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.089+0000 7fea7b717640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fea74135160 con 0x7fea7406e530 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.089+0000 7fea794b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406e530 0x7fea74130610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea794b3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406e530 0x7fea74130610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58028/0 (socket says 172.21.15.22:58028) 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea794b3640 1 -- 172.21.15.22:0/2829864942 learned_addr learned my addr 172.21.15.22:0/2829864942 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:27:45.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea794b3640 1 -- 172.21.15.22:0/2829864942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7412f250 con 0x7fea7406e530 2024-09-19T11:27:45.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea794b3640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406e530 0x7fea74130610 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c7c0 tx=0x7fea7000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:45.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea69ffb640 1 -- 172.21.15.22:0/2829864942 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea70018070 con 0x7fea7406e530 2024-09-19T11:27:45.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea69ffb640 1 -- 172.21.15.22:0/2829864942 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea70004d70 con 0x7fea7406e530 2024-09-19T11:27:45.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7412f970 con 0x7fea7406e530 2024-09-19T11:27:45.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea69ffb640 1 -- 172.21.15.22:0/2829864942 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea7000a710 con 0x7fea7406e530 2024-09-19T11:27:45.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.090+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea74131eb0 con 0x7fea7406e530 2024-09-19T11:27:45.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.091+0000 7fea69ffb640 1 -- 172.21.15.22:0/2829864942 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fea7001b070 con 0x7fea7406e530 2024-09-19T11:27:45.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.091+0000 7fea69ffb640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fea5803e740 0x7fea58040c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:27:45.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.091+0000 7fea69ffb640 1 -- 172.21.15.22:0/2829864942 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fea70052e60 con 0x7fea7406e530 2024-09-19T11:27:45.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.092+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7406e910 con 0x7fea7406e530 2024-09-19T11:27:45.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.092+0000 7fea78cb2640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fea5803e740 0x7fea58040c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:27:45.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.092+0000 7fea78cb2640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fea5803e740 0x7fea58040c00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fea64034000 tx=0x7fea6402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:27:45.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.096+0000 7fea69ffb640 1 -- 172.21.15.22:0/2829864942 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fea70011db0 con 0x7fea7406e530 2024-09-19T11:27:45.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.281+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fea74129f90 con 0x7fea5803e740 2024-09-19T11:27:45.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.292+0000 7fea69ffb640 1 -- 172.21.15.22:0/2829864942 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+666163 (secure 0 0 0) 0x7fea74129f90 con 0x7fea5803e740 2024-09-19T11:27:45.304 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:27:45.309 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fea5803e740 msgr2=0x7fea58040c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fea5803e740 0x7fea58040c00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fea64034000 tx=0x7fea6402e000 comp rx=0 tx=0).stop 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406e530 msgr2=0x7fea74130610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406e530 0x7fea74130610 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c7c0 tx=0x7fea7000cc80 comp rx=0 tx=0).stop 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 shutdown_connections 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fea5803e740 0x7fea58040c00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 --2- 172.21.15.22:0/2829864942 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea7406e530 0x7fea74130610 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 >> 172.21.15.22:0/2829864942 conn(0x7fea740a22c0 msgr2=0x7fea7409df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 shutdown_connections 2024-09-19T11:27:45.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:27:45.297+0000 7fea7b717640 1 -- 172.21.15.22:0/2829864942 wait complete. 2024-09-19T11:27:45.313 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-19T11:27:44.875714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605410,"num_objects":222,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":0,"num_read":5189,"num_read_kb":4731,"num_write":2121,"num_write_kb":60587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4781,"ondisk_log_size":4781,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":119296,"kb_used_data":2880,"kb_used_omap":6,"kb_used_meta":116345,"kb_avail":377368064,"statfs":{"total":386547056640,"available":386424897536,"internally_reserved":0,"allocated":2949120,"data_stored":1775768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":119138070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004608"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.932419+0000","last_change":"2024-09-19T11:26:37.932354+0000","last_active":"2024-09-19T11:26:37.932419+0000","last_peered":"2024-09-19T11:26:37.932419+0000","last_clean":"2024-09-19T11:26:37.932419+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:26:37.932419+0000","last_undegraded":"2024-09-19T11:26:37.932419+0000","last_fullsized":"2024-09-19T11:26:37.932419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.932266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:41.727377+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.069528+0000","last_change":"2024-09-19T11:26:55.069528+0000","last_active":"2024-09-19T11:26:55.069528+0000","last_peered":"2024-09-19T11:26:55.069528+0000","last_clean":"2024-09-19T11:26:55.069528+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:55.069528+0000","last_undegraded":"2024-09-19T11:26:55.069528+0000","last_fullsized":"2024-09-19T11:26:55.069528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.069440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:55.494351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.990965+0000","last_change":"2024-09-19T11:26:36.990885+0000","last_active":"2024-09-19T11:26:36.990965+0000","last_peered":"2024-09-19T11:26:36.990965+0000","last_clean":"2024-09-19T11:26:36.990965+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:26:36.990965+0000","last_undegraded":"2024-09-19T11:26:36.990965+0000","last_fullsized":"2024-09-19T11:26:36.990965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.990795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:53.995221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.480743+0000","last_change":"2024-09-19T11:27:05.480743+0000","last_active":"2024-09-19T11:27:05.480743+0000","last_peered":"2024-09-19T11:27:05.480743+0000","last_clean":"2024-09-19T11:27:05.480743+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:27:05.480743+0000","last_undegraded":"2024-09-19T11:27:05.480743+0000","last_fullsized":"2024-09-19T11:27:05.480743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:05.480656+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:05.480656+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.480656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:16.117082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.390858+0000","last_change":"2024-09-19T11:26:51.390768+0000","last_active":"2024-09-19T11:26:51.390858+0000","last_peered":"2024-09-19T11:26:51.390858+0000","last_clean":"2024-09-19T11:26:51.390858+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:26:51.390858+0000","last_undegraded":"2024-09-19T11:26:51.390858+0000","last_fullsized":"2024-09-19T11:26:51.390858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.390678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:06.584463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'12","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.945164+0000","last_change":"2024-09-19T11:26:50.945164+0000","last_active":"2024-09-19T11:26:50.945164+0000","last_peered":"2024-09-19T11:26:50.945164+0000","last_clean":"2024-09-19T11:26:50.945164+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:50.945164+0000","last_undegraded":"2024-09-19T11:26:50.945164+0000","last_fullsized":"2024-09-19T11:26:50.945164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.945069+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:26:56.726703+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.958237+0000","last_change":"2024-09-19T11:26:36.958139+0000","last_active":"2024-09-19T11:26:36.958237+0000","last_peered":"2024-09-19T11:26:36.958237+0000","last_clean":"2024-09-19T11:26:36.958237+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:26:36.958237+0000","last_undegraded":"2024-09-19T11:26:36.958237+0000","last_fullsized":"2024-09-19T11:26:36.958237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.958025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:34.544025+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.497840+0000","last_change":"2024-09-19T11:27:07.497840+0000","last_active":"2024-09-19T11:27:07.497840+0000","last_peered":"2024-09-19T11:27:07.497840+0000","last_clean":"2024-09-19T11:27:07.497840+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:27:07.497840+0000","last_undegraded":"2024-09-19T11:27:07.497840+0000","last_fullsized":"2024-09-19T11:27:07.497840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.497726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.497726+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.497726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:24.485123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.373571+0000","last_change":"2024-09-19T11:26:50.373571+0000","last_active":"2024-09-19T11:26:50.373571+0000","last_peered":"2024-09-19T11:26:50.373571+0000","last_clean":"2024-09-19T11:26:50.373571+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:26:50.373571+0000","last_undegraded":"2024-09-19T11:26:50.373571+0000","last_fullsized":"2024-09-19T11:26:50.373571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.373464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:22:03.662429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'12","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.036262+0000","last_change":"2024-09-19T11:26:56.036130+0000","last_active":"2024-09-19T11:26:56.036262+0000","last_peered":"2024-09-19T11:26:56.036262+0000","last_clean":"2024-09-19T11:26:56.036262+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:56.036262+0000","last_undegraded":"2024-09-19T11:26:56.036262+0000","last_fullsized":"2024-09-19T11:26:56.036262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.036029+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.756679+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.972860+0000","last_change":"2024-09-19T11:26:35.972860+0000","last_active":"2024-09-19T11:26:35.972860+0000","last_peered":"2024-09-19T11:26:35.972860+0000","last_clean":"2024-09-19T11:26:35.972860+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:26:35.972860+0000","last_undegraded":"2024-09-19T11:26:35.972860+0000","last_fullsized":"2024-09-19T11:26:35.972860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.972760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:17.815936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.524392+0000","last_change":"2024-09-19T11:27:08.524312+0000","last_active":"2024-09-19T11:27:08.524392+0000","last_peered":"2024-09-19T11:27:08.524392+0000","last_clean":"2024-09-19T11:27:08.524392+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:27:08.524392+0000","last_undegraded":"2024-09-19T11:27:08.524392+0000","last_fullsized":"2024-09-19T11:27:08.524392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:27:08.524226+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:27:08.524226+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.524226+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:57.580270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.431257+0000","last_change":"2024-09-19T11:26:48.431257+0000","last_active":"2024-09-19T11:26:48.431257+0000","last_peered":"2024-09-19T11:26:48.431257+0000","last_clean":"2024-09-19T11:26:48.431257+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:48.431257+0000","last_undegraded":"2024-09-19T11:26:48.431257+0000","last_fullsized":"2024-09-19T11:26:48.431257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.431171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:57.441652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.745217+0000","last_change":"2024-09-19T11:26:49.745135+0000","last_active":"2024-09-19T11:26:49.745217+0000","last_peered":"2024-09-19T11:26:49.745217+0000","last_clean":"2024-09-19T11:26:49.745217+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:49.745217+0000","last_undegraded":"2024-09-19T11:26:49.745217+0000","last_fullsized":"2024-09-19T11:26:49.745217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.745054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:51.276429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.989858+0000","last_change":"2024-09-19T11:26:35.989858+0000","last_active":"2024-09-19T11:26:35.989858+0000","last_peered":"2024-09-19T11:26:35.989858+0000","last_clean":"2024-09-19T11:26:35.989858+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:26:35.989858+0000","last_undegraded":"2024-09-19T11:26:35.989858+0000","last_fullsized":"2024-09-19T11:26:35.989858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.989770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.409598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.783741+0000","last_change":"2024-09-19T11:26:50.783644+0000","last_active":"2024-09-19T11:26:50.783741+0000","last_peered":"2024-09-19T11:26:50.783741+0000","last_clean":"2024-09-19T11:26:50.783741+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:26:50.783741+0000","last_undegraded":"2024-09-19T11:26:50.783741+0000","last_fullsized":"2024-09-19T11:26:50.783741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.783546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:56.110032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.464525+0000","last_change":"2024-09-19T11:27:06.464525+0000","last_active":"2024-09-19T11:27:06.464525+0000","last_peered":"2024-09-19T11:27:06.464525+0000","last_clean":"2024-09-19T11:27:06.464525+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:27:06.464525+0000","last_undegraded":"2024-09-19T11:27:06.464525+0000","last_fullsized":"2024-09-19T11:27:06.464525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:06.464436+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:06.464436+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.464436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:12.768140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.946461+0000","last_change":"2024-09-19T11:26:34.946461+0000","last_active":"2024-09-19T11:26:34.946461+0000","last_peered":"2024-09-19T11:26:34.946461+0000","last_clean":"2024-09-19T11:26:34.946461+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:26:34.946461+0000","last_undegraded":"2024-09-19T11:26:34.946461+0000","last_fullsized":"2024-09-19T11:26:34.946461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.946370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:48.027671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.017232+0000","last_change":"2024-09-19T11:26:51.017232+0000","last_active":"2024-09-19T11:26:51.017232+0000","last_peered":"2024-09-19T11:26:51.017232+0000","last_clean":"2024-09-19T11:26:51.017232+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:26:51.017232+0000","last_undegraded":"2024-09-19T11:26:51.017232+0000","last_fullsized":"2024-09-19T11:26:51.017232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.017146+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.668961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.952151+0000","last_change":"2024-09-19T11:26:34.952151+0000","last_active":"2024-09-19T11:26:34.952151+0000","last_peered":"2024-09-19T11:26:34.952151+0000","last_clean":"2024-09-19T11:26:34.952151+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:26:34.952151+0000","last_undegraded":"2024-09-19T11:26:34.952151+0000","last_fullsized":"2024-09-19T11:26:34.952151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.952058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:03.872887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.018264+0000","last_change":"2024-09-19T11:26:52.018264+0000","last_active":"2024-09-19T11:26:52.018264+0000","last_peered":"2024-09-19T11:26:52.018264+0000","last_clean":"2024-09-19T11:26:52.018264+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:26:52.018264+0000","last_undegraded":"2024-09-19T11:26:52.018264+0000","last_fullsized":"2024-09-19T11:26:52.018264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.018180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:31.898878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.916754+0000","last_change":"2024-09-19T11:26:33.916754+0000","last_active":"2024-09-19T11:26:33.916754+0000","last_peered":"2024-09-19T11:26:33.916754+0000","last_clean":"2024-09-19T11:26:33.916754+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:33.916754+0000","last_undegraded":"2024-09-19T11:26:33.916754+0000","last_fullsized":"2024-09-19T11:26:33.916754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.916663+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:35.684322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":68,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.898475+0000","last_change":"2024-09-19T11:26:51.898475+0000","last_active":"2024-09-19T11:26:51.898475+0000","last_peered":"2024-09-19T11:26:51.898475+0000","last_clean":"2024-09-19T11:26:51.898475+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:51.898475+0000","last_undegraded":"2024-09-19T11:26:51.898475+0000","last_fullsized":"2024-09-19T11:26:51.898475+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.898386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:37.296273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.987445+0000","last_change":"2024-09-19T11:26:37.987445+0000","last_active":"2024-09-19T11:26:37.987445+0000","last_peered":"2024-09-19T11:26:37.987445+0000","last_clean":"2024-09-19T11:26:37.987445+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:26:37.987445+0000","last_undegraded":"2024-09-19T11:26:37.987445+0000","last_fullsized":"2024-09-19T11:26:37.987445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.987357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:27.173996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.779288+0000","last_change":"2024-09-19T11:26:48.779288+0000","last_active":"2024-09-19T11:26:48.779288+0000","last_peered":"2024-09-19T11:26:48.779288+0000","last_clean":"2024-09-19T11:26:48.779288+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:48.779288+0000","last_undegraded":"2024-09-19T11:26:48.779288+0000","last_fullsized":"2024-09-19T11:26:48.779288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.779185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:49.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.001114+0000","last_change":"2024-09-19T11:26:35.001114+0000","last_active":"2024-09-19T11:26:35.001114+0000","last_peered":"2024-09-19T11:26:35.001114+0000","last_clean":"2024-09-19T11:26:35.001114+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:26:35.001114+0000","last_undegraded":"2024-09-19T11:26:35.001114+0000","last_fullsized":"2024-09-19T11:26:35.001114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.001027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:09.576343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.033350+0000","last_change":"2024-09-19T11:26:53.033350+0000","last_active":"2024-09-19T11:26:53.033350+0000","last_peered":"2024-09-19T11:26:53.033350+0000","last_clean":"2024-09-19T11:26:53.033350+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:53.033350+0000","last_undegraded":"2024-09-19T11:26:53.033350+0000","last_fullsized":"2024-09-19T11:26:53.033350+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.033252+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:11.509923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.392963+0000","last_change":"2024-09-19T11:26:49.392963+0000","last_active":"2024-09-19T11:26:49.392963+0000","last_peered":"2024-09-19T11:26:49.392963+0000","last_clean":"2024-09-19T11:26:49.392963+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:26:49.392963+0000","last_undegraded":"2024-09-19T11:26:49.392963+0000","last_fullsized":"2024-09-19T11:26:49.392963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.392873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:20.014824+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'38","reported_seq":111,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:22.907954+0000","last_change":"2024-09-19T11:27:22.907954+0000","last_active":"2024-09-19T11:27:22.907954+0000","last_peered":"2024-09-19T11:27:22.907954+0000","last_clean":"2024-09-19T11:27:22.907954+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:27:22.907954+0000","last_undegraded":"2024-09-19T11:27:22.907954+0000","last_fullsized":"2024-09-19T11:27:22.907954+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:22.907870+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:22.907870+0000","last_clean_scrub_stamp":"2024-09-19T11:27:22.907870+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:56.381170+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.766852+0000","last_change":"2024-09-19T11:26:43.766852+0000","last_active":"2024-09-19T11:26:43.766852+0000","last_peered":"2024-09-19T11:26:43.766852+0000","last_clean":"2024-09-19T11:26:43.766852+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:43.766852+0000","last_undegraded":"2024-09-19T11:26:43.766852+0000","last_fullsized":"2024-09-19T11:26:43.766852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.766755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:14.579342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.365713+0000","last_change":"2024-09-19T11:26:37.365713+0000","last_active":"2024-09-19T11:26:37.365713+0000","last_peered":"2024-09-19T11:26:37.365713+0000","last_clean":"2024-09-19T11:26:37.365713+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:26:37.365713+0000","last_undegraded":"2024-09-19T11:26:37.365713+0000","last_fullsized":"2024-09-19T11:26:37.365713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.365625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:20.032640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.572512+0000","last_change":"2024-09-19T11:27:15.572512+0000","last_active":"2024-09-19T11:27:15.572512+0000","last_peered":"2024-09-19T11:27:15.572512+0000","last_clean":"2024-09-19T11:27:15.572512+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:27:15.572512+0000","last_undegraded":"2024-09-19T11:27:15.572512+0000","last_fullsized":"2024-09-19T11:27:15.572512+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.572426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.572426+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.572426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:44.280324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'38","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:25.602045+0000","last_change":"2024-09-19T11:27:25.602045+0000","last_active":"2024-09-19T11:27:25.602045+0000","last_peered":"2024-09-19T11:27:25.602045+0000","last_clean":"2024-09-19T11:27:25.602045+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:27:25.602045+0000","last_undegraded":"2024-09-19T11:27:25.602045+0000","last_fullsized":"2024-09-19T11:27:25.602045+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:25.601947+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:25.601947+0000","last_clean_scrub_stamp":"2024-09-19T11:27:25.601947+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:07:11.813100+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.903454+0000","last_change":"2024-09-19T11:27:14.903454+0000","last_active":"2024-09-19T11:27:14.903454+0000","last_peered":"2024-09-19T11:27:14.903454+0000","last_clean":"2024-09-19T11:27:14.903454+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:27:14.903454+0000","last_undegraded":"2024-09-19T11:27:14.903454+0000","last_fullsized":"2024-09-19T11:27:14.903454+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.903359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.903359+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.903359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:25.564676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:23.881144+0000","last_change":"2024-09-19T11:27:23.881144+0000","last_active":"2024-09-19T11:27:23.881144+0000","last_peered":"2024-09-19T11:27:23.881144+0000","last_clean":"2024-09-19T11:27:23.881144+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:27:23.881144+0000","last_undegraded":"2024-09-19T11:27:23.881144+0000","last_fullsized":"2024-09-19T11:27:23.881144+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'19","last_scrub_stamp":"2024-09-19T11:27:23.881053+0000","last_deep_scrub":"33'19","last_deep_scrub_stamp":"2024-09-19T11:27:23.881053+0000","last_clean_scrub_stamp":"2024-09-19T11:27:23.881053+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:01.072315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.402917+0000","last_change":"2024-09-19T11:26:57.402917+0000","last_active":"2024-09-19T11:26:57.402917+0000","last_peered":"2024-09-19T11:26:57.402917+0000","last_clean":"2024-09-19T11:26:57.402917+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:57.402917+0000","last_undegraded":"2024-09-19T11:26:57.402917+0000","last_fullsized":"2024-09-19T11:26:57.402917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.402830+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:15.259544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.907862+0000","last_change":"2024-09-19T11:26:30.907786+0000","last_active":"2024-09-19T11:26:30.907862+0000","last_peered":"2024-09-19T11:26:30.907862+0000","last_clean":"2024-09-19T11:26:30.907862+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:26:30.907862+0000","last_undegraded":"2024-09-19T11:26:30.907862+0000","last_fullsized":"2024-09-19T11:26:30.907862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.907688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:01.389670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.767856+0000","last_change":"2024-09-19T11:26:59.767856+0000","last_active":"2024-09-19T11:26:59.767856+0000","last_peered":"2024-09-19T11:26:59.767856+0000","last_clean":"2024-09-19T11:26:59.767856+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:26:59.767856+0000","last_undegraded":"2024-09-19T11:26:59.767856+0000","last_fullsized":"2024-09-19T11:26:59.767856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.767773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:00:22.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'10","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.925116+0000","last_change":"2024-09-19T11:27:09.925116+0000","last_active":"2024-09-19T11:27:09.925116+0000","last_peered":"2024-09-19T11:27:09.925116+0000","last_clean":"2024-09-19T11:27:09.925116+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:27:09.925116+0000","last_undegraded":"2024-09-19T11:27:09.925116+0000","last_fullsized":"2024-09-19T11:27:09.925116+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:27:09.925027+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:27:09.925027+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.925027+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:01.267066+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.901960+0000","last_change":"2024-09-19T11:27:13.901960+0000","last_active":"2024-09-19T11:27:13.901960+0000","last_peered":"2024-09-19T11:27:13.901960+0000","last_clean":"2024-09-19T11:27:13.901960+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:27:13.901960+0000","last_undegraded":"2024-09-19T11:27:13.901960+0000","last_fullsized":"2024-09-19T11:27:13.901960+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:13.901857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:13.901857+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.901857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:59.640755+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:21.839133+0000","last_change":"2024-09-19T11:27:21.839133+0000","last_active":"2024-09-19T11:27:21.839133+0000","last_peered":"2024-09-19T11:27:21.839133+0000","last_clean":"2024-09-19T11:27:21.839133+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:27:21.839133+0000","last_undegraded":"2024-09-19T11:27:21.839133+0000","last_fullsized":"2024-09-19T11:27:21.839133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:21.839044+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:21.839044+0000","last_clean_scrub_stamp":"2024-09-19T11:27:21.839044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:49.430324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.434595+0000","last_change":"2024-09-19T11:26:59.434595+0000","last_active":"2024-09-19T11:26:59.434595+0000","last_peered":"2024-09-19T11:26:59.434595+0000","last_clean":"2024-09-19T11:26:59.434595+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:26:59.434595+0000","last_undegraded":"2024-09-19T11:26:59.434595+0000","last_fullsized":"2024-09-19T11:26:59.434595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.434505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:29.091390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.378428+0000","last_change":"2024-09-19T11:26:35.378428+0000","last_active":"2024-09-19T11:26:35.378428+0000","last_peered":"2024-09-19T11:26:35.378428+0000","last_clean":"2024-09-19T11:26:35.378428+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:26:35.378428+0000","last_undegraded":"2024-09-19T11:26:35.378428+0000","last_fullsized":"2024-09-19T11:26:35.378428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.378338+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:26.014662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.815608+0000","last_change":"2024-09-19T11:26:54.815608+0000","last_active":"2024-09-19T11:26:54.815608+0000","last_peered":"2024-09-19T11:26:54.815608+0000","last_clean":"2024-09-19T11:26:54.815608+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:26:54.815608+0000","last_undegraded":"2024-09-19T11:26:54.815608+0000","last_fullsized":"2024-09-19T11:26:54.815608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.815526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:36.935748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"33'15","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.846706+0000","last_change":"2024-09-19T11:27:04.846706+0000","last_active":"2024-09-19T11:27:04.846706+0000","last_peered":"2024-09-19T11:27:04.846706+0000","last_clean":"2024-09-19T11:27:04.846706+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:27:04.846706+0000","last_undegraded":"2024-09-19T11:27:04.846706+0000","last_fullsized":"2024-09-19T11:27:04.846706+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'15","last_scrub_stamp":"2024-09-19T11:27:04.846607+0000","last_deep_scrub":"33'15","last_deep_scrub_stamp":"2024-09-19T11:27:04.846607+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.846607+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:56.058206+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'286","reported_seq":398,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:35.677247+0000","last_change":"2024-09-19T11:27:35.677247+0000","last_active":"2024-09-19T11:27:35.677247+0000","last_peered":"2024-09-19T11:27:35.677247+0000","last_clean":"2024-09-19T11:27:35.677247+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:27:35.677247+0000","last_undegraded":"2024-09-19T11:27:35.677247+0000","last_fullsized":"2024-09-19T11:27:35.677247+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:35.677118+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:35.677118+0000","last_clean_scrub_stamp":"2024-09-19T11:27:35.677118+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:04.854093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.970806+0000","last_change":"2024-09-19T11:27:17.970806+0000","last_active":"2024-09-19T11:27:17.970806+0000","last_peered":"2024-09-19T11:27:17.970806+0000","last_clean":"2024-09-19T11:27:17.970806+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:27:17.970806+0000","last_undegraded":"2024-09-19T11:27:17.970806+0000","last_fullsized":"2024-09-19T11:27:17.970806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:27:17.970666+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:27:17.970666+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.970666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:52.045636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.025004+0000","last_change":"2024-09-19T11:26:42.025004+0000","last_active":"2024-09-19T11:26:42.025004+0000","last_peered":"2024-09-19T11:26:42.025004+0000","last_clean":"2024-09-19T11:26:42.025004+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:26:42.025004+0000","last_undegraded":"2024-09-19T11:26:42.025004+0000","last_fullsized":"2024-09-19T11:26:42.025004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.024919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:31.267851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.870975+0000","last_change":"2024-09-19T11:26:24.870975+0000","last_active":"2024-09-19T11:26:24.870975+0000","last_peered":"2024-09-19T11:26:24.870975+0000","last_clean":"2024-09-19T11:26:24.870975+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:26:24.870975+0000","last_undegraded":"2024-09-19T11:26:24.870975+0000","last_fullsized":"2024-09-19T11:26:24.870975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.870884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:14.567909+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.528989+0000","last_change":"2024-09-19T11:27:17.528989+0000","last_active":"2024-09-19T11:27:17.528989+0000","last_peered":"2024-09-19T11:27:17.528989+0000","last_clean":"2024-09-19T11:27:17.528989+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:27:17.528989+0000","last_undegraded":"2024-09-19T11:27:17.528989+0000","last_fullsized":"2024-09-19T11:27:17.528989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:17.528899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:17.528899+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.528899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:52:06.431179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'38","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.841593+0000","last_change":"2024-09-19T11:27:05.841593+0000","last_active":"2024-09-19T11:27:05.841593+0000","last_peered":"2024-09-19T11:27:05.841593+0000","last_clean":"2024-09-19T11:27:05.841593+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:27:05.841593+0000","last_undegraded":"2024-09-19T11:27:05.841593+0000","last_fullsized":"2024-09-19T11:27:05.841593+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:05.841496+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:05.841496+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.841496+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:33.086820+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'286","reported_seq":414,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.828626+0000","last_change":"2024-09-19T11:27:13.828626+0000","last_active":"2024-09-19T11:27:13.828626+0000","last_peered":"2024-09-19T11:27:13.828626+0000","last_clean":"2024-09-19T11:27:13.828626+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:27:13.828626+0000","last_undegraded":"2024-09-19T11:27:13.828626+0000","last_fullsized":"2024-09-19T11:27:13.828626+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:13.828521+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:13.828521+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.828521+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:34.119736+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.381390+0000","last_change":"2024-09-19T11:26:55.381320+0000","last_active":"2024-09-19T11:26:55.381390+0000","last_peered":"2024-09-19T11:26:55.381390+0000","last_clean":"2024-09-19T11:26:55.381390+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:26:55.381390+0000","last_undegraded":"2024-09-19T11:26:55.381390+0000","last_fullsized":"2024-09-19T11:26:55.381390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.381228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:24:06.217118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.956228+0000","last_change":"2024-09-19T11:26:27.956170+0000","last_active":"2024-09-19T11:26:27.956228+0000","last_peered":"2024-09-19T11:26:27.956228+0000","last_clean":"2024-09-19T11:26:27.956228+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:26:27.956228+0000","last_undegraded":"2024-09-19T11:26:27.956228+0000","last_fullsized":"2024-09-19T11:26:27.956228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.956081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:59.956862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.969564+0000","last_change":"2024-09-19T11:26:57.969498+0000","last_active":"2024-09-19T11:26:57.969564+0000","last_peered":"2024-09-19T11:26:57.969564+0000","last_clean":"2024-09-19T11:26:57.969564+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:26:57.969564+0000","last_undegraded":"2024-09-19T11:26:57.969564+0000","last_fullsized":"2024-09-19T11:26:57.969564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.969412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.712881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'52","reported_seq":113,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.849298+0000","last_change":"2024-09-19T11:27:01.849236+0000","last_active":"2024-09-19T11:27:01.849298+0000","last_peered":"2024-09-19T11:27:01.849298+0000","last_clean":"2024-09-19T11:27:01.849298+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:27:01.849298+0000","last_undegraded":"2024-09-19T11:27:01.849298+0000","last_fullsized":"2024-09-19T11:27:01.849298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'52","last_scrub_stamp":"2024-09-19T11:27:01.849143+0000","last_deep_scrub":"33'52","last_deep_scrub_stamp":"2024-09-19T11:27:01.849143+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.849143+0000","objects_scrubbed":9,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:24.838105+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.808259+0000","last_change":"2024-09-19T11:27:12.808200+0000","last_active":"2024-09-19T11:27:12.808259+0000","last_peered":"2024-09-19T11:27:12.808259+0000","last_clean":"2024-09-19T11:27:12.808259+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:27:12.808259+0000","last_undegraded":"2024-09-19T11:27:12.808259+0000","last_fullsized":"2024-09-19T11:27:12.808259+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:12.808111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:12.808111+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.808111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:53.887293+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.942123+0000","last_change":"2024-09-19T11:27:14.942063+0000","last_active":"2024-09-19T11:27:14.942123+0000","last_peered":"2024-09-19T11:27:14.942123+0000","last_clean":"2024-09-19T11:27:14.942123+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:27:14.942123+0000","last_undegraded":"2024-09-19T11:27:14.942123+0000","last_fullsized":"2024-09-19T11:27:14.942123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:14.941960+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:14.941960+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.941960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:20:48.525210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.014137+0000","last_change":"2024-09-19T11:26:39.014072+0000","last_active":"2024-09-19T11:26:39.014137+0000","last_peered":"2024-09-19T11:26:39.014137+0000","last_clean":"2024-09-19T11:26:39.014137+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:26:39.014137+0000","last_undegraded":"2024-09-19T11:26:39.014137+0000","last_fullsized":"2024-09-19T11:26:39.014137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.013977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:33.627023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.962177+0000","last_change":"2024-09-19T11:26:26.962098+0000","last_active":"2024-09-19T11:26:26.962177+0000","last_peered":"2024-09-19T11:26:26.962177+0000","last_clean":"2024-09-19T11:26:26.962177+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:26:26.962177+0000","last_undegraded":"2024-09-19T11:26:26.962177+0000","last_fullsized":"2024-09-19T11:26:26.962177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.962016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:19.643957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.852350+0000","last_change":"2024-09-19T11:26:53.852269+0000","last_active":"2024-09-19T11:26:53.852350+0000","last_peered":"2024-09-19T11:26:53.852350+0000","last_clean":"2024-09-19T11:26:53.852350+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:26:53.852350+0000","last_undegraded":"2024-09-19T11:26:53.852350+0000","last_fullsized":"2024-09-19T11:26:53.852350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.852186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:52.995416+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'37","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:28.621224+0000","last_change":"2024-09-19T11:27:28.621154+0000","last_active":"2024-09-19T11:27:28.621224+0000","last_peered":"2024-09-19T11:27:28.621224+0000","last_clean":"2024-09-19T11:27:28.621224+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:27:28.621224+0000","last_undegraded":"2024-09-19T11:27:28.621224+0000","last_fullsized":"2024-09-19T11:27:28.621224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'37","last_scrub_stamp":"2024-09-19T11:27:28.621059+0000","last_deep_scrub":"33'37","last_deep_scrub_stamp":"2024-09-19T11:27:28.621059+0000","last_clean_scrub_stamp":"2024-09-19T11:27:28.621059+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:17.237043+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.801081+0000","last_change":"2024-09-19T11:27:11.801023+0000","last_active":"2024-09-19T11:27:11.801081+0000","last_peered":"2024-09-19T11:27:11.801081+0000","last_clean":"2024-09-19T11:27:11.801081+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:27:11.801081+0000","last_undegraded":"2024-09-19T11:27:11.801081+0000","last_fullsized":"2024-09-19T11:27:11.801081+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.800931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.800931+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.800931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:52:07.495727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'6","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.973097+0000","last_change":"2024-09-19T11:27:17.973033+0000","last_active":"2024-09-19T11:27:17.973097+0000","last_peered":"2024-09-19T11:27:17.973097+0000","last_clean":"2024-09-19T11:27:17.973097+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:27:17.973097+0000","last_undegraded":"2024-09-19T11:27:17.973097+0000","last_fullsized":"2024-09-19T11:27:17.973097+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:27:17.972934+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:27:17.972934+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.972934+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:31.636218+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.889558+0000","last_change":"2024-09-19T11:26:38.889489+0000","last_active":"2024-09-19T11:26:38.889558+0000","last_peered":"2024-09-19T11:26:38.889558+0000","last_clean":"2024-09-19T11:26:38.889558+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:26:38.889558+0000","last_undegraded":"2024-09-19T11:26:38.889558+0000","last_fullsized":"2024-09-19T11:26:38.889558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.889400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:14.595110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.412920+0000","last_change":"2024-09-19T11:26:36.412815+0000","last_active":"2024-09-19T11:26:36.412920+0000","last_peered":"2024-09-19T11:26:36.412920+0000","last_clean":"2024-09-19T11:26:36.412920+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:36.412920+0000","last_undegraded":"2024-09-19T11:26:36.412920+0000","last_fullsized":"2024-09-19T11:26:36.412920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.412703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:59.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.839182+0000","last_change":"2024-09-19T11:26:55.839126+0000","last_active":"2024-09-19T11:26:55.839182+0000","last_peered":"2024-09-19T11:26:55.839182+0000","last_clean":"2024-09-19T11:26:55.839182+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:26:55.839182+0000","last_undegraded":"2024-09-19T11:26:55.839182+0000","last_fullsized":"2024-09-19T11:26:55.839182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:08.800957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'46","reported_seq":105,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:21.581544+0000","last_change":"2024-09-19T11:27:21.581442+0000","last_active":"2024-09-19T11:27:21.581544+0000","last_peered":"2024-09-19T11:27:21.581544+0000","last_clean":"2024-09-19T11:27:21.581544+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:27:21.581544+0000","last_undegraded":"2024-09-19T11:27:21.581544+0000","last_fullsized":"2024-09-19T11:27:21.581544+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:21.581324+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:21.581324+0000","last_clean_scrub_stamp":"2024-09-19T11:27:21.581324+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:02.933515+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'286","reported_seq":362,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.777546+0000","last_change":"2024-09-19T11:27:10.777433+0000","last_active":"2024-09-19T11:27:10.777546+0000","last_peered":"2024-09-19T11:27:10.777546+0000","last_clean":"2024-09-19T11:27:10.777546+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:27:10.777546+0000","last_undegraded":"2024-09-19T11:27:10.777546+0000","last_fullsized":"2024-09-19T11:27:10.777546+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:10.777329+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:10.777329+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.777329+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:10.736169+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.854886+0000","last_change":"2024-09-19T11:27:17.854808+0000","last_active":"2024-09-19T11:27:17.854886+0000","last_peered":"2024-09-19T11:27:17.854886+0000","last_clean":"2024-09-19T11:27:17.854886+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:27:17.854886+0000","last_undegraded":"2024-09-19T11:27:17.854886+0000","last_fullsized":"2024-09-19T11:27:17.854886+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:17.854703+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:17.854703+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.854703+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:23.432636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.810432+0000","last_change":"2024-09-19T11:26:40.810354+0000","last_active":"2024-09-19T11:26:40.810432+0000","last_peered":"2024-09-19T11:26:40.810432+0000","last_clean":"2024-09-19T11:26:40.810432+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:40.810432+0000","last_undegraded":"2024-09-19T11:26:40.810432+0000","last_fullsized":"2024-09-19T11:26:40.810432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.810272+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.328610+0000","last_change":"2024-09-19T11:26:34.328550+0000","last_active":"2024-09-19T11:26:34.328610+0000","last_peered":"2024-09-19T11:26:34.328610+0000","last_clean":"2024-09-19T11:26:34.328610+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:26:34.328610+0000","last_undegraded":"2024-09-19T11:26:34.328610+0000","last_fullsized":"2024-09-19T11:26:34.328610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.328463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:54.619678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.868804+0000","last_change":"2024-09-19T11:26:54.868748+0000","last_active":"2024-09-19T11:26:54.868804+0000","last_peered":"2024-09-19T11:26:54.868804+0000","last_clean":"2024-09-19T11:26:54.868804+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:26:54.868804+0000","last_undegraded":"2024-09-19T11:26:54.868804+0000","last_fullsized":"2024-09-19T11:26:54.868804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.868649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:17.358870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'66","reported_seq":140,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:22.556698+0000","last_change":"2024-09-19T11:27:22.556636+0000","last_active":"2024-09-19T11:27:22.556698+0000","last_peered":"2024-09-19T11:27:22.556698+0000","last_clean":"2024-09-19T11:27:22.556698+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:27:22.556698+0000","last_undegraded":"2024-09-19T11:27:22.556698+0000","last_fullsized":"2024-09-19T11:27:22.556698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'66","last_scrub_stamp":"2024-09-19T11:27:22.556541+0000","last_deep_scrub":"33'66","last_deep_scrub_stamp":"2024-09-19T11:27:22.556541+0000","last_clean_scrub_stamp":"2024-09-19T11:27:22.556541+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:40.992302+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.744390+0000","last_change":"2024-09-19T11:27:08.744324+0000","last_active":"2024-09-19T11:27:08.744390+0000","last_peered":"2024-09-19T11:27:08.744390+0000","last_clean":"2024-09-19T11:27:08.744390+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:27:08.744390+0000","last_undegraded":"2024-09-19T11:27:08.744390+0000","last_fullsized":"2024-09-19T11:27:08.744390+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:08.744222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:08.744222+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.744222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:52.725769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'21","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.840237+0000","last_change":"2024-09-19T11:27:18.840174+0000","last_active":"2024-09-19T11:27:18.840237+0000","last_peered":"2024-09-19T11:27:18.840237+0000","last_clean":"2024-09-19T11:27:18.840237+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:27:18.840237+0000","last_undegraded":"2024-09-19T11:27:18.840237+0000","last_fullsized":"2024-09-19T11:27:18.840237+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'21","last_scrub_stamp":"2024-09-19T11:27:18.840078+0000","last_deep_scrub":"33'21","last_deep_scrub_stamp":"2024-09-19T11:27:18.840078+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.840078+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:42.094194+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.048396+0000","last_change":"2024-09-19T11:26:40.048324+0000","last_active":"2024-09-19T11:26:40.048396+0000","last_peered":"2024-09-19T11:26:40.048396+0000","last_clean":"2024-09-19T11:26:40.048396+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:26:40.048396+0000","last_undegraded":"2024-09-19T11:26:40.048396+0000","last_fullsized":"2024-09-19T11:26:40.048396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.048219+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:04:18.644192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.937593+0000","last_change":"2024-09-19T11:26:28.937533+0000","last_active":"2024-09-19T11:26:28.937593+0000","last_peered":"2024-09-19T11:26:28.937593+0000","last_clean":"2024-09-19T11:26:28.937593+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:28.937593+0000","last_undegraded":"2024-09-19T11:26:28.937593+0000","last_fullsized":"2024-09-19T11:26:28.937593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.937444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:51.513741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.828733+0000","last_change":"2024-09-19T11:26:52.828665+0000","last_active":"2024-09-19T11:26:52.828733+0000","last_peered":"2024-09-19T11:26:52.828733+0000","last_clean":"2024-09-19T11:26:52.828733+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:26:52.828733+0000","last_undegraded":"2024-09-19T11:26:52.828733+0000","last_fullsized":"2024-09-19T11:26:52.828733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.828574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'41","reported_seq":98,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.915307+0000","last_change":"2024-09-19T11:27:07.915209+0000","last_active":"2024-09-19T11:27:07.915307+0000","last_peered":"2024-09-19T11:27:07.915307+0000","last_clean":"2024-09-19T11:27:07.915307+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:27:07.915307+0000","last_undegraded":"2024-09-19T11:27:07.915307+0000","last_fullsized":"2024-09-19T11:27:07.915307+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'41","last_scrub_stamp":"2024-09-19T11:27:07.915099+0000","last_deep_scrub":"33'41","last_deep_scrub_stamp":"2024-09-19T11:27:07.915099+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.915099+0000","objects_scrubbed":5,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:28.937882+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:33.712113+0000","last_change":"2024-09-19T11:27:33.712035+0000","last_active":"2024-09-19T11:27:33.712113+0000","last_peered":"2024-09-19T11:27:33.712113+0000","last_clean":"2024-09-19T11:27:33.712113+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:27:33.712113+0000","last_undegraded":"2024-09-19T11:27:33.712113+0000","last_fullsized":"2024-09-19T11:27:33.712113+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:33.711945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:33.711945+0000","last_clean_scrub_stamp":"2024-09-19T11:27:33.711945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:16.723573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'4","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:38.649606+0000","last_change":"2024-09-19T11:27:38.649606+0000","last_active":"2024-09-19T11:27:38.649606+0000","last_peered":"2024-09-19T11:27:38.649606+0000","last_clean":"2024-09-19T11:27:38.649606+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:27:38.649606+0000","last_undegraded":"2024-09-19T11:27:38.649606+0000","last_fullsized":"2024-09-19T11:27:38.649606+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:38.649512+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:38.649512+0000","last_clean_scrub_stamp":"2024-09-19T11:27:38.649512+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:50:52.845888+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.393784+0000","last_change":"2024-09-19T11:26:56.393655+0000","last_active":"2024-09-19T11:26:56.393784+0000","last_peered":"2024-09-19T11:26:56.393784+0000","last_clean":"2024-09-19T11:26:56.393784+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:26:56.393784+0000","last_undegraded":"2024-09-19T11:26:56.393784+0000","last_fullsized":"2024-09-19T11:26:56.393784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.393557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:06.327682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.308945+0000","last_change":"2024-09-19T11:26:33.308889+0000","last_active":"2024-09-19T11:26:33.308945+0000","last_peered":"2024-09-19T11:26:33.308945+0000","last_clean":"2024-09-19T11:26:33.308945+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:26:33.308945+0000","last_undegraded":"2024-09-19T11:26:33.308945+0000","last_fullsized":"2024-09-19T11:26:33.308945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.308801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:19.525046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.829879+0000","last_change":"2024-09-19T11:26:57.829806+0000","last_active":"2024-09-19T11:26:57.829879+0000","last_peered":"2024-09-19T11:26:57.829879+0000","last_clean":"2024-09-19T11:26:57.829879+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:26:57.829879+0000","last_undegraded":"2024-09-19T11:26:57.829879+0000","last_fullsized":"2024-09-19T11:26:57.829879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.829722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:47.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'46","reported_seq":105,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.818978+0000","last_change":"2024-09-19T11:27:02.818874+0000","last_active":"2024-09-19T11:27:02.818978+0000","last_peered":"2024-09-19T11:27:02.818978+0000","last_clean":"2024-09-19T11:27:02.818978+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:27:02.818978+0000","last_undegraded":"2024-09-19T11:27:02.818978+0000","last_fullsized":"2024-09-19T11:27:02.818978+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:02.818754+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:02.818754+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.818754+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:54.439165+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'286","reported_seq":417,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.815155+0000","last_change":"2024-09-19T11:27:06.815074+0000","last_active":"2024-09-19T11:27:06.815155+0000","last_peered":"2024-09-19T11:27:06.815155+0000","last_clean":"2024-09-19T11:27:06.815155+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:27:06.815155+0000","last_undegraded":"2024-09-19T11:27:06.815155+0000","last_fullsized":"2024-09-19T11:27:06.815155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:06.814991+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:06.814991+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.814991+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:24.746148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'16","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:24.811986+0000","last_change":"2024-09-19T11:27:24.811936+0000","last_active":"2024-09-19T11:27:24.811986+0000","last_peered":"2024-09-19T11:27:24.811986+0000","last_clean":"2024-09-19T11:27:24.811986+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:27:24.811986+0000","last_undegraded":"2024-09-19T11:27:24.811986+0000","last_fullsized":"2024-09-19T11:27:24.811986+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:27:24.811864+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:27:24.811864+0000","last_clean_scrub_stamp":"2024-09-19T11:27:24.811864+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:58.503153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.810939+0000","last_change":"2024-09-19T11:26:41.810833+0000","last_active":"2024-09-19T11:26:41.810939+0000","last_peered":"2024-09-19T11:26:41.810939+0000","last_clean":"2024-09-19T11:26:41.810939+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:41.810939+0000","last_undegraded":"2024-09-19T11:26:41.810939+0000","last_fullsized":"2024-09-19T11:26:41.810939+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.810721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:31.441015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.269065+0000","last_change":"2024-09-19T11:26:31.268981+0000","last_active":"2024-09-19T11:26:31.269065+0000","last_peered":"2024-09-19T11:26:31.269065+0000","last_clean":"2024-09-19T11:26:31.269065+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:26:31.269065+0000","last_undegraded":"2024-09-19T11:26:31.269065+0000","last_fullsized":"2024-09-19T11:26:31.269065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.268893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:31.227733+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'5","reported_seq":429,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.945365+0000","last_change":"2024-09-19T11:26:58.945300+0000","last_active":"2024-09-19T11:26:58.945365+0000","last_peered":"2024-09-19T11:26:58.945365+0000","last_clean":"2024-09-19T11:26:58.945365+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:26:58.945365+0000","last_undegraded":"2024-09-19T11:26:58.945365+0000","last_fullsized":"2024-09-19T11:26:58.945365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.945214+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:11.607957+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"33'15","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:23.597656+0000","last_change":"2024-09-19T11:27:23.597594+0000","last_active":"2024-09-19T11:27:23.597656+0000","last_peered":"2024-09-19T11:27:23.597656+0000","last_clean":"2024-09-19T11:27:23.597656+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:27:23.597656+0000","last_undegraded":"2024-09-19T11:27:23.597656+0000","last_fullsized":"2024-09-19T11:27:23.597656+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'15","last_scrub_stamp":"2024-09-19T11:27:23.597495+0000","last_deep_scrub":"33'15","last_deep_scrub_stamp":"2024-09-19T11:27:23.597495+0000","last_clean_scrub_stamp":"2024-09-19T11:27:23.597495+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:50:04.826176+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'286","reported_seq":479,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.877425+0000","last_change":"2024-09-19T11:27:12.877361+0000","last_active":"2024-09-19T11:27:12.877425+0000","last_peered":"2024-09-19T11:27:12.877425+0000","last_clean":"2024-09-19T11:27:12.877425+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:27:12.877425+0000","last_undegraded":"2024-09-19T11:27:12.877425+0000","last_fullsized":"2024-09-19T11:27:12.877425+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:12.877271+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:12.877271+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.877271+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:29.229183+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.902199+0000","last_change":"2024-09-19T11:27:13.902118+0000","last_active":"2024-09-19T11:27:13.902199+0000","last_peered":"2024-09-19T11:27:13.902199+0000","last_clean":"2024-09-19T11:27:13.902199+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:27:13.902199+0000","last_undegraded":"2024-09-19T11:27:13.902199+0000","last_fullsized":"2024-09-19T11:27:13.902199+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:27:13.902022+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:27:13.902022+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.902022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:10.843373+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":110,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.028259+0000","last_change":"2024-09-19T11:26:42.028189+0000","last_active":"2024-09-19T11:26:42.028259+0000","last_peered":"2024-09-19T11:26:42.028259+0000","last_clean":"2024-09-19T11:26:42.028259+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:42.028259+0000","last_undegraded":"2024-09-19T11:26:42.028259+0000","last_fullsized":"2024-09-19T11:26:42.028259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'23","last_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_deep_scrub":"33'23","last_deep_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.028101+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.215685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.225710+0000","last_change":"2024-09-19T11:26:29.225653+0000","last_active":"2024-09-19T11:26:29.225710+0000","last_peered":"2024-09-19T11:26:29.225710+0000","last_clean":"2024-09-19T11:26:29.225710+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:26:29.225710+0000","last_undegraded":"2024-09-19T11:26:29.225710+0000","last_fullsized":"2024-09-19T11:26:29.225710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.225549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:47.310948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'7","reported_seq":431,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.565109+0000","last_change":"2024-09-19T11:27:16.565040+0000","last_active":"2024-09-19T11:27:16.565109+0000","last_peered":"2024-09-19T11:27:16.565109+0000","last_clean":"2024-09-19T11:27:16.565109+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:27:16.565109+0000","last_undegraded":"2024-09-19T11:27:16.565109+0000","last_fullsized":"2024-09-19T11:27:16.565109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:27:16.564928+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:27:16.564928+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.564928+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:52:13.697680+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":586,"num_read_kb":586,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'49","reported_seq":109,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.884490+0000","last_change":"2024-09-19T11:27:06.884406+0000","last_active":"2024-09-19T11:27:06.884490+0000","last_peered":"2024-09-19T11:27:06.884490+0000","last_clean":"2024-09-19T11:27:06.884490+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:27:06.884490+0000","last_undegraded":"2024-09-19T11:27:06.884490+0000","last_fullsized":"2024-09-19T11:27:06.884490+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'49","last_scrub_stamp":"2024-09-19T11:27:06.884296+0000","last_deep_scrub":"33'49","last_deep_scrub_stamp":"2024-09-19T11:27:06.884296+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.884296+0000","objects_scrubbed":8,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:16.451590+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.819681+0000","last_change":"2024-09-19T11:27:08.819603+0000","last_active":"2024-09-19T11:27:08.819681+0000","last_peered":"2024-09-19T11:27:08.819681+0000","last_clean":"2024-09-19T11:27:08.819681+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:27:08.819681+0000","last_undegraded":"2024-09-19T11:27:08.819681+0000","last_fullsized":"2024-09-19T11:27:08.819681+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:08.819506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:08.819506+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.819506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:54.217393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'7","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.938535+0000","last_change":"2024-09-19T11:27:16.938455+0000","last_active":"2024-09-19T11:27:16.938535+0000","last_peered":"2024-09-19T11:27:16.938535+0000","last_clean":"2024-09-19T11:27:16.938535+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:27:16.938535+0000","last_undegraded":"2024-09-19T11:27:16.938535+0000","last_fullsized":"2024-09-19T11:27:16.938535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:27:16.938366+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:27:16.938366+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.938366+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:08.561118+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'8","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.369733+0000","last_change":"2024-09-19T11:26:52.369670+0000","last_active":"2024-09-19T11:26:52.369733+0000","last_peered":"2024-09-19T11:26:52.369733+0000","last_clean":"2024-09-19T11:26:52.369733+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:52.369733+0000","last_undegraded":"2024-09-19T11:26:52.369733+0000","last_fullsized":"2024-09-19T11:26:52.369733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.369580+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:43.275474+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"33'74","reported_seq":157,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.856490+0000","last_change":"2024-09-19T11:26:23.856409+0000","last_active":"2024-09-19T11:26:23.856490+0000","last_peered":"2024-09-19T11:26:23.856490+0000","last_clean":"2024-09-19T11:26:23.856490+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:26:23.856490+0000","last_undegraded":"2024-09-19T11:26:23.856490+0000","last_fullsized":"2024-09-19T11:26:23.856490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'74","last_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_deep_scrub":"33'74","last_deep_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.856290+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:02.254545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'2","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.143039+0000","last_change":"2024-09-19T11:26:27.142962+0000","last_active":"2024-09-19T11:26:27.143039+0000","last_peered":"2024-09-19T11:26:27.143039+0000","last_clean":"2024-09-19T11:26:27.143039+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:26:27.143039+0000","last_undegraded":"2024-09-19T11:26:27.143039+0000","last_fullsized":"2024-09-19T11:26:27.143039+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.142873+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.914383+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.274051+0000","last_change":"2024-09-19T11:26:30.273990+0000","last_active":"2024-09-19T11:26:30.274051+0000","last_peered":"2024-09-19T11:26:30.274051+0000","last_clean":"2024-09-19T11:26:30.274051+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:26:30.274051+0000","last_undegraded":"2024-09-19T11:26:30.274051+0000","last_fullsized":"2024-09-19T11:26:30.274051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.273902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:45.300652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.985765+0000","last_change":"2024-09-19T11:26:56.985644+0000","last_active":"2024-09-19T11:26:56.985765+0000","last_peered":"2024-09-19T11:26:56.985765+0000","last_clean":"2024-09-19T11:26:56.985765+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:26:56.985765+0000","last_undegraded":"2024-09-19T11:26:56.985765+0000","last_fullsized":"2024-09-19T11:26:56.985765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.985544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:47.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'90","reported_seq":181,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.543157+0000","last_change":"2024-09-19T11:27:20.543099+0000","last_active":"2024-09-19T11:27:20.543157+0000","last_peered":"2024-09-19T11:27:20.543157+0000","last_clean":"2024-09-19T11:27:20.543157+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:27:20.543157+0000","last_undegraded":"2024-09-19T11:27:20.543157+0000","last_fullsized":"2024-09-19T11:27:20.543157+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'90","last_scrub_stamp":"2024-09-19T11:27:20.542999+0000","last_deep_scrub":"33'90","last_deep_scrub_stamp":"2024-09-19T11:27:20.542999+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.542999+0000","objects_scrubbed":5,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:14.545112+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.736754+0000","last_change":"2024-09-19T11:27:07.736634+0000","last_active":"2024-09-19T11:27:07.736754+0000","last_peered":"2024-09-19T11:27:07.736754+0000","last_clean":"2024-09-19T11:27:07.736754+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:27:07.736754+0000","last_undegraded":"2024-09-19T11:27:07.736754+0000","last_fullsized":"2024-09-19T11:27:07.736754+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.736538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.736538+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.736538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:27.768987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":172,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.953984+0000","last_change":"2024-09-19T11:27:15.953879+0000","last_active":"2024-09-19T11:27:15.953984+0000","last_peered":"2024-09-19T11:27:15.953984+0000","last_clean":"2024-09-19T11:27:15.953984+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:27:15.953984+0000","last_undegraded":"2024-09-19T11:27:15.953984+0000","last_fullsized":"2024-09-19T11:27:15.953984+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'81","last_scrub_stamp":"2024-09-19T11:27:15.953765+0000","last_deep_scrub":"33'81","last_deep_scrub_stamp":"2024-09-19T11:27:15.953765+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.953765+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:43.808777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'6","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.415571+0000","last_change":"2024-09-19T11:26:54.415515+0000","last_active":"2024-09-19T11:26:54.415571+0000","last_peered":"2024-09-19T11:26:54.415571+0000","last_clean":"2024-09-19T11:26:54.415571+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:26:54.415571+0000","last_undegraded":"2024-09-19T11:26:54.415571+0000","last_fullsized":"2024-09-19T11:26:54.415571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.415426+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:50.714030+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.079877+0000","last_change":"2024-09-19T11:26:24.079800+0000","last_active":"2024-09-19T11:26:24.079877+0000","last_peered":"2024-09-19T11:26:24.079877+0000","last_clean":"2024-09-19T11:26:24.079877+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:26:24.079877+0000","last_undegraded":"2024-09-19T11:26:24.079877+0000","last_fullsized":"2024-09-19T11:26:24.079877+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.079699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:48:18.789176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.923570+0000","last_change":"2024-09-19T11:26:25.923493+0000","last_active":"2024-09-19T11:26:25.923570+0000","last_peered":"2024-09-19T11:26:25.923570+0000","last_clean":"2024-09-19T11:26:25.923570+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:26:25.923570+0000","last_undegraded":"2024-09-19T11:26:25.923570+0000","last_fullsized":"2024-09-19T11:26:25.923570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.923407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:56.735995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.613109+0000","last_change":"2024-09-19T11:27:13.613049+0000","last_active":"2024-09-19T11:27:13.613109+0000","last_peered":"2024-09-19T11:27:13.613109+0000","last_clean":"2024-09-19T11:27:13.613109+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:27:13.613109+0000","last_undegraded":"2024-09-19T11:27:13.613109+0000","last_fullsized":"2024-09-19T11:27:13.613109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:13.612955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:13.612955+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.612955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:02.028356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'294","reported_seq":520,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.901143+0000","last_change":"2024-09-19T11:27:05.901072+0000","last_active":"2024-09-19T11:27:05.901143+0000","last_peered":"2024-09-19T11:27:05.901143+0000","last_clean":"2024-09-19T11:27:05.901143+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:27:05.901143+0000","last_undegraded":"2024-09-19T11:27:05.901143+0000","last_fullsized":"2024-09-19T11:27:05.901143+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'294","last_scrub_stamp":"2024-09-19T11:27:05.900953+0000","last_deep_scrub":"33'294","last_deep_scrub_stamp":"2024-09-19T11:27:05.900953+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.900953+0000","objects_scrubbed":10,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:53.378733+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:30.677490+0000","last_change":"2024-09-19T11:27:30.677490+0000","last_active":"2024-09-19T11:27:30.677490+0000","last_peered":"2024-09-19T11:27:30.677490+0000","last_clean":"2024-09-19T11:27:30.677490+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:27:30.677490+0000","last_undegraded":"2024-09-19T11:27:30.677490+0000","last_fullsized":"2024-09-19T11:27:30.677490+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:30.677403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:30.677403+0000","last_clean_scrub_stamp":"2024-09-19T11:27:30.677403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:32.369255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.945274+0000","last_change":"2024-09-19T11:27:19.945189+0000","last_active":"2024-09-19T11:27:19.945274+0000","last_peered":"2024-09-19T11:27:19.945274+0000","last_clean":"2024-09-19T11:27:19.945274+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:27:19.945274+0000","last_undegraded":"2024-09-19T11:27:19.945274+0000","last_fullsized":"2024-09-19T11:27:19.945274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:27:19.945103+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:27:19.945103+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.945103+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:34.266283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.967905+0000","last_change":"2024-09-19T11:26:39.967834+0000","last_active":"2024-09-19T11:26:39.967905+0000","last_peered":"2024-09-19T11:26:39.967905+0000","last_clean":"2024-09-19T11:26:39.967905+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:39.967905+0000","last_undegraded":"2024-09-19T11:26:39.967905+0000","last_fullsized":"2024-09-19T11:26:39.967905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.967748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:48.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.952580+0000","last_change":"2024-09-19T11:26:23.952524+0000","last_active":"2024-09-19T11:26:23.952580+0000","last_peered":"2024-09-19T11:26:23.952580+0000","last_clean":"2024-09-19T11:26:23.952580+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:26:23.952580+0000","last_undegraded":"2024-09-19T11:26:23.952580+0000","last_fullsized":"2024-09-19T11:26:23.952580+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.952436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:33.569870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.922891+0000","last_change":"2024-09-19T11:26:24.922822+0000","last_active":"2024-09-19T11:26:24.922891+0000","last_peered":"2024-09-19T11:26:24.922891+0000","last_clean":"2024-09-19T11:26:24.922891+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:26:24.922891+0000","last_undegraded":"2024-09-19T11:26:24.922891+0000","last_fullsized":"2024-09-19T11:26:24.922891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.922736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:47.473806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.833301+0000","last_change":"2024-09-19T11:26:53.833236+0000","last_active":"2024-09-19T11:26:53.833301+0000","last_peered":"2024-09-19T11:26:53.833301+0000","last_clean":"2024-09-19T11:26:53.833301+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:26:53.833301+0000","last_undegraded":"2024-09-19T11:26:53.833301+0000","last_fullsized":"2024-09-19T11:26:53.833301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.833148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:46.323076+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'24","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.835137+0000","last_change":"2024-09-19T11:26:59.835062+0000","last_active":"2024-09-19T11:26:59.835137+0000","last_peered":"2024-09-19T11:26:59.835137+0000","last_clean":"2024-09-19T11:26:59.835137+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:59.835137+0000","last_undegraded":"2024-09-19T11:26:59.835137+0000","last_fullsized":"2024-09-19T11:26:59.835137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'24","last_scrub_stamp":"2024-09-19T11:26:59.834946+0000","last_deep_scrub":"33'24","last_deep_scrub_stamp":"2024-09-19T11:26:59.834946+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.834946+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:40.523084+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.753997+0000","last_change":"2024-09-19T11:27:06.753941+0000","last_active":"2024-09-19T11:27:06.753997+0000","last_peered":"2024-09-19T11:27:06.753997+0000","last_clean":"2024-09-19T11:27:06.753997+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:27:06.753997+0000","last_undegraded":"2024-09-19T11:27:06.753997+0000","last_fullsized":"2024-09-19T11:27:06.753997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:06.753841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:06.753841+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.753841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:57.677312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:39.657173+0000","last_change":"2024-09-19T11:27:39.657173+0000","last_active":"2024-09-19T11:27:39.657173+0000","last_peered":"2024-09-19T11:27:39.657173+0000","last_clean":"2024-09-19T11:27:39.657173+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:27:39.657173+0000","last_undegraded":"2024-09-19T11:27:39.657173+0000","last_fullsized":"2024-09-19T11:27:39.657173+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:39.657088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:39.657088+0000","last_clean_scrub_stamp":"2024-09-19T11:27:39.657088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:33.294178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":68,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.977214+0000","last_change":"2024-09-19T11:26:37.977150+0000","last_active":"2024-09-19T11:26:37.977214+0000","last_peered":"2024-09-19T11:26:37.977214+0000","last_clean":"2024-09-19T11:26:37.977214+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:37.977214+0000","last_undegraded":"2024-09-19T11:26:37.977214+0000","last_fullsized":"2024-09-19T11:26:37.977214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.977048+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:20.097270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"33'3","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.109984+0000","last_change":"2024-09-19T11:26:25.109916+0000","last_active":"2024-09-19T11:26:25.109984+0000","last_peered":"2024-09-19T11:26:25.109984+0000","last_clean":"2024-09-19T11:26:25.109984+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:26:25.109984+0000","last_undegraded":"2024-09-19T11:26:25.109984+0000","last_fullsized":"2024-09-19T11:26:25.109984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.109824+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:48.020763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.938101+0000","last_change":"2024-09-19T11:26:25.938045+0000","last_active":"2024-09-19T11:26:25.938101+0000","last_peered":"2024-09-19T11:26:25.938101+0000","last_clean":"2024-09-19T11:26:25.938101+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:25.938101+0000","last_undegraded":"2024-09-19T11:26:25.938101+0000","last_fullsized":"2024-09-19T11:26:25.938101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.937963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:58:57.848583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.801326+0000","last_change":"2024-09-19T11:26:56.801257+0000","last_active":"2024-09-19T11:26:56.801326+0000","last_peered":"2024-09-19T11:26:56.801326+0000","last_clean":"2024-09-19T11:26:56.801326+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:26:56.801326+0000","last_undegraded":"2024-09-19T11:26:56.801326+0000","last_fullsized":"2024-09-19T11:26:56.801326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.801170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:01.667240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'20","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.527738+0000","last_change":"2024-09-19T11:27:19.527663+0000","last_active":"2024-09-19T11:27:19.527738+0000","last_peered":"2024-09-19T11:27:19.527738+0000","last_clean":"2024-09-19T11:27:19.527738+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:27:19.527738+0000","last_undegraded":"2024-09-19T11:27:19.527738+0000","last_fullsized":"2024-09-19T11:27:19.527738+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'20","last_scrub_stamp":"2024-09-19T11:27:19.527564+0000","last_deep_scrub":"33'20","last_deep_scrub_stamp":"2024-09-19T11:27:19.527564+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.527564+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:21.167261+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:31.653248+0000","last_change":"2024-09-19T11:27:31.653190+0000","last_active":"2024-09-19T11:27:31.653248+0000","last_peered":"2024-09-19T11:27:31.653248+0000","last_clean":"2024-09-19T11:27:31.653248+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:27:31.653248+0000","last_undegraded":"2024-09-19T11:27:31.653248+0000","last_fullsized":"2024-09-19T11:27:31.653248+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:31.653102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:31.653102+0000","last_clean_scrub_stamp":"2024-09-19T11:27:31.653102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:07.186773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.934985+0000","last_change":"2024-09-19T11:27:18.934889+0000","last_active":"2024-09-19T11:27:18.934985+0000","last_peered":"2024-09-19T11:27:18.934985+0000","last_clean":"2024-09-19T11:27:18.934985+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:27:18.934985+0000","last_undegraded":"2024-09-19T11:27:18.934985+0000","last_fullsized":"2024-09-19T11:27:18.934985+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:18.934732+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:18.934732+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.934732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:23.688618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.916131+0000","last_change":"2024-09-19T11:26:35.916064+0000","last_active":"2024-09-19T11:26:35.916131+0000","last_peered":"2024-09-19T11:26:35.916131+0000","last_clean":"2024-09-19T11:26:35.916131+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:35.916131+0000","last_undegraded":"2024-09-19T11:26:35.916131+0000","last_fullsized":"2024-09-19T11:26:35.916131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.915972+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:33.729135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.933807+0000","last_change":"2024-09-19T11:26:24.933741+0000","last_active":"2024-09-19T11:26:24.933807+0000","last_peered":"2024-09-19T11:26:24.933807+0000","last_clean":"2024-09-19T11:26:24.933807+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:26:24.933807+0000","last_undegraded":"2024-09-19T11:26:24.933807+0000","last_fullsized":"2024-09-19T11:26:24.933807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.933645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.620054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.910643+0000","last_change":"2024-09-19T11:26:22.910571+0000","last_active":"2024-09-19T11:26:22.910643+0000","last_peered":"2024-09-19T11:26:22.910643+0000","last_clean":"2024-09-19T11:26:22.910643+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:26:22.910643+0000","last_undegraded":"2024-09-19T11:26:22.910643+0000","last_fullsized":"2024-09-19T11:26:22.910643+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.910497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:20.068073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.866238+0000","last_change":"2024-09-19T11:26:52.866167+0000","last_active":"2024-09-19T11:26:52.866238+0000","last_peered":"2024-09-19T11:26:52.866238+0000","last_clean":"2024-09-19T11:26:52.866238+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:26:52.866238+0000","last_undegraded":"2024-09-19T11:26:52.866238+0000","last_fullsized":"2024-09-19T11:26:52.866238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.866082+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:47.631328+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'77","reported_seq":154,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.785719+0000","last_change":"2024-09-19T11:26:58.785636+0000","last_active":"2024-09-19T11:26:58.785719+0000","last_peered":"2024-09-19T11:26:58.785719+0000","last_clean":"2024-09-19T11:26:58.785719+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:26:58.785719+0000","last_undegraded":"2024-09-19T11:26:58.785719+0000","last_fullsized":"2024-09-19T11:26:58.785719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'77","last_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_deep_scrub":"33'77","last_deep_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.785526+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:40.190489+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.829172+0000","last_change":"2024-09-19T11:27:07.829111+0000","last_active":"2024-09-19T11:27:07.829172+0000","last_peered":"2024-09-19T11:27:07.829172+0000","last_clean":"2024-09-19T11:27:07.829172+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:27:07.829172+0000","last_undegraded":"2024-09-19T11:27:07.829172+0000","last_fullsized":"2024-09-19T11:27:07.829172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.829015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.829015+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.829015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:13.610649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.843376+0000","last_change":"2024-09-19T11:27:19.843376+0000","last_active":"2024-09-19T11:27:19.843376+0000","last_peered":"2024-09-19T11:27:19.843376+0000","last_clean":"2024-09-19T11:27:19.843376+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:27:19.843376+0000","last_undegraded":"2024-09-19T11:27:19.843376+0000","last_fullsized":"2024-09-19T11:27:19.843376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:27:19.843296+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:27:19.843296+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.843296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:43.970345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.952834+0000","last_change":"2024-09-19T11:26:36.952752+0000","last_active":"2024-09-19T11:26:36.952834+0000","last_peered":"2024-09-19T11:26:36.952834+0000","last_clean":"2024-09-19T11:26:36.952834+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:36.952834+0000","last_undegraded":"2024-09-19T11:26:36.952834+0000","last_fullsized":"2024-09-19T11:26:36.952834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.952660+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.496448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.969573+0000","last_change":"2024-09-19T11:26:38.969511+0000","last_active":"2024-09-19T11:26:38.969573+0000","last_peered":"2024-09-19T11:26:38.969573+0000","last_clean":"2024-09-19T11:26:38.969573+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:26:38.969573+0000","last_undegraded":"2024-09-19T11:26:38.969573+0000","last_fullsized":"2024-09-19T11:26:38.969573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.969423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:49.370950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.293541+0000","last_change":"2024-09-19T11:26:32.293468+0000","last_active":"2024-09-19T11:26:32.293541+0000","last_peered":"2024-09-19T11:26:32.293541+0000","last_clean":"2024-09-19T11:26:32.293541+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:26:32.293541+0000","last_undegraded":"2024-09-19T11:26:32.293541+0000","last_fullsized":"2024-09-19T11:26:32.293541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.293378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:48.996006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.538549+0000","last_change":"2024-09-19T11:27:10.538492+0000","last_active":"2024-09-19T11:27:10.538549+0000","last_peered":"2024-09-19T11:27:10.538549+0000","last_clean":"2024-09-19T11:27:10.538549+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:27:10.538549+0000","last_undegraded":"2024-09-19T11:27:10.538549+0000","last_fullsized":"2024-09-19T11:27:10.538549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:10.538407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:10.538407+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.538407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:33.331328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'22","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.936170+0000","last_change":"2024-09-19T11:27:04.936063+0000","last_active":"2024-09-19T11:27:04.936170+0000","last_peered":"2024-09-19T11:27:04.936170+0000","last_clean":"2024-09-19T11:27:04.936170+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:27:04.936170+0000","last_undegraded":"2024-09-19T11:27:04.936170+0000","last_fullsized":"2024-09-19T11:27:04.936170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'22","last_scrub_stamp":"2024-09-19T11:27:04.935933+0000","last_deep_scrub":"33'22","last_deep_scrub_stamp":"2024-09-19T11:27:04.935933+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.935933+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.148853+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'286","reported_seq":369,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.915549+0000","last_change":"2024-09-19T11:27:10.915484+0000","last_active":"2024-09-19T11:27:10.915549+0000","last_peered":"2024-09-19T11:27:10.915549+0000","last_clean":"2024-09-19T11:27:10.915549+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:27:10.915549+0000","last_undegraded":"2024-09-19T11:27:10.915549+0000","last_fullsized":"2024-09-19T11:27:10.915549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:10.915397+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:10.915397+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.915397+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:07.096881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:40.651836+0000","last_change":"2024-09-19T11:27:40.651836+0000","last_active":"2024-09-19T11:27:40.651836+0000","last_peered":"2024-09-19T11:27:40.651836+0000","last_clean":"2024-09-19T11:27:40.651836+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:27:40.651836+0000","last_undegraded":"2024-09-19T11:27:40.651836+0000","last_fullsized":"2024-09-19T11:27:40.651836+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:40.651729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:40.651729+0000","last_clean_scrub_stamp":"2024-09-19T11:27:40.651729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:01.744367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'12","reported_seq":79,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.381797+0000","last_change":"2024-09-19T11:26:53.381726+0000","last_active":"2024-09-19T11:26:53.381797+0000","last_peered":"2024-09-19T11:26:53.381797+0000","last_clean":"2024-09-19T11:26:53.381797+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:53.381797+0000","last_undegraded":"2024-09-19T11:26:53.381797+0000","last_fullsized":"2024-09-19T11:26:53.381797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.381618+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:56.071102+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.185454+0000","last_change":"2024-09-19T11:26:28.185387+0000","last_active":"2024-09-19T11:26:28.185454+0000","last_peered":"2024-09-19T11:26:28.185454+0000","last_clean":"2024-09-19T11:26:28.185454+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:26:28.185454+0000","last_undegraded":"2024-09-19T11:26:28.185454+0000","last_fullsized":"2024-09-19T11:26:28.185454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.185291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:48.217740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.897506+0000","last_change":"2024-09-19T11:26:23.897455+0000","last_active":"2024-09-19T11:26:23.897506+0000","last_peered":"2024-09-19T11:26:23.897506+0000","last_clean":"2024-09-19T11:26:23.897506+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:26:23.897506+0000","last_undegraded":"2024-09-19T11:26:23.897506+0000","last_fullsized":"2024-09-19T11:26:23.897506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.897373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:13.527667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.503825+0000","last_change":"2024-09-19T11:27:09.503759+0000","last_active":"2024-09-19T11:27:09.503825+0000","last_peered":"2024-09-19T11:27:09.503825+0000","last_clean":"2024-09-19T11:27:09.503825+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:27:09.503825+0000","last_undegraded":"2024-09-19T11:27:09.503825+0000","last_fullsized":"2024-09-19T11:27:09.503825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:09.503672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:09.503672+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.503672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:53.319658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"33'16","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.512281+0000","last_change":"2024-09-19T11:27:18.512211+0000","last_active":"2024-09-19T11:27:18.512281+0000","last_peered":"2024-09-19T11:27:18.512281+0000","last_clean":"2024-09-19T11:27:18.512281+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:27:18.512281+0000","last_undegraded":"2024-09-19T11:27:18.512281+0000","last_fullsized":"2024-09-19T11:27:18.512281+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:27:18.512110+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:27:18.512110+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.512110+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:13.848124+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'286","reported_seq":443,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.791846+0000","last_change":"2024-09-19T11:27:09.791720+0000","last_active":"2024-09-19T11:27:09.791846+0000","last_peered":"2024-09-19T11:27:09.791846+0000","last_clean":"2024-09-19T11:27:09.791846+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:27:09.791846+0000","last_undegraded":"2024-09-19T11:27:09.791846+0000","last_fullsized":"2024-09-19T11:27:09.791846+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:09.791625+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:09.791625+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.791625+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:24.818374+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.916756+0000","last_change":"2024-09-19T11:27:20.916756+0000","last_active":"2024-09-19T11:27:20.916756+0000","last_peered":"2024-09-19T11:27:20.916756+0000","last_clean":"2024-09-19T11:27:20.916756+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:27:20.916756+0000","last_undegraded":"2024-09-19T11:27:20.916756+0000","last_fullsized":"2024-09-19T11:27:20.916756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-09-19T11:27:20.916668+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-09-19T11:27:20.916668+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.916668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:56.260631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.850294+0000","last_change":"2024-09-19T11:26:39.850188+0000","last_active":"2024-09-19T11:26:39.850294+0000","last_peered":"2024-09-19T11:26:39.850294+0000","last_clean":"2024-09-19T11:26:39.850294+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:26:39.850294+0000","last_undegraded":"2024-09-19T11:26:39.850294+0000","last_fullsized":"2024-09-19T11:26:39.850294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.850077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:05.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.886575+0000","last_change":"2024-09-19T11:26:22.886518+0000","last_active":"2024-09-19T11:26:22.886575+0000","last_peered":"2024-09-19T11:26:22.886575+0000","last_clean":"2024-09-19T11:26:22.886575+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:26:22.886575+0000","last_undegraded":"2024-09-19T11:26:22.886575+0000","last_fullsized":"2024-09-19T11:26:22.886575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.886419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:07.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.146522+0000","last_change":"2024-09-19T11:26:26.146468+0000","last_active":"2024-09-19T11:26:26.146522+0000","last_peered":"2024-09-19T11:26:26.146522+0000","last_clean":"2024-09-19T11:26:26.146522+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:26:26.146522+0000","last_undegraded":"2024-09-19T11:26:26.146522+0000","last_fullsized":"2024-09-19T11:26:26.146522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.146376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:37.768766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.572672+0000","last_change":"2024-09-19T11:27:11.572672+0000","last_active":"2024-09-19T11:27:11.572672+0000","last_peered":"2024-09-19T11:27:11.572672+0000","last_clean":"2024-09-19T11:27:11.572672+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:27:11.572672+0000","last_undegraded":"2024-09-19T11:27:11.572672+0000","last_fullsized":"2024-09-19T11:27:11.572672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.572582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.572582+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.572582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:13.389084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'99","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.848493+0000","last_change":"2024-09-19T11:26:58.848423+0000","last_active":"2024-09-19T11:26:58.848493+0000","last_peered":"2024-09-19T11:26:58.848493+0000","last_clean":"2024-09-19T11:26:58.848493+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:26:58.848493+0000","last_undegraded":"2024-09-19T11:26:58.848493+0000","last_fullsized":"2024-09-19T11:26:58.848493+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'99","last_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_deep_scrub":"33'99","last_deep_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.848330+0000","objects_scrubbed":8,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:48.091262+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.742808+0000","last_change":"2024-09-19T11:27:05.742747+0000","last_active":"2024-09-19T11:27:05.742808+0000","last_peered":"2024-09-19T11:27:05.742808+0000","last_clean":"2024-09-19T11:27:05.742808+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:27:05.742808+0000","last_undegraded":"2024-09-19T11:27:05.742808+0000","last_fullsized":"2024-09-19T11:27:05.742808+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:05.742667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:05.742667+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.742667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.395498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.964661+0000","last_change":"2024-09-19T11:27:16.964584+0000","last_active":"2024-09-19T11:27:16.964661+0000","last_peered":"2024-09-19T11:27:16.964661+0000","last_clean":"2024-09-19T11:27:16.964661+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:27:16.964661+0000","last_undegraded":"2024-09-19T11:27:16.964661+0000","last_fullsized":"2024-09-19T11:27:16.964661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:16.964498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:16.964498+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.964498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:58.369461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.936328+0000","last_change":"2024-09-19T11:27:18.936226+0000","last_active":"2024-09-19T11:27:18.936328+0000","last_peered":"2024-09-19T11:27:18.936328+0000","last_clean":"2024-09-19T11:27:18.936328+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:27:18.936328+0000","last_undegraded":"2024-09-19T11:27:18.936328+0000","last_fullsized":"2024-09-19T11:27:18.936328+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:18.936125+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:18.936125+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.936125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:50.662583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.771752+0000","last_change":"2024-09-19T11:26:42.771683+0000","last_active":"2024-09-19T11:26:42.771752+0000","last_peered":"2024-09-19T11:26:42.771752+0000","last_clean":"2024-09-19T11:26:42.771752+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:26:42.771752+0000","last_undegraded":"2024-09-19T11:26:42.771752+0000","last_fullsized":"2024-09-19T11:26:42.771752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.771596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:48:05.322666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.396108+0000","last_change":"2024-09-19T11:26:38.396108+0000","last_active":"2024-09-19T11:26:38.396108+0000","last_peered":"2024-09-19T11:26:38.396108+0000","last_clean":"2024-09-19T11:26:38.396108+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:26:38.396108+0000","last_undegraded":"2024-09-19T11:26:38.396108+0000","last_fullsized":"2024-09-19T11:26:38.396108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.396006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:14.129195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.922151+0000","last_change":"2024-09-19T11:27:00.922151+0000","last_active":"2024-09-19T11:27:00.922151+0000","last_peered":"2024-09-19T11:27:00.922151+0000","last_clean":"2024-09-19T11:27:00.922151+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:27:00.922151+0000","last_undegraded":"2024-09-19T11:27:00.922151+0000","last_fullsized":"2024-09-19T11:27:00.922151+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.922068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:51.324340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'40","reported_seq":96,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:24.594970+0000","last_change":"2024-09-19T11:27:24.594970+0000","last_active":"2024-09-19T11:27:24.594970+0000","last_peered":"2024-09-19T11:27:24.594970+0000","last_clean":"2024-09-19T11:27:24.594970+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:27:24.594970+0000","last_undegraded":"2024-09-19T11:27:24.594970+0000","last_fullsized":"2024-09-19T11:27:24.594970+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'40","last_scrub_stamp":"2024-09-19T11:27:24.594878+0000","last_deep_scrub":"33'40","last_deep_scrub_stamp":"2024-09-19T11:27:24.594878+0000","last_clean_scrub_stamp":"2024-09-19T11:27:24.594878+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:48:58.860644+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:34.693641+0000","last_change":"2024-09-19T11:27:34.693641+0000","last_active":"2024-09-19T11:27:34.693641+0000","last_peered":"2024-09-19T11:27:34.693641+0000","last_clean":"2024-09-19T11:27:34.693641+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:27:34.693641+0000","last_undegraded":"2024-09-19T11:27:34.693641+0000","last_fullsized":"2024-09-19T11:27:34.693641+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:34.693536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:34.693536+0000","last_clean_scrub_stamp":"2024-09-19T11:27:34.693536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:09.572432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:23.851220+0000","last_change":"2024-09-19T11:27:23.851220+0000","last_active":"2024-09-19T11:27:23.851220+0000","last_peered":"2024-09-19T11:27:23.851220+0000","last_clean":"2024-09-19T11:27:23.851220+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:27:23.851220+0000","last_undegraded":"2024-09-19T11:27:23.851220+0000","last_fullsized":"2024-09-19T11:27:23.851220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:23.851126+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:23.851126+0000","last_clean_scrub_stamp":"2024-09-19T11:27:23.851126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:40.839787+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.036281+0000","last_change":"2024-09-19T11:26:41.036281+0000","last_active":"2024-09-19T11:26:41.036281+0000","last_peered":"2024-09-19T11:26:41.036281+0000","last_clean":"2024-09-19T11:26:41.036281+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:26:41.036281+0000","last_undegraded":"2024-09-19T11:26:41.036281+0000","last_fullsized":"2024-09-19T11:26:41.036281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.036169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:34.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.921998+0000","last_change":"2024-09-19T11:26:28.921998+0000","last_active":"2024-09-19T11:26:28.921998+0000","last_peered":"2024-09-19T11:26:28.921998+0000","last_clean":"2024-09-19T11:26:28.921998+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:26:28.921998+0000","last_undegraded":"2024-09-19T11:26:28.921998+0000","last_fullsized":"2024-09-19T11:26:28.921998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.921906+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:00.637589+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.583167+0000","last_change":"2024-09-19T11:27:14.583167+0000","last_active":"2024-09-19T11:27:14.583167+0000","last_peered":"2024-09-19T11:27:14.583167+0000","last_clean":"2024-09-19T11:27:14.583167+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:27:14.583167+0000","last_undegraded":"2024-09-19T11:27:14.583167+0000","last_fullsized":"2024-09-19T11:27:14.583167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.583070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.583070+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.583070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:47.425828+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'8","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.727321+0000","last_change":"2024-09-19T11:27:01.727321+0000","last_active":"2024-09-19T11:27:01.727321+0000","last_peered":"2024-09-19T11:27:01.727321+0000","last_clean":"2024-09-19T11:27:01.727321+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:27:01.727321+0000","last_undegraded":"2024-09-19T11:27:01.727321+0000","last_fullsized":"2024-09-19T11:27:01.727321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:01.727234+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:01.727234+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.727234+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:28.373856+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.813741+0000","last_change":"2024-09-19T11:27:09.813741+0000","last_active":"2024-09-19T11:27:09.813741+0000","last_peered":"2024-09-19T11:27:09.813741+0000","last_clean":"2024-09-19T11:27:09.813741+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:27:09.813741+0000","last_undegraded":"2024-09-19T11:27:09.813741+0000","last_fullsized":"2024-09-19T11:27:09.813741+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:09.813641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:09.813641+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.813641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:52:43.359127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:22.823406+0000","last_change":"2024-09-19T11:27:22.823406+0000","last_active":"2024-09-19T11:27:22.823406+0000","last_peered":"2024-09-19T11:27:22.823406+0000","last_clean":"2024-09-19T11:27:22.823406+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:27:22.823406+0000","last_undegraded":"2024-09-19T11:27:22.823406+0000","last_fullsized":"2024-09-19T11:27:22.823406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:22.823310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:22.823310+0000","last_clean_scrub_stamp":"2024-09-19T11:27:22.823310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:50.964355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.001963+0000","last_change":"2024-09-19T11:26:41.001963+0000","last_active":"2024-09-19T11:26:41.001963+0000","last_peered":"2024-09-19T11:26:41.001963+0000","last_clean":"2024-09-19T11:26:41.001963+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:41.001963+0000","last_undegraded":"2024-09-19T11:26:41.001963+0000","last_fullsized":"2024-09-19T11:26:41.001963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.001858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:06.599582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.937976+0000","last_change":"2024-09-19T11:26:29.937976+0000","last_active":"2024-09-19T11:26:29.937976+0000","last_peered":"2024-09-19T11:26:29.937976+0000","last_clean":"2024-09-19T11:26:29.937976+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:29.937976+0000","last_undegraded":"2024-09-19T11:26:29.937976+0000","last_fullsized":"2024-09-19T11:26:29.937976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.937882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:14.058304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.813274+0000","last_change":"2024-09-19T11:26:51.813274+0000","last_active":"2024-09-19T11:26:51.813274+0000","last_peered":"2024-09-19T11:26:51.813274+0000","last_clean":"2024-09-19T11:26:51.813274+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:26:51.813274+0000","last_undegraded":"2024-09-19T11:26:51.813274+0000","last_fullsized":"2024-09-19T11:26:51.813274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.813187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:32.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'49","reported_seq":121,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.759116+0000","last_change":"2024-09-19T11:27:00.759116+0000","last_active":"2024-09-19T11:27:00.759116+0000","last_peered":"2024-09-19T11:27:00.759116+0000","last_clean":"2024-09-19T11:27:00.759116+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:27:00.759116+0000","last_undegraded":"2024-09-19T11:27:00.759116+0000","last_fullsized":"2024-09-19T11:27:00.759116+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'49","last_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_deep_scrub":"33'49","last_deep_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.759010+0000","objects_scrubbed":6,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:16.347162+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.892506+0000","last_change":"2024-09-19T11:27:11.892506+0000","last_active":"2024-09-19T11:27:11.892506+0000","last_peered":"2024-09-19T11:27:11.892506+0000","last_clean":"2024-09-19T11:27:11.892506+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:27:11.892506+0000","last_undegraded":"2024-09-19T11:27:11.892506+0000","last_fullsized":"2024-09-19T11:27:11.892506+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.892416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.892416+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.892416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:56:17.528967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946240+0000","last_change":"2024-09-19T11:23:13.274850+0000","last_active":"2024-09-19T11:23:42.946240+0000","last_peered":"2024-09-19T11:23:42.946240+0000","last_clean":"2024-09-19T11:23:42.946240+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:23:42.946240+0000","last_undegraded":"2024-09-19T11:23:42.946240+0000","last_fullsized":"2024-09-19T11:23:42.946240+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:12:04.734850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.392197+0000","last_change":"2024-09-19T11:26:58.392197+0000","last_active":"2024-09-19T11:26:58.392197+0000","last_peered":"2024-09-19T11:26:58.392197+0000","last_clean":"2024-09-19T11:26:58.392197+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:26:58.392197+0000","last_undegraded":"2024-09-19T11:26:58.392197+0000","last_fullsized":"2024-09-19T11:26:58.392197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.392102+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:10.174526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.886507+0000","last_change":"2024-09-19T11:26:25.886507+0000","last_active":"2024-09-19T11:26:25.886507+0000","last_peered":"2024-09-19T11:26:25.886507+0000","last_clean":"2024-09-19T11:26:25.886507+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:26:25.886507+0000","last_undegraded":"2024-09-19T11:26:25.886507+0000","last_fullsized":"2024-09-19T11:26:25.886507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.886414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:38.573945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.584986+0000","last_change":"2024-09-19T11:27:12.584986+0000","last_active":"2024-09-19T11:27:12.584986+0000","last_peered":"2024-09-19T11:27:12.584986+0000","last_clean":"2024-09-19T11:27:12.584986+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:27:12.584986+0000","last_undegraded":"2024-09-19T11:27:12.584986+0000","last_fullsized":"2024-09-19T11:27:12.584986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:12.584897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:12.584897+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.584897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:55.169943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'38","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.810233+0000","last_change":"2024-09-19T11:27:03.810233+0000","last_active":"2024-09-19T11:27:03.810233+0000","last_peered":"2024-09-19T11:27:03.810233+0000","last_clean":"2024-09-19T11:27:03.810233+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:27:03.810233+0000","last_undegraded":"2024-09-19T11:27:03.810233+0000","last_fullsized":"2024-09-19T11:27:03.810233+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:03.810143+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:03.810143+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.810143+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:31.289316+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.822230+0000","last_change":"2024-09-19T11:27:11.822230+0000","last_active":"2024-09-19T11:27:11.822230+0000","last_peered":"2024-09-19T11:27:11.822230+0000","last_clean":"2024-09-19T11:27:11.822230+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:27:11.822230+0000","last_undegraded":"2024-09-19T11:27:11.822230+0000","last_fullsized":"2024-09-19T11:27:11.822230+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.822138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.822138+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.822138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:55.083098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.486672+0000","last_change":"2024-09-19T11:27:01.486612+0000","last_active":"2024-09-19T11:27:01.486672+0000","last_peered":"2024-09-19T11:27:01.486672+0000","last_clean":"2024-09-19T11:27:01.486672+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:27:01.486672+0000","last_undegraded":"2024-09-19T11:27:01.486672+0000","last_fullsized":"2024-09-19T11:27:01.486672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.486507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:11.419750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'4","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.386729+0000","last_change":"2024-09-19T11:26:39.386729+0000","last_active":"2024-09-19T11:26:39.386729+0000","last_peered":"2024-09-19T11:26:39.386729+0000","last_clean":"2024-09-19T11:26:39.386729+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:26:39.386729+0000","last_undegraded":"2024-09-19T11:26:39.386729+0000","last_fullsized":"2024-09-19T11:26:39.386729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.386636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:26.742196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.840087+0000","last_change":"2024-09-19T11:26:55.840003+0000","last_active":"2024-09-19T11:26:55.840087+0000","last_peered":"2024-09-19T11:26:55.840087+0000","last_clean":"2024-09-19T11:26:55.840087+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:26:55.840087+0000","last_undegraded":"2024-09-19T11:26:55.840087+0000","last_fullsized":"2024-09-19T11:26:55.840087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.241449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'67","reported_seq":140,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.770359+0000","last_change":"2024-09-19T11:27:03.770359+0000","last_active":"2024-09-19T11:27:03.770359+0000","last_peered":"2024-09-19T11:27:03.770359+0000","last_clean":"2024-09-19T11:27:03.770359+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:27:03.770359+0000","last_undegraded":"2024-09-19T11:27:03.770359+0000","last_fullsized":"2024-09-19T11:27:03.770359+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'67","last_scrub_stamp":"2024-09-19T11:27:03.770268+0000","last_deep_scrub":"33'67","last_deep_scrub_stamp":"2024-09-19T11:27:03.770268+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.770268+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.209518+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":65,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.859547+0000","last_change":"2024-09-19T11:27:10.859547+0000","last_active":"2024-09-19T11:27:10.859547+0000","last_peered":"2024-09-19T11:27:10.859547+0000","last_clean":"2024-09-19T11:27:10.859547+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:27:10.859547+0000","last_undegraded":"2024-09-19T11:27:10.859547+0000","last_fullsized":"2024-09-19T11:27:10.859547+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:10.859446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:10.859446+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.859446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:44.106754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.861251+0000","last_change":"2024-09-19T11:27:20.861251+0000","last_active":"2024-09-19T11:27:20.861251+0000","last_peered":"2024-09-19T11:27:20.861251+0000","last_clean":"2024-09-19T11:27:20.861251+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:27:20.861251+0000","last_undegraded":"2024-09-19T11:27:20.861251+0000","last_fullsized":"2024-09-19T11:27:20.861251+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:27:20.861155+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:27:20.861155+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.861155+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:45.440786+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.954173+0000","last_change":"2024-09-19T11:26:43.954113+0000","last_active":"2024-09-19T11:26:43.954173+0000","last_peered":"2024-09-19T11:26:43.954173+0000","last_clean":"2024-09-19T11:26:43.954173+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:43.954173+0000","last_undegraded":"2024-09-19T11:26:43.954173+0000","last_fullsized":"2024-09-19T11:26:43.954173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.954028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:24.870979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.929509+0000","last_change":"2024-09-19T11:26:26.929509+0000","last_active":"2024-09-19T11:26:26.929509+0000","last_peered":"2024-09-19T11:26:26.929509+0000","last_clean":"2024-09-19T11:26:26.929509+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:26:26.929509+0000","last_undegraded":"2024-09-19T11:26:26.929509+0000","last_fullsized":"2024-09-19T11:26:26.929509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.929422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:53.997533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.861748+0000","last_change":"2024-09-19T11:27:00.861682+0000","last_active":"2024-09-19T11:27:00.861748+0000","last_peered":"2024-09-19T11:27:00.861748+0000","last_clean":"2024-09-19T11:27:00.861748+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:27:00.861748+0000","last_undegraded":"2024-09-19T11:27:00.861748+0000","last_fullsized":"2024-09-19T11:27:00.861748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.861599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.861599+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.861599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:41.778551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'46","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:29.645354+0000","last_change":"2024-09-19T11:27:29.645354+0000","last_active":"2024-09-19T11:27:29.645354+0000","last_peered":"2024-09-19T11:27:29.645354+0000","last_clean":"2024-09-19T11:27:29.645354+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:27:29.645354+0000","last_undegraded":"2024-09-19T11:27:29.645354+0000","last_fullsized":"2024-09-19T11:27:29.645354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:29.645252+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:29.645252+0000","last_clean_scrub_stamp":"2024-09-19T11:27:29.645252+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:55:18.923250+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.953402+0000","last_change":"2024-09-19T11:27:15.953402+0000","last_active":"2024-09-19T11:27:15.953402+0000","last_peered":"2024-09-19T11:27:15.953402+0000","last_clean":"2024-09-19T11:27:15.953402+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:27:15.953402+0000","last_undegraded":"2024-09-19T11:27:15.953402+0000","last_fullsized":"2024-09-19T11:27:15.953402+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.953297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.953297+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.953297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:28.014945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.943714+0000","last_change":"2024-09-19T11:23:13.273469+0000","last_active":"2024-09-19T11:23:42.943714+0000","last_peered":"2024-09-19T11:23:42.943714+0000","last_clean":"2024-09-19T11:23:42.943714+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:23:42.943714+0000","last_undegraded":"2024-09-19T11:23:42.943714+0000","last_fullsized":"2024-09-19T11:23:42.943714+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.473757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.474660+0000","last_change":"2024-09-19T11:27:00.474660+0000","last_active":"2024-09-19T11:27:00.474660+0000","last_peered":"2024-09-19T11:27:00.474660+0000","last_clean":"2024-09-19T11:27:00.474660+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:27:00.474660+0000","last_undegraded":"2024-09-19T11:27:00.474660+0000","last_fullsized":"2024-09-19T11:27:00.474660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.474573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:59.322724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.877728+0000","last_change":"2024-09-19T11:26:28.877728+0000","last_active":"2024-09-19T11:26:28.877728+0000","last_peered":"2024-09-19T11:26:28.877728+0000","last_clean":"2024-09-19T11:26:28.877728+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:26:28.877728+0000","last_undegraded":"2024-09-19T11:26:28.877728+0000","last_fullsized":"2024-09-19T11:26:28.877728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.877640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:09.086406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.801547+0000","last_change":"2024-09-19T11:26:57.801547+0000","last_active":"2024-09-19T11:26:57.801547+0000","last_peered":"2024-09-19T11:26:57.801547+0000","last_clean":"2024-09-19T11:26:57.801547+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:26:57.801547+0000","last_undegraded":"2024-09-19T11:26:57.801547+0000","last_fullsized":"2024-09-19T11:26:57.801547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.801457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:23.399181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'42","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.773714+0000","last_change":"2024-09-19T11:27:04.773660+0000","last_active":"2024-09-19T11:27:04.773714+0000","last_peered":"2024-09-19T11:27:04.773714+0000","last_clean":"2024-09-19T11:27:04.773714+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:27:04.773714+0000","last_undegraded":"2024-09-19T11:27:04.773714+0000","last_fullsized":"2024-09-19T11:27:04.773714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'42","last_scrub_stamp":"2024-09-19T11:27:04.773575+0000","last_deep_scrub":"33'42","last_deep_scrub_stamp":"2024-09-19T11:27:04.773575+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.773575+0000","objects_scrubbed":5,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:44.880940+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.829408+0000","last_change":"2024-09-19T11:27:16.829408+0000","last_active":"2024-09-19T11:27:16.829408+0000","last_peered":"2024-09-19T11:27:16.829408+0000","last_clean":"2024-09-19T11:27:16.829408+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:27:16.829408+0000","last_undegraded":"2024-09-19T11:27:16.829408+0000","last_fullsized":"2024-09-19T11:27:16.829408+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:16.829319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:16.829319+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.829319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:59.496334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.506705+0000","last_change":"2024-09-19T11:23:13.273390+0000","last_active":"2024-09-19T11:25:11.506705+0000","last_peered":"2024-09-19T11:25:11.506705+0000","last_clean":"2024-09-19T11:25:11.506705+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:25:11.506705+0000","last_undegraded":"2024-09-19T11:25:11.506705+0000","last_fullsized":"2024-09-19T11:25:11.506705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:15:45.777523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.000020+0000","last_change":"2024-09-19T11:26:43.000020+0000","last_active":"2024-09-19T11:26:43.000020+0000","last_peered":"2024-09-19T11:26:43.000020+0000","last_clean":"2024-09-19T11:26:43.000020+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:26:43.000020+0000","last_undegraded":"2024-09-19T11:26:43.000020+0000","last_fullsized":"2024-09-19T11:26:43.000020+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.999915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:23.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.342895+0000","last_change":"2024-09-19T11:26:40.342895+0000","last_active":"2024-09-19T11:26:40.342895+0000","last_peered":"2024-09-19T11:26:40.342895+0000","last_clean":"2024-09-19T11:26:40.342895+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:26:40.342895+0000","last_undegraded":"2024-09-19T11:26:40.342895+0000","last_fullsized":"2024-09-19T11:26:40.342895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.342808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.126397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.896501+0000","last_change":"2024-09-19T11:27:03.896501+0000","last_active":"2024-09-19T11:27:03.896501+0000","last_peered":"2024-09-19T11:27:03.896501+0000","last_clean":"2024-09-19T11:27:03.896501+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:27:03.896501+0000","last_undegraded":"2024-09-19T11:27:03.896501+0000","last_fullsized":"2024-09-19T11:27:03.896501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:03.896413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:03.896413+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.896413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:17.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"33'15","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:32.675558+0000","last_change":"2024-09-19T11:27:32.675558+0000","last_active":"2024-09-19T11:27:32.675558+0000","last_peered":"2024-09-19T11:27:32.675558+0000","last_clean":"2024-09-19T11:27:32.675558+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:27:32.675558+0000","last_undegraded":"2024-09-19T11:27:32.675558+0000","last_fullsized":"2024-09-19T11:27:32.675558+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'15","last_scrub_stamp":"2024-09-19T11:27:32.675456+0000","last_deep_scrub":"33'15","last_deep_scrub_stamp":"2024-09-19T11:27:32.675456+0000","last_clean_scrub_stamp":"2024-09-19T11:27:32.675456+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:09.848845+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'286","reported_seq":431,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.871742+0000","last_change":"2024-09-19T11:27:12.871624+0000","last_active":"2024-09-19T11:27:12.871742+0000","last_peered":"2024-09-19T11:27:12.871742+0000","last_clean":"2024-09-19T11:27:12.871742+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:27:12.871742+0000","last_undegraded":"2024-09-19T11:27:12.871742+0000","last_fullsized":"2024-09-19T11:27:12.871742+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:12.871525+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:12.871525+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.871525+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:18.160862+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:41.641595+0000","last_change":"2024-09-19T11:27:41.641595+0000","last_active":"2024-09-19T11:27:41.641595+0000","last_peered":"2024-09-19T11:27:41.641595+0000","last_clean":"2024-09-19T11:27:41.641595+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:27:41.641595+0000","last_undegraded":"2024-09-19T11:27:41.641595+0000","last_fullsized":"2024-09-19T11:27:41.641595+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:41.641507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:41.641507+0000","last_clean_scrub_stamp":"2024-09-19T11:27:41.641507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:08.499395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.773400+0000","last_change":"2024-09-19T11:26:47.773400+0000","last_active":"2024-09-19T11:26:47.773400+0000","last_peered":"2024-09-19T11:26:47.773400+0000","last_clean":"2024-09-19T11:26:47.773400+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:47.773400+0000","last_undegraded":"2024-09-19T11:26:47.773400+0000","last_fullsized":"2024-09-19T11:26:47.773400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.773301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:48.002135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.907839+0000","last_change":"2024-09-19T11:26:27.907839+0000","last_active":"2024-09-19T11:26:27.907839+0000","last_peered":"2024-09-19T11:26:27.907839+0000","last_clean":"2024-09-19T11:26:27.907839+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:26:27.907839+0000","last_undegraded":"2024-09-19T11:26:27.907839+0000","last_fullsized":"2024-09-19T11:26:27.907839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.907726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:48.160182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.884954+0000","last_change":"2024-09-19T11:27:02.884954+0000","last_active":"2024-09-19T11:27:02.884954+0000","last_peered":"2024-09-19T11:27:02.884954+0000","last_clean":"2024-09-19T11:27:02.884954+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:27:02.884954+0000","last_undegraded":"2024-09-19T11:27:02.884954+0000","last_fullsized":"2024-09-19T11:27:02.884954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:02.884869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:02.884869+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.884869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:00.122518+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'46","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.959960+0000","last_change":"2024-09-19T11:27:08.959960+0000","last_active":"2024-09-19T11:27:08.959960+0000","last_peered":"2024-09-19T11:27:08.959960+0000","last_clean":"2024-09-19T11:27:08.959960+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:27:08.959960+0000","last_undegraded":"2024-09-19T11:27:08.959960+0000","last_fullsized":"2024-09-19T11:27:08.959960+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:08.959862+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:08.959862+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.959862+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:37.904449+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.836563+0000","last_change":"2024-09-19T11:27:14.836563+0000","last_active":"2024-09-19T11:27:14.836563+0000","last_peered":"2024-09-19T11:27:14.836563+0000","last_clean":"2024-09-19T11:27:14.836563+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:27:14.836563+0000","last_undegraded":"2024-09-19T11:27:14.836563+0000","last_fullsized":"2024-09-19T11:27:14.836563+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.836468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.836468+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.836468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:05.472792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:21.916333+0000","last_change":"2024-09-19T11:27:21.916333+0000","last_active":"2024-09-19T11:27:21.916333+0000","last_peered":"2024-09-19T11:27:21.916333+0000","last_clean":"2024-09-19T11:27:21.916333+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:27:21.916333+0000","last_undegraded":"2024-09-19T11:27:21.916333+0000","last_fullsized":"2024-09-19T11:27:21.916333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:21.916240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:21.916240+0000","last_clean_scrub_stamp":"2024-09-19T11:27:21.916240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:27:07.280285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.724840+0000","last_change":"2024-09-19T11:26:46.724840+0000","last_active":"2024-09-19T11:26:46.724840+0000","last_peered":"2024-09-19T11:26:46.724840+0000","last_clean":"2024-09-19T11:26:46.724840+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:46.724840+0000","last_undegraded":"2024-09-19T11:26:46.724840+0000","last_fullsized":"2024-09-19T11:26:46.724840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.724740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:47.953525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.382940+0000","last_change":"2024-09-19T11:26:41.382940+0000","last_active":"2024-09-19T11:26:41.382940+0000","last_peered":"2024-09-19T11:26:41.382940+0000","last_clean":"2024-09-19T11:26:41.382940+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:26:41.382940+0000","last_undegraded":"2024-09-19T11:26:41.382940+0000","last_fullsized":"2024-09-19T11:26:41.382940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.382839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:55.560428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.798773+0000","last_change":"2024-09-19T11:26:56.798773+0000","last_active":"2024-09-19T11:26:56.798773+0000","last_peered":"2024-09-19T11:26:56.798773+0000","last_clean":"2024-09-19T11:26:56.798773+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:26:56.798773+0000","last_undegraded":"2024-09-19T11:26:56.798773+0000","last_fullsized":"2024-09-19T11:26:56.798773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.798674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:05.764652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'38","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.766754+0000","last_change":"2024-09-19T11:27:02.766754+0000","last_active":"2024-09-19T11:27:02.766754+0000","last_peered":"2024-09-19T11:27:02.766754+0000","last_clean":"2024-09-19T11:27:02.766754+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:27:02.766754+0000","last_undegraded":"2024-09-19T11:27:02.766754+0000","last_fullsized":"2024-09-19T11:27:02.766754+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:02.766665+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:02.766665+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.766665+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:23.822170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:36.641873+0000","last_change":"2024-09-19T11:27:36.641873+0000","last_active":"2024-09-19T11:27:36.641873+0000","last_peered":"2024-09-19T11:27:36.641873+0000","last_clean":"2024-09-19T11:27:36.641873+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:27:36.641873+0000","last_undegraded":"2024-09-19T11:27:36.641873+0000","last_fullsized":"2024-09-19T11:27:36.641873+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:36.641782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:36.641782+0000","last_clean_scrub_stamp":"2024-09-19T11:27:36.641782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:48.099365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:23:42.946323+0000","last_change":"2024-09-19T11:23:13.273745+0000","last_active":"2024-09-19T11:23:42.946323+0000","last_peered":"2024-09-19T11:23:42.946323+0000","last_clean":"2024-09-19T11:23:42.946323+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:23:42.946323+0000","last_undegraded":"2024-09-19T11:23:42.946323+0000","last_fullsized":"2024-09-19T11:23:42.946323+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:13.294520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.728999+0000","last_change":"2024-09-19T11:26:45.728999+0000","last_active":"2024-09-19T11:26:45.728999+0000","last_peered":"2024-09-19T11:26:45.728999+0000","last_clean":"2024-09-19T11:26:45.728999+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:45.728999+0000","last_undegraded":"2024-09-19T11:26:45.728999+0000","last_fullsized":"2024-09-19T11:26:45.728999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.728900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:26.277302+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.898920+0000","last_change":"2024-09-19T11:26:26.898841+0000","last_active":"2024-09-19T11:26:26.898920+0000","last_peered":"2024-09-19T11:26:26.898920+0000","last_clean":"2024-09-19T11:26:26.898920+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:26:26.898920+0000","last_undegraded":"2024-09-19T11:26:26.898920+0000","last_fullsized":"2024-09-19T11:26:26.898920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.898758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:04.348903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.929630+0000","last_change":"2024-09-19T11:26:59.929630+0000","last_active":"2024-09-19T11:26:59.929630+0000","last_peered":"2024-09-19T11:26:59.929630+0000","last_clean":"2024-09-19T11:26:59.929630+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:26:59.929630+0000","last_undegraded":"2024-09-19T11:26:59.929630+0000","last_fullsized":"2024-09-19T11:26:59.929630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.929544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:14.552058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'31","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:27.654733+0000","last_change":"2024-09-19T11:27:27.654733+0000","last_active":"2024-09-19T11:27:27.654733+0000","last_peered":"2024-09-19T11:27:27.654733+0000","last_clean":"2024-09-19T11:27:27.654733+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:27:27.654733+0000","last_undegraded":"2024-09-19T11:27:27.654733+0000","last_fullsized":"2024-09-19T11:27:27.654733+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'31","last_scrub_stamp":"2024-09-19T11:27:27.654639+0000","last_deep_scrub":"33'31","last_deep_scrub_stamp":"2024-09-19T11:27:27.654639+0000","last_clean_scrub_stamp":"2024-09-19T11:27:27.654639+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:43.311480+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:37.662963+0000","last_change":"2024-09-19T11:27:37.662963+0000","last_active":"2024-09-19T11:27:37.662963+0000","last_peered":"2024-09-19T11:27:37.662963+0000","last_clean":"2024-09-19T11:27:37.662963+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:27:37.662963+0000","last_undegraded":"2024-09-19T11:27:37.662963+0000","last_fullsized":"2024-09-19T11:27:37.662963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:37.662870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:37.662870+0000","last_clean_scrub_stamp":"2024-09-19T11:27:37.662870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:05.648114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:09.765840+0000","last_change":"2024-09-19T11:23:13.274766+0000","last_active":"2024-09-19T11:25:09.765840+0000","last_peered":"2024-09-19T11:25:09.765840+0000","last_clean":"2024-09-19T11:25:09.765840+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:25:09.765840+0000","last_undegraded":"2024-09-19T11:25:09.765840+0000","last_fullsized":"2024-09-19T11:25:09.765840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:01:53.277099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.773207+0000","last_change":"2024-09-19T11:26:44.773207+0000","last_active":"2024-09-19T11:26:44.773207+0000","last_peered":"2024-09-19T11:26:44.773207+0000","last_clean":"2024-09-19T11:26:44.773207+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:26:44.773207+0000","last_undegraded":"2024-09-19T11:26:44.773207+0000","last_fullsized":"2024-09-19T11:26:44.773207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.773116+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:30.345346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.916369+0000","last_change":"2024-09-19T11:26:29.916292+0000","last_active":"2024-09-19T11:26:29.916369+0000","last_peered":"2024-09-19T11:26:29.916369+0000","last_clean":"2024-09-19T11:26:29.916369+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:26:29.916369+0000","last_undegraded":"2024-09-19T11:26:29.916369+0000","last_fullsized":"2024-09-19T11:26:29.916369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.916202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:50.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'3","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.924054+0000","last_change":"2024-09-19T11:27:01.924054+0000","last_active":"2024-09-19T11:27:01.924054+0000","last_peered":"2024-09-19T11:27:01.924054+0000","last_clean":"2024-09-19T11:27:01.924054+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:27:01.924054+0000","last_undegraded":"2024-09-19T11:27:01.924054+0000","last_fullsized":"2024-09-19T11:27:01.924054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:01.923971+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:01.923971+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.923971+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:26:44.012364+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'42","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:26.635927+0000","last_change":"2024-09-19T11:27:26.635927+0000","last_active":"2024-09-19T11:27:26.635927+0000","last_peered":"2024-09-19T11:27:26.635927+0000","last_clean":"2024-09-19T11:27:26.635927+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:27:26.635927+0000","last_undegraded":"2024-09-19T11:27:26.635927+0000","last_fullsized":"2024-09-19T11:27:26.635927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'42","last_scrub_stamp":"2024-09-19T11:27:26.635831+0000","last_deep_scrub":"33'42","last_deep_scrub_stamp":"2024-09-19T11:27:26.635831+0000","last_clean_scrub_stamp":"2024-09-19T11:27:26.635831+0000","objects_scrubbed":5,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:23.640501+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.837684+0000","last_change":"2024-09-19T11:27:15.837684+0000","last_active":"2024-09-19T11:27:15.837684+0000","last_peered":"2024-09-19T11:27:15.837684+0000","last_clean":"2024-09-19T11:27:15.837684+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:27:15.837684+0000","last_undegraded":"2024-09-19T11:27:15.837684+0000","last_fullsized":"2024-09-19T11:27:15.837684+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.837602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.837602+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.837602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:39.151539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:25:11.508246+0000","last_change":"2024-09-19T11:23:13.274653+0000","last_active":"2024-09-19T11:25:11.508246+0000","last_peered":"2024-09-19T11:25:11.508246+0000","last_clean":"2024-09-19T11:25:11.508246+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:25:11.508246+0000","last_undegraded":"2024-09-19T11:25:11.508246+0000","last_fullsized":"2024-09-19T11:25:11.508246+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:23:12.244923+0000","last_clean_scrub_stamp":"2024-09-19T11:23:12.244923+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:13.776298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.120490+0000","last_change":"2024-09-19T11:26:45.120385+0000","last_active":"2024-09-19T11:26:45.120490+0000","last_peered":"2024-09-19T11:26:45.120490+0000","last_clean":"2024-09-19T11:26:45.120490+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:26:45.120490+0000","last_undegraded":"2024-09-19T11:26:45.120490+0000","last_fullsized":"2024-09-19T11:26:45.120490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.120277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:55.577999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.949943+0000","last_change":"2024-09-19T11:26:31.949825+0000","last_active":"2024-09-19T11:26:31.949943+0000","last_peered":"2024-09-19T11:26:31.949943+0000","last_clean":"2024-09-19T11:26:31.949943+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:26:31.949943+0000","last_undegraded":"2024-09-19T11:26:31.949943+0000","last_fullsized":"2024-09-19T11:26:31.949943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.949702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:38.587259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'18","reported_seq":76,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.879892+0000","last_change":"2024-09-19T11:26:46.879819+0000","last_active":"2024-09-19T11:26:46.879892+0000","last_peered":"2024-09-19T11:26:46.879892+0000","last_clean":"2024-09-19T11:26:46.879892+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:46.879892+0000","last_undegraded":"2024-09-19T11:26:46.879892+0000","last_fullsized":"2024-09-19T11:26:46.879892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'18","last_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_deep_scrub":"33'18","last_deep_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.879729+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:14.358902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.905354+0000","last_change":"2024-09-19T11:26:27.905294+0000","last_active":"2024-09-19T11:26:27.905354+0000","last_peered":"2024-09-19T11:26:27.905354+0000","last_clean":"2024-09-19T11:26:27.905354+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:26:27.905354+0000","last_undegraded":"2024-09-19T11:26:27.905354+0000","last_fullsized":"2024-09-19T11:26:27.905354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.905202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:43:38.666093+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.503655+0000","last_change":"2024-09-19T11:27:04.503595+0000","last_active":"2024-09-19T11:27:04.503655+0000","last_peered":"2024-09-19T11:27:04.503655+0000","last_clean":"2024-09-19T11:27:04.503655+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:27:04.503655+0000","last_undegraded":"2024-09-19T11:27:04.503655+0000","last_fullsized":"2024-09-19T11:27:04.503655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:04.503510+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:04.503510+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.503510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:01:27.895549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.388722+0000","last_change":"2024-09-19T11:26:42.388646+0000","last_active":"2024-09-19T11:26:42.388722+0000","last_peered":"2024-09-19T11:26:42.388722+0000","last_clean":"2024-09-19T11:26:42.388722+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:26:42.388722+0000","last_undegraded":"2024-09-19T11:26:42.388722+0000","last_fullsized":"2024-09-19T11:26:42.388722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.388560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:26.548057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.897022+0000","last_change":"2024-09-19T11:26:45.896965+0000","last_active":"2024-09-19T11:26:45.897022+0000","last_peered":"2024-09-19T11:26:45.897022+0000","last_clean":"2024-09-19T11:26:45.897022+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:26:45.897022+0000","last_undegraded":"2024-09-19T11:26:45.897022+0000","last_fullsized":"2024-09-19T11:26:45.897022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.896872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:37.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.953844+0000","last_change":"2024-09-19T11:26:30.953783+0000","last_active":"2024-09-19T11:26:30.953844+0000","last_peered":"2024-09-19T11:26:30.953844+0000","last_clean":"2024-09-19T11:26:30.953844+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:26:30.953844+0000","last_undegraded":"2024-09-19T11:26:30.953844+0000","last_fullsized":"2024-09-19T11:26:30.953844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.953693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:32.144901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.913054+0000","last_change":"2024-09-19T11:26:49.912991+0000","last_active":"2024-09-19T11:26:49.913054+0000","last_peered":"2024-09-19T11:26:49.913054+0000","last_clean":"2024-09-19T11:26:49.913054+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:49.913054+0000","last_undegraded":"2024-09-19T11:26:49.913054+0000","last_fullsized":"2024-09-19T11:26:49.913054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.912905+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:07.844062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.952205+0000","last_change":"2024-09-19T11:26:29.952079+0000","last_active":"2024-09-19T11:26:29.952205+0000","last_peered":"2024-09-19T11:26:29.952205+0000","last_clean":"2024-09-19T11:26:29.952205+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:29.952205+0000","last_undegraded":"2024-09-19T11:26:29.952205+0000","last_fullsized":"2024-09-19T11:26:29.952205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.951980+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:30.180814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.104057+0000","last_change":"2024-09-19T11:26:44.103981+0000","last_active":"2024-09-19T11:26:44.104057+0000","last_peered":"2024-09-19T11:26:44.104057+0000","last_clean":"2024-09-19T11:26:44.104057+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:26:44.104057+0000","last_undegraded":"2024-09-19T11:26:44.104057+0000","last_fullsized":"2024-09-19T11:26:44.104057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.103889+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:14.482038+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.867345+0000","last_change":"2024-09-19T11:26:30.867275+0000","last_active":"2024-09-19T11:26:30.867345+0000","last_peered":"2024-09-19T11:26:30.867345+0000","last_clean":"2024-09-19T11:26:30.867345+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:26:30.867345+0000","last_undegraded":"2024-09-19T11:26:30.867345+0000","last_fullsized":"2024-09-19T11:26:30.867345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.867184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:38.055814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.932867+0000","last_change":"2024-09-19T11:26:44.932807+0000","last_active":"2024-09-19T11:26:44.932867+0000","last_peered":"2024-09-19T11:26:44.932867+0000","last_clean":"2024-09-19T11:26:44.932867+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:26:44.932867+0000","last_undegraded":"2024-09-19T11:26:44.932867+0000","last_fullsized":"2024-09-19T11:26:44.932867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.932715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:07.921677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.978046+0000","last_change":"2024-09-19T11:26:32.977981+0000","last_active":"2024-09-19T11:26:32.978046+0000","last_peered":"2024-09-19T11:26:32.978046+0000","last_clean":"2024-09-19T11:26:32.978046+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:32.978046+0000","last_undegraded":"2024-09-19T11:26:32.978046+0000","last_fullsized":"2024-09-19T11:26:32.978046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.977891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:36.076149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.063998+0000","last_change":"2024-09-19T11:26:43.063925+0000","last_active":"2024-09-19T11:26:43.063998+0000","last_peered":"2024-09-19T11:26:43.063998+0000","last_clean":"2024-09-19T11:26:43.063998+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:43.063998+0000","last_undegraded":"2024-09-19T11:26:43.063998+0000","last_fullsized":"2024-09-19T11:26:43.063998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.063836+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:23.821074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.990936+0000","last_change":"2024-09-19T11:26:31.990845+0000","last_active":"2024-09-19T11:26:31.990936+0000","last_peered":"2024-09-19T11:26:31.990936+0000","last_clean":"2024-09-19T11:26:31.990936+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:26:31.990936+0000","last_undegraded":"2024-09-19T11:26:31.990936+0000","last_fullsized":"2024-09-19T11:26:31.990936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.990752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:33.522127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.044177+0000","last_change":"2024-09-19T11:26:50.044177+0000","last_active":"2024-09-19T11:26:50.044177+0000","last_peered":"2024-09-19T11:26:50.044177+0000","last_clean":"2024-09-19T11:26:50.044177+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:26:50.044177+0000","last_undegraded":"2024-09-19T11:26:50.044177+0000","last_fullsized":"2024-09-19T11:26:50.044177+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.044090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:50.741427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.359821+0000","last_change":"2024-09-19T11:26:43.359747+0000","last_active":"2024-09-19T11:26:43.359821+0000","last_peered":"2024-09-19T11:26:43.359821+0000","last_clean":"2024-09-19T11:26:43.359821+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:26:43.359821+0000","last_undegraded":"2024-09-19T11:26:43.359821+0000","last_fullsized":"2024-09-19T11:26:43.359821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.359650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.024659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.528597+0000","last_change":"2024-09-19T11:27:03.528597+0000","last_active":"2024-09-19T11:27:03.528597+0000","last_peered":"2024-09-19T11:27:03.528597+0000","last_clean":"2024-09-19T11:27:03.528597+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:27:03.528597+0000","last_undegraded":"2024-09-19T11:27:03.528597+0000","last_fullsized":"2024-09-19T11:27:03.528597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:27:03.528502+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:27:03.528502+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.528502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:34.145386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.017153+0000","last_change":"2024-09-19T11:26:33.017041+0000","last_active":"2024-09-19T11:26:33.017153+0000","last_peered":"2024-09-19T11:26:33.017153+0000","last_clean":"2024-09-19T11:26:33.017153+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:26:33.017153+0000","last_undegraded":"2024-09-19T11:26:33.017153+0000","last_fullsized":"2024-09-19T11:26:33.017153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.016950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:06.549550+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.883583+0000","last_change":"2024-09-19T11:26:47.883512+0000","last_active":"2024-09-19T11:26:47.883583+0000","last_peered":"2024-09-19T11:26:47.883583+0000","last_clean":"2024-09-19T11:26:47.883583+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:26:47.883583+0000","last_undegraded":"2024-09-19T11:26:47.883583+0000","last_fullsized":"2024-09-19T11:26:47.883583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.883414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:39.095013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.853715+0000","last_change":"2024-09-19T11:26:31.853602+0000","last_active":"2024-09-19T11:26:31.853715+0000","last_peered":"2024-09-19T11:26:31.853715+0000","last_clean":"2024-09-19T11:26:31.853715+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:26:31.853715+0000","last_undegraded":"2024-09-19T11:26:31.853715+0000","last_fullsized":"2024-09-19T11:26:31.853715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.853479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:16:07.826853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.073256+0000","last_change":"2024-09-19T11:26:49.073137+0000","last_active":"2024-09-19T11:26:49.073256+0000","last_peered":"2024-09-19T11:26:49.073256+0000","last_clean":"2024-09-19T11:26:49.073256+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:26:49.073256+0000","last_undegraded":"2024-09-19T11:26:49.073256+0000","last_fullsized":"2024-09-19T11:26:49.073256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.073043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:26.833881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.886986+0000","last_change":"2024-09-19T11:26:32.886892+0000","last_active":"2024-09-19T11:26:32.886986+0000","last_peered":"2024-09-19T11:26:32.886986+0000","last_clean":"2024-09-19T11:26:32.886986+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:26:32.886986+0000","last_undegraded":"2024-09-19T11:26:32.886986+0000","last_fullsized":"2024-09-19T11:26:32.886986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.886787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:19.114834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.038331+0000","last_change":"2024-09-19T11:26:48.038260+0000","last_active":"2024-09-19T11:26:48.038331+0000","last_peered":"2024-09-19T11:26:48.038331+0000","last_clean":"2024-09-19T11:26:48.038331+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:26:48.038331+0000","last_undegraded":"2024-09-19T11:26:48.038331+0000","last_fullsized":"2024-09-19T11:26:48.038331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.038149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:16.505179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.342771+0000","last_change":"2024-09-19T11:26:44.342671+0000","last_active":"2024-09-19T11:26:44.342771+0000","last_peered":"2024-09-19T11:26:44.342771+0000","last_clean":"2024-09-19T11:26:44.342771+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:26:44.342771+0000","last_undegraded":"2024-09-19T11:26:44.342771+0000","last_fullsized":"2024-09-19T11:26:44.342771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.342574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:38.309444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.082768+0000","last_change":"2024-09-19T11:26:47.082623+0000","last_active":"2024-09-19T11:26:47.082768+0000","last_peered":"2024-09-19T11:26:47.082768+0000","last_clean":"2024-09-19T11:26:47.082768+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:47.082768+0000","last_undegraded":"2024-09-19T11:26:47.082768+0000","last_fullsized":"2024-09-19T11:26:47.082768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.082498+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:22.862275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.395348+0000","last_change":"2024-09-19T11:26:46.395278+0000","last_active":"2024-09-19T11:26:46.395348+0000","last_peered":"2024-09-19T11:26:46.395348+0000","last_clean":"2024-09-19T11:26:46.395348+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:26:46.395348+0000","last_undegraded":"2024-09-19T11:26:46.395348+0000","last_fullsized":"2024-09-19T11:26:46.395348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.395181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:36.015668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.516399+0000","last_change":"2024-09-19T11:27:02.516337+0000","last_active":"2024-09-19T11:27:02.516399+0000","last_peered":"2024-09-19T11:27:02.516399+0000","last_clean":"2024-09-19T11:27:02.516399+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:27:02.516399+0000","last_undegraded":"2024-09-19T11:27:02.516399+0000","last_fullsized":"2024-09-19T11:27:02.516399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:02.516249+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:02.516249+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.516249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:11.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.979209+0000","last_change":"2024-09-19T11:26:33.979119+0000","last_active":"2024-09-19T11:26:33.979209+0000","last_peered":"2024-09-19T11:26:33.979209+0000","last_clean":"2024-09-19T11:26:33.979209+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:26:33.979209+0000","last_undegraded":"2024-09-19T11:26:33.979209+0000","last_fullsized":"2024-09-19T11:26:33.979209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.979000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:24.446915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":87,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.075697+0000","last_change":"2024-09-19T11:26:46.075580+0000","last_active":"2024-09-19T11:26:46.075697+0000","last_peered":"2024-09-19T11:26:46.075697+0000","last_clean":"2024-09-19T11:26:46.075697+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:46.075697+0000","last_undegraded":"2024-09-19T11:26:46.075697+0000","last_fullsized":"2024-09-19T11:26:46.075697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'26","last_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_deep_scrub":"33'26","last_deep_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.075475+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:51.242922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.388148+0000","last_change":"2024-09-19T11:26:45.388148+0000","last_active":"2024-09-19T11:26:45.388148+0000","last_peered":"2024-09-19T11:26:45.388148+0000","last_clean":"2024-09-19T11:26:45.388148+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:26:45.388148+0000","last_undegraded":"2024-09-19T11:26:45.388148+0000","last_fullsized":"2024-09-19T11:26:45.388148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.388045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:47.891066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.428183+0000","last_change":"2024-09-19T11:26:47.428183+0000","last_active":"2024-09-19T11:26:47.428183+0000","last_peered":"2024-09-19T11:26:47.428183+0000","last_clean":"2024-09-19T11:26:47.428183+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:26:47.428183+0000","last_undegraded":"2024-09-19T11:26:47.428183+0000","last_fullsized":"2024-09-19T11:26:47.428183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.428079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:59:02.198551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.933041+0000","last_change":"2024-09-19T11:26:48.933041+0000","last_active":"2024-09-19T11:26:48.933041+0000","last_peered":"2024-09-19T11:26:48.933041+0000","last_clean":"2024-09-19T11:26:48.933041+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:48.933041+0000","last_undegraded":"2024-09-19T11:26:48.933041+0000","last_fullsized":"2024-09-19T11:26:48.933041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.932953+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:16.178277+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.990707+0000","last_change":"2024-09-19T11:26:33.990707+0000","last_active":"2024-09-19T11:26:33.990707+0000","last_peered":"2024-09-19T11:26:33.990707+0000","last_clean":"2024-09-19T11:26:33.990707+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:26:33.990707+0000","last_undegraded":"2024-09-19T11:26:33.990707+0000","last_fullsized":"2024-09-19T11:26:33.990707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.990617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:35.040214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.051717+0000","last_change":"2024-09-19T11:26:54.051717+0000","last_active":"2024-09-19T11:26:54.051717+0000","last_peered":"2024-09-19T11:26:54.051717+0000","last_clean":"2024-09-19T11:26:54.051717+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:26:54.051717+0000","last_undegraded":"2024-09-19T11:26:54.051717+0000","last_fullsized":"2024-09-19T11:26:54.051717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.051625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:27.912591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2288,"ondisk_log_size":2288,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1557,"ondisk_log_size":1557,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":479,"ondisk_log_size":479,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673022,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":479232,"data_stored":150572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673022,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":737565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705727,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":739071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705726,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":148560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:27:45.324 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:45.325 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:45.325 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:45.326 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:45.326 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:45.326 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-09-19T11:23:12.244923+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=23, tm_sec=12, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=11, tm_min=26, tm_sec=20, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T11:27:45.326 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T11:28:05.328 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T11:28:05.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.441+0000 7fb31934b640 1 Processor -- start 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.441+0000 7fb31934b640 1 -- start start 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.441+0000 7fb31934b640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb31412c0b0 0x7fb31412c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.441+0000 7fb31934b640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb31409a290 con 0x7fb31412c9d0 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.441+0000 7fb31934b640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb314071860 con 0x7fb31412c0b0 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.442+0000 7fb312ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb31412c0b0 0x7fb31412c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.442+0000 7fb3127fc640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb31412c9d0 0x7fb314097090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36846/0 (socket says 172.21.15.22:36846) 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.442+0000 7fb312ffd640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb31412c0b0 0x7fb31412c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39250/0 (socket says 172.21.15.22:39250) 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.442+0000 7fb3127fc640 1 -- 172.21.15.22:0/4163695485 learned_addr learned my addr 172.21.15.22:0/4163695485 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.442+0000 7fb312ffd640 1 -- 172.21.15.22:0/4163695485 >> v1:172.21.15.22:6789/0 conn(0x7fb31412c9d0 legacy=0x7fb314097090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:05.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.442+0000 7fb312ffd640 1 -- 172.21.15.22:0/4163695485 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb314099e20 con 0x7fb31412c0b0 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb312ffd640 1 --2- 172.21.15.22:0/4163695485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb31412c490 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b9f0 tx=0x7fb30402c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed2718f38f2ba41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb311ffb640 1 -- 172.21.15.22:0/4163695485 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb30403a070 con 0x7fb31412c0b0 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb311ffb640 1 -- 172.21.15.22:0/4163695485 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb304031070 con 0x7fb31412c0b0 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb311ffb640 1 -- 172.21.15.22:0/4163695485 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb304032620 con 0x7fb31412c0b0 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb31934b640 1 -- 172.21.15.22:0/4163695485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 msgr2=0x7fb31412c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb31934b640 1 --2- 172.21.15.22:0/4163695485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb31412c490 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b9f0 tx=0x7fb30402c810 comp rx=0 tx=0).stop 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb31934b640 1 -- 172.21.15.22:0/4163695485 shutdown_connections 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb31934b640 1 --2- 172.21.15.22:0/4163695485 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb31412c490 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb31934b640 1 -- 172.21.15.22:0/4163695485 >> 172.21.15.22:0/4163695485 conn(0x7fb31409abc0 msgr2=0x7fb314098490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:28:05.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb31934b640 1 -- 172.21.15.22:0/4163695485 shutdown_connections 2024-09-19T11:28:05.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.443+0000 7fb31934b640 1 -- 172.21.15.22:0/4163695485 wait complete. 2024-09-19T11:28:05.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.444+0000 7fb31934b640 1 Processor -- start 2024-09-19T11:28:05.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb31934b640 1 -- start start 2024-09-19T11:28:05.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb31934b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb314127210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb31934b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb31412ce00 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb312ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb314127210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb312ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb314127210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39260/0 (socket says 172.21.15.22:39260) 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb312ffd640 1 -- 172.21.15.22:0/1517914820 learned_addr learned my addr 172.21.15.22:0/1517914820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb312ffd640 1 -- 172.21.15.22:0/1517914820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb314097030 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.445+0000 7fb312ffd640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb314127210 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b9f0 tx=0x7fb3040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.446+0000 7fb2fb7fe640 1 -- 172.21.15.22:0/1517914820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb304045070 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.446+0000 7fb2fb7fe640 1 -- 172.21.15.22:0/1517914820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3040357a0 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.446+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb314129850 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.446+0000 7fb2fb7fe640 1 -- 172.21.15.22:0/1517914820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb30403a040 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.446+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31408c170 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.447+0000 7fb2fb7fe640 1 -- 172.21.15.22:0/1517914820 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fb304048070 con 0x7fb31412c0b0 2024-09-19T11:28:05.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.447+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb314128b70 con 0x7fb31412c0b0 2024-09-19T11:28:05.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.447+0000 7fb2fb7fe640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb30003e710 0x7fb300040bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:28:05.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.447+0000 7fb2fb7fe640 1 -- 172.21.15.22:0/1517914820 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fb304076740 con 0x7fb31412c0b0 2024-09-19T11:28:05.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.447+0000 7fb3127fc640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb30003e710 0x7fb300040bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:28:05.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.448+0000 7fb3127fc640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb30003e710 0x7fb300040bd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb308007fd0 tx=0x7fb308007aa0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:28:05.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.452+0000 7fb2fb7fe640 1 -- 172.21.15.22:0/1517914820 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fb30403fe20 con 0x7fb31412c0b0 2024-09-19T11:28:05.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.634+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 --> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb31412c490 con 0x7fb30003e710 2024-09-19T11:28:05.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.645+0000 7fb2fb7fe640 1 -- 172.21.15.22:0/1517914820 <== mgr.4102 v2:172.21.15.28:6824/2061368145 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+666169 (secure 0 0 0) 0x7fb31412c490 con 0x7fb30003e710 2024-09-19T11:28:05.654 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-19T11:28:05.659 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-19T11:28:05.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb30003e710 msgr2=0x7fb300040bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:05.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb30003e710 0x7fb300040bd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb308007fd0 tx=0x7fb308007aa0 comp rx=0 tx=0).stop 2024-09-19T11:28:05.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 msgr2=0x7fb314127210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:05.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb314127210 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb30400b9f0 tx=0x7fb3040046c0 comp rx=0 tx=0).stop 2024-09-19T11:28:05.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 shutdown_connections 2024-09-19T11:28:05.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb30003e710 0x7fb300040bd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:28:05.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 --2- 172.21.15.22:0/1517914820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb31412c0b0 0x7fb314127210 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:28:05.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 >> 172.21.15.22:0/1517914820 conn(0x7fb31409abc0 msgr2=0x7fb314097550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:28:05.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 shutdown_connections 2024-09-19T11:28:05.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.650+0000 7fb31934b640 1 -- 172.21.15.22:0/1517914820 wait complete. 2024-09-19T11:28:05.663 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-19T11:28:04.883840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605410,"num_objects":222,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":222,"num_whiteouts":0,"num_read":5189,"num_read_kb":4731,"num_write":2121,"num_write_kb":60587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4781,"ondisk_log_size":4781,"up":594,"acting":594,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":594,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":119296,"kb_used_data":2880,"kb_used_omap":6,"kb_used_meta":116345,"kb_avail":377368064,"statfs":{"total":386547056640,"available":386424897536,"internally_reserved":0,"allocated":2949120,"data_stored":1775768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":119138070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004793"},"pg_stats":[{"pgid":"3.3f","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.932419+0000","last_change":"2024-09-19T11:26:37.932354+0000","last_active":"2024-09-19T11:26:37.932419+0000","last_peered":"2024-09-19T11:26:37.932419+0000","last_clean":"2024-09-19T11:26:37.932419+0000","last_became_active":"2024-09-19T11:23:03.258656+0000","last_became_peered":"2024-09-19T11:23:03.258656+0000","last_unstale":"2024-09-19T11:26:37.932419+0000","last_undegraded":"2024-09-19T11:26:37.932419+0000","last_fullsized":"2024-09-19T11:26:37.932419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.932266+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.932266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:41.727377+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"33'2","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.069528+0000","last_change":"2024-09-19T11:26:55.069528+0000","last_active":"2024-09-19T11:26:55.069528+0000","last_peered":"2024-09-19T11:26:55.069528+0000","last_clean":"2024-09-19T11:26:55.069528+0000","last_became_active":"2024-09-19T11:23:05.244984+0000","last_became_peered":"2024-09-19T11:23:05.244984+0000","last_unstale":"2024-09-19T11:26:55.069528+0000","last_undegraded":"2024-09-19T11:26:55.069528+0000","last_fullsized":"2024-09-19T11:26:55.069528+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:55.069440+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.069440+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:55.494351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.990965+0000","last_change":"2024-09-19T11:26:36.990885+0000","last_active":"2024-09-19T11:26:36.990965+0000","last_peered":"2024-09-19T11:26:36.990965+0000","last_clean":"2024-09-19T11:26:36.990965+0000","last_became_active":"2024-09-19T11:23:03.237147+0000","last_became_peered":"2024-09-19T11:23:03.237147+0000","last_unstale":"2024-09-19T11:26:36.990965+0000","last_undegraded":"2024-09-19T11:26:36.990965+0000","last_fullsized":"2024-09-19T11:26:36.990965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:36.990795+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.990795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:14:53.995221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.480743+0000","last_change":"2024-09-19T11:27:05.480743+0000","last_active":"2024-09-19T11:27:05.480743+0000","last_peered":"2024-09-19T11:27:05.480743+0000","last_clean":"2024-09-19T11:27:05.480743+0000","last_became_active":"2024-09-19T11:23:05.242378+0000","last_became_peered":"2024-09-19T11:23:05.242378+0000","last_unstale":"2024-09-19T11:27:05.480743+0000","last_undegraded":"2024-09-19T11:27:05.480743+0000","last_fullsized":"2024-09-19T11:27:05.480743+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:05.480656+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:05.480656+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.480656+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:16.117082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.390858+0000","last_change":"2024-09-19T11:26:51.390768+0000","last_active":"2024-09-19T11:26:51.390858+0000","last_peered":"2024-09-19T11:26:51.390858+0000","last_clean":"2024-09-19T11:26:51.390858+0000","last_became_active":"2024-09-19T11:23:03.251660+0000","last_became_peered":"2024-09-19T11:23:03.251660+0000","last_unstale":"2024-09-19T11:26:51.390858+0000","last_undegraded":"2024-09-19T11:26:51.390858+0000","last_fullsized":"2024-09-19T11:26:51.390858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:51.390678+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.390678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:06.584463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"33'12","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.945164+0000","last_change":"2024-09-19T11:26:50.945164+0000","last_active":"2024-09-19T11:26:50.945164+0000","last_peered":"2024-09-19T11:26:50.945164+0000","last_clean":"2024-09-19T11:26:50.945164+0000","last_became_active":"2024-09-19T11:23:42.284715+0000","last_became_peered":"2024-09-19T11:23:42.284715+0000","last_unstale":"2024-09-19T11:26:50.945164+0000","last_undegraded":"2024-09-19T11:26:50.945164+0000","last_fullsized":"2024-09-19T11:26:50.945164+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:50.945069+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.945069+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:26:56.726703+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.958237+0000","last_change":"2024-09-19T11:26:36.958139+0000","last_active":"2024-09-19T11:26:36.958237+0000","last_peered":"2024-09-19T11:26:36.958237+0000","last_clean":"2024-09-19T11:26:36.958237+0000","last_became_active":"2024-09-19T11:23:03.257001+0000","last_became_peered":"2024-09-19T11:23:03.257001+0000","last_unstale":"2024-09-19T11:26:36.958237+0000","last_undegraded":"2024-09-19T11:26:36.958237+0000","last_fullsized":"2024-09-19T11:26:36.958237+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:36.958025+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.958025+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:34.544025+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.497840+0000","last_change":"2024-09-19T11:27:07.497840+0000","last_active":"2024-09-19T11:27:07.497840+0000","last_peered":"2024-09-19T11:27:07.497840+0000","last_clean":"2024-09-19T11:27:07.497840+0000","last_became_active":"2024-09-19T11:23:05.232882+0000","last_became_peered":"2024-09-19T11:23:05.232882+0000","last_unstale":"2024-09-19T11:27:07.497840+0000","last_undegraded":"2024-09-19T11:27:07.497840+0000","last_fullsized":"2024-09-19T11:27:07.497840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.497726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.497726+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.497726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:24.485123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"33'2","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.373571+0000","last_change":"2024-09-19T11:26:50.373571+0000","last_active":"2024-09-19T11:26:50.373571+0000","last_peered":"2024-09-19T11:26:50.373571+0000","last_clean":"2024-09-19T11:26:50.373571+0000","last_became_active":"2024-09-19T11:23:03.254126+0000","last_became_peered":"2024-09-19T11:23:03.254126+0000","last_unstale":"2024-09-19T11:26:50.373571+0000","last_undegraded":"2024-09-19T11:26:50.373571+0000","last_fullsized":"2024-09-19T11:26:50.373571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.373464+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.373464+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:22:03.662429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"33'12","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.036262+0000","last_change":"2024-09-19T11:26:56.036130+0000","last_active":"2024-09-19T11:26:56.036262+0000","last_peered":"2024-09-19T11:26:56.036262+0000","last_clean":"2024-09-19T11:26:56.036262+0000","last_became_active":"2024-09-19T11:23:05.246715+0000","last_became_peered":"2024-09-19T11:23:05.246715+0000","last_unstale":"2024-09-19T11:26:56.036262+0000","last_undegraded":"2024-09-19T11:26:56.036262+0000","last_fullsized":"2024-09-19T11:26:56.036262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:56.036029+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.036029+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.756679+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.972916+0000","last_change":"2024-09-19T11:26:35.972860+0000","last_active":"2024-09-19T11:26:35.972916+0000","last_peered":"2024-09-19T11:26:35.972916+0000","last_clean":"2024-09-19T11:26:35.972916+0000","last_became_active":"2024-09-19T11:23:03.256324+0000","last_became_peered":"2024-09-19T11:23:03.256324+0000","last_unstale":"2024-09-19T11:26:35.972916+0000","last_undegraded":"2024-09-19T11:26:35.972916+0000","last_fullsized":"2024-09-19T11:26:35.972916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.972760+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.972760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:17.815936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"33'10","reported_seq":74,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.524392+0000","last_change":"2024-09-19T11:27:08.524312+0000","last_active":"2024-09-19T11:27:08.524392+0000","last_peered":"2024-09-19T11:27:08.524392+0000","last_clean":"2024-09-19T11:27:08.524392+0000","last_became_active":"2024-09-19T11:23:05.253396+0000","last_became_peered":"2024-09-19T11:23:05.253396+0000","last_unstale":"2024-09-19T11:27:08.524392+0000","last_undegraded":"2024-09-19T11:27:08.524392+0000","last_fullsized":"2024-09-19T11:27:08.524392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:27:08.524226+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:27:08.524226+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.524226+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:57.580270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.431257+0000","last_change":"2024-09-19T11:26:48.431257+0000","last_active":"2024-09-19T11:26:48.431257+0000","last_peered":"2024-09-19T11:26:48.431257+0000","last_clean":"2024-09-19T11:26:48.431257+0000","last_became_active":"2024-09-19T11:23:03.239262+0000","last_became_peered":"2024-09-19T11:23:03.239262+0000","last_unstale":"2024-09-19T11:26:48.431257+0000","last_undegraded":"2024-09-19T11:26:48.431257+0000","last_fullsized":"2024-09-19T11:26:48.431257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:48.431171+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.431171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:57.441652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"33'6","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.745217+0000","last_change":"2024-09-19T11:26:49.745135+0000","last_active":"2024-09-19T11:26:49.745217+0000","last_peered":"2024-09-19T11:26:49.745217+0000","last_clean":"2024-09-19T11:26:49.745217+0000","last_became_active":"2024-09-19T11:23:05.249558+0000","last_became_peered":"2024-09-19T11:23:05.249558+0000","last_unstale":"2024-09-19T11:26:49.745217+0000","last_undegraded":"2024-09-19T11:26:49.745217+0000","last_fullsized":"2024-09-19T11:26:49.745217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.745054+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.745054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:51.276429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.989858+0000","last_change":"2024-09-19T11:26:35.989858+0000","last_active":"2024-09-19T11:26:35.989858+0000","last_peered":"2024-09-19T11:26:35.989858+0000","last_clean":"2024-09-19T11:26:35.989858+0000","last_became_active":"2024-09-19T11:23:03.237097+0000","last_became_peered":"2024-09-19T11:23:03.237097+0000","last_unstale":"2024-09-19T11:26:35.989858+0000","last_undegraded":"2024-09-19T11:26:35.989858+0000","last_fullsized":"2024-09-19T11:26:35.989858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:35.989770+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.989770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:45.409598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"33'2","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.783741+0000","last_change":"2024-09-19T11:26:50.783644+0000","last_active":"2024-09-19T11:26:50.783741+0000","last_peered":"2024-09-19T11:26:50.783741+0000","last_clean":"2024-09-19T11:26:50.783741+0000","last_became_active":"2024-09-19T11:23:05.246493+0000","last_became_peered":"2024-09-19T11:23:05.246493+0000","last_unstale":"2024-09-19T11:26:50.783741+0000","last_undegraded":"2024-09-19T11:26:50.783741+0000","last_fullsized":"2024-09-19T11:26:50.783741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:50.783546+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.783546+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:21:56.110032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"31'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.464580+0000","last_change":"2024-09-19T11:27:06.464525+0000","last_active":"2024-09-19T11:27:06.464580+0000","last_peered":"2024-09-19T11:27:06.464580+0000","last_clean":"2024-09-19T11:27:06.464580+0000","last_became_active":"2024-09-19T11:23:05.241544+0000","last_became_peered":"2024-09-19T11:23:05.241544+0000","last_unstale":"2024-09-19T11:27:06.464580+0000","last_undegraded":"2024-09-19T11:27:06.464580+0000","last_fullsized":"2024-09-19T11:27:06.464580+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:06.464436+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:06.464436+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.464436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:12.768140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.946531+0000","last_change":"2024-09-19T11:26:34.946461+0000","last_active":"2024-09-19T11:26:34.946531+0000","last_peered":"2024-09-19T11:26:34.946531+0000","last_clean":"2024-09-19T11:26:34.946531+0000","last_became_active":"2024-09-19T11:23:03.247934+0000","last_became_peered":"2024-09-19T11:23:03.247934+0000","last_unstale":"2024-09-19T11:26:34.946531+0000","last_undegraded":"2024-09-19T11:26:34.946531+0000","last_fullsized":"2024-09-19T11:26:34.946531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.946370+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.946370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:48.027671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"33'12","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.017301+0000","last_change":"2024-09-19T11:26:51.017232+0000","last_active":"2024-09-19T11:26:51.017301+0000","last_peered":"2024-09-19T11:26:51.017301+0000","last_clean":"2024-09-19T11:26:51.017301+0000","last_became_active":"2024-09-19T11:23:05.245934+0000","last_became_peered":"2024-09-19T11:23:05.245934+0000","last_unstale":"2024-09-19T11:26:51.017301+0000","last_undegraded":"2024-09-19T11:26:51.017301+0000","last_fullsized":"2024-09-19T11:26:51.017301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:51.017146+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.017146+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.668961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"31'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.952210+0000","last_change":"2024-09-19T11:26:34.952151+0000","last_active":"2024-09-19T11:26:34.952210+0000","last_peered":"2024-09-19T11:26:34.952210+0000","last_clean":"2024-09-19T11:26:34.952210+0000","last_became_active":"2024-09-19T11:23:03.258832+0000","last_became_peered":"2024-09-19T11:23:03.258832+0000","last_unstale":"2024-09-19T11:26:34.952210+0000","last_undegraded":"2024-09-19T11:26:34.952210+0000","last_fullsized":"2024-09-19T11:26:34.952210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:34.952058+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.952058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:03.872887+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"33'6","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.018325+0000","last_change":"2024-09-19T11:26:52.018264+0000","last_active":"2024-09-19T11:26:52.018325+0000","last_peered":"2024-09-19T11:26:52.018325+0000","last_clean":"2024-09-19T11:26:52.018325+0000","last_became_active":"2024-09-19T11:23:05.248010+0000","last_became_peered":"2024-09-19T11:23:05.248010+0000","last_unstale":"2024-09-19T11:26:52.018325+0000","last_undegraded":"2024-09-19T11:26:52.018325+0000","last_fullsized":"2024-09-19T11:26:52.018325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:52.018180+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.018180+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:31.898878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.916825+0000","last_change":"2024-09-19T11:26:33.916754+0000","last_active":"2024-09-19T11:26:33.916825+0000","last_peered":"2024-09-19T11:26:33.916825+0000","last_clean":"2024-09-19T11:26:33.916825+0000","last_became_active":"2024-09-19T11:23:03.248835+0000","last_became_peered":"2024-09-19T11:23:03.248835+0000","last_unstale":"2024-09-19T11:26:33.916825+0000","last_undegraded":"2024-09-19T11:26:33.916825+0000","last_fullsized":"2024-09-19T11:26:33.916825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.916663+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.916663+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:36:35.684322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"33'6","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.898540+0000","last_change":"2024-09-19T11:26:51.898475+0000","last_active":"2024-09-19T11:26:51.898540+0000","last_peered":"2024-09-19T11:26:51.898540+0000","last_clean":"2024-09-19T11:26:51.898540+0000","last_became_active":"2024-09-19T11:23:05.240221+0000","last_became_peered":"2024-09-19T11:23:05.240221+0000","last_unstale":"2024-09-19T11:26:51.898540+0000","last_undegraded":"2024-09-19T11:26:51.898540+0000","last_fullsized":"2024-09-19T11:26:51.898540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:51.898386+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.898386+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:37.296273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.987507+0000","last_change":"2024-09-19T11:26:37.987445+0000","last_active":"2024-09-19T11:26:37.987507+0000","last_peered":"2024-09-19T11:26:37.987507+0000","last_clean":"2024-09-19T11:26:37.987507+0000","last_became_active":"2024-09-19T11:23:03.245119+0000","last_became_peered":"2024-09-19T11:23:03.245119+0000","last_unstale":"2024-09-19T11:26:37.987507+0000","last_undegraded":"2024-09-19T11:26:37.987507+0000","last_fullsized":"2024-09-19T11:26:37.987507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:37.987357+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.987357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:27.173996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"33'8","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.779409+0000","last_change":"2024-09-19T11:26:48.779288+0000","last_active":"2024-09-19T11:26:48.779409+0000","last_peered":"2024-09-19T11:26:48.779409+0000","last_clean":"2024-09-19T11:26:48.779409+0000","last_became_active":"2024-09-19T11:23:05.245128+0000","last_became_peered":"2024-09-19T11:23:05.245128+0000","last_unstale":"2024-09-19T11:26:48.779409+0000","last_undegraded":"2024-09-19T11:26:48.779409+0000","last_fullsized":"2024-09-19T11:26:48.779409+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:48.779185+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.779185+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:49.970393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":34,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.001183+0000","last_change":"2024-09-19T11:26:35.001114+0000","last_active":"2024-09-19T11:26:35.001183+0000","last_peered":"2024-09-19T11:26:35.001183+0000","last_clean":"2024-09-19T11:26:35.001183+0000","last_became_active":"2024-09-19T11:23:03.243109+0000","last_became_peered":"2024-09-19T11:23:03.243109+0000","last_unstale":"2024-09-19T11:26:35.001183+0000","last_undegraded":"2024-09-19T11:26:35.001183+0000","last_fullsized":"2024-09-19T11:26:35.001183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:35.001027+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.001027+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:09.576343+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"33'6","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.033448+0000","last_change":"2024-09-19T11:26:53.033350+0000","last_active":"2024-09-19T11:26:53.033448+0000","last_peered":"2024-09-19T11:26:53.033448+0000","last_clean":"2024-09-19T11:26:53.033448+0000","last_became_active":"2024-09-19T11:23:05.248084+0000","last_became_peered":"2024-09-19T11:23:05.248084+0000","last_unstale":"2024-09-19T11:26:53.033448+0000","last_undegraded":"2024-09-19T11:26:53.033448+0000","last_fullsized":"2024-09-19T11:26:53.033448+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:53.033252+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.033252+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:11.509923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.393031+0000","last_change":"2024-09-19T11:26:49.392963+0000","last_active":"2024-09-19T11:26:49.393031+0000","last_peered":"2024-09-19T11:26:49.393031+0000","last_clean":"2024-09-19T11:26:49.393031+0000","last_became_active":"2024-09-19T11:23:03.240198+0000","last_became_peered":"2024-09-19T11:23:03.240198+0000","last_unstale":"2024-09-19T11:26:49.393031+0000","last_undegraded":"2024-09-19T11:26:49.393031+0000","last_fullsized":"2024-09-19T11:26:49.393031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:49.392873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.392873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:20.014824+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"33'38","reported_seq":111,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:22.907954+0000","last_change":"2024-09-19T11:27:22.907954+0000","last_active":"2024-09-19T11:27:22.907954+0000","last_peered":"2024-09-19T11:27:22.907954+0000","last_clean":"2024-09-19T11:27:22.907954+0000","last_became_active":"2024-09-19T11:23:13.275234+0000","last_became_peered":"2024-09-19T11:23:13.275234+0000","last_unstale":"2024-09-19T11:27:22.907954+0000","last_undegraded":"2024-09-19T11:27:22.907954+0000","last_fullsized":"2024-09-19T11:27:22.907954+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:22.907870+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:22.907870+0000","last_clean_scrub_stamp":"2024-09-19T11:27:22.907870+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:56.381170+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":175,"num_read_kb":175,"num_write":102,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"33'4","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.766852+0000","last_change":"2024-09-19T11:26:43.766852+0000","last_active":"2024-09-19T11:26:43.766852+0000","last_peered":"2024-09-19T11:26:43.766852+0000","last_clean":"2024-09-19T11:26:43.766852+0000","last_became_active":"2024-09-19T11:23:42.282680+0000","last_became_peered":"2024-09-19T11:23:42.282680+0000","last_unstale":"2024-09-19T11:26:43.766852+0000","last_undegraded":"2024-09-19T11:26:43.766852+0000","last_fullsized":"2024-09-19T11:26:43.766852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:43.766755+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.766755+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:40:14.579342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.365792+0000","last_change":"2024-09-19T11:26:37.365713+0000","last_active":"2024-09-19T11:26:37.365792+0000","last_peered":"2024-09-19T11:26:37.365792+0000","last_clean":"2024-09-19T11:26:37.365792+0000","last_became_active":"2024-09-19T11:23:03.255151+0000","last_became_peered":"2024-09-19T11:23:03.255151+0000","last_unstale":"2024-09-19T11:26:37.365792+0000","last_undegraded":"2024-09-19T11:26:37.365792+0000","last_fullsized":"2024-09-19T11:26:37.365792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:37.365625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.365625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:20.032640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.572567+0000","last_change":"2024-09-19T11:27:15.572512+0000","last_active":"2024-09-19T11:27:15.572567+0000","last_peered":"2024-09-19T11:27:15.572567+0000","last_clean":"2024-09-19T11:27:15.572567+0000","last_became_active":"2024-09-19T11:23:07.244596+0000","last_became_peered":"2024-09-19T11:23:07.244596+0000","last_unstale":"2024-09-19T11:27:15.572567+0000","last_undegraded":"2024-09-19T11:27:15.572567+0000","last_fullsized":"2024-09-19T11:27:15.572567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.572426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.572426+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.572426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:44.280324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"33'38","reported_seq":94,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:25.602109+0000","last_change":"2024-09-19T11:27:25.602045+0000","last_active":"2024-09-19T11:27:25.602109+0000","last_peered":"2024-09-19T11:27:25.602109+0000","last_clean":"2024-09-19T11:27:25.602109+0000","last_became_active":"2024-09-19T11:23:09.255524+0000","last_became_peered":"2024-09-19T11:23:09.255524+0000","last_unstale":"2024-09-19T11:27:25.602109+0000","last_undegraded":"2024-09-19T11:27:25.602109+0000","last_fullsized":"2024-09-19T11:27:25.602109+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:25.601947+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:25.601947+0000","last_clean_scrub_stamp":"2024-09-19T11:27:25.601947+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:07:11.813100+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.903574+0000","last_change":"2024-09-19T11:27:14.903454+0000","last_active":"2024-09-19T11:27:14.903574+0000","last_peered":"2024-09-19T11:27:14.903574+0000","last_clean":"2024-09-19T11:27:14.903574+0000","last_became_active":"2024-09-19T11:23:11.266566+0000","last_became_peered":"2024-09-19T11:23:11.266566+0000","last_unstale":"2024-09-19T11:27:14.903574+0000","last_undegraded":"2024-09-19T11:27:14.903574+0000","last_fullsized":"2024-09-19T11:27:14.903574+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.903359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.903359+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.903359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:25.564676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"33'19","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:23.881144+0000","last_change":"2024-09-19T11:27:23.881144+0000","last_active":"2024-09-19T11:27:23.881144+0000","last_peered":"2024-09-19T11:27:23.881144+0000","last_clean":"2024-09-19T11:27:23.881144+0000","last_became_active":"2024-09-19T11:23:13.273957+0000","last_became_peered":"2024-09-19T11:23:13.273957+0000","last_unstale":"2024-09-19T11:27:23.881144+0000","last_undegraded":"2024-09-19T11:27:23.881144+0000","last_fullsized":"2024-09-19T11:27:23.881144+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'19","last_scrub_stamp":"2024-09-19T11:27:23.881053+0000","last_deep_scrub":"33'19","last_deep_scrub_stamp":"2024-09-19T11:27:23.881053+0000","last_clean_scrub_stamp":"2024-09-19T11:27:23.881053+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:01.072315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":33,"num_write":50,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.402980+0000","last_change":"2024-09-19T11:26:57.402917+0000","last_active":"2024-09-19T11:26:57.402980+0000","last_peered":"2024-09-19T11:26:57.402980+0000","last_clean":"2024-09-19T11:26:57.402980+0000","last_became_active":"2024-09-19T11:23:05.237326+0000","last_became_peered":"2024-09-19T11:23:05.237326+0000","last_unstale":"2024-09-19T11:26:57.402980+0000","last_undegraded":"2024-09-19T11:26:57.402980+0000","last_fullsized":"2024-09-19T11:26:57.402980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:57.402830+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.402830+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:15.259544+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"33'6","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.907862+0000","last_change":"2024-09-19T11:26:30.907786+0000","last_active":"2024-09-19T11:26:30.907862+0000","last_peered":"2024-09-19T11:26:30.907862+0000","last_clean":"2024-09-19T11:26:30.907862+0000","last_became_active":"2024-09-19T11:23:03.256153+0000","last_became_peered":"2024-09-19T11:23:03.256153+0000","last_unstale":"2024-09-19T11:26:30.907862+0000","last_undegraded":"2024-09-19T11:26:30.907862+0000","last_fullsized":"2024-09-19T11:26:30.907862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:30.907688+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.907688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:55:01.389670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":8,"num_write":22,"num_write_kb":627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.767929+0000","last_change":"2024-09-19T11:26:59.767856+0000","last_active":"2024-09-19T11:26:59.767929+0000","last_peered":"2024-09-19T11:26:59.767929+0000","last_clean":"2024-09-19T11:26:59.767929+0000","last_became_active":"2024-09-19T11:23:07.244888+0000","last_became_peered":"2024-09-19T11:23:07.244888+0000","last_unstale":"2024-09-19T11:26:59.767929+0000","last_undegraded":"2024-09-19T11:26:59.767929+0000","last_fullsized":"2024-09-19T11:26:59.767929+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.767773+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.767773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:00:22.982882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"33'10","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.925177+0000","last_change":"2024-09-19T11:27:09.925116+0000","last_active":"2024-09-19T11:27:09.925177+0000","last_peered":"2024-09-19T11:27:09.925177+0000","last_clean":"2024-09-19T11:27:09.925177+0000","last_became_active":"2024-09-19T11:23:09.247844+0000","last_became_peered":"2024-09-19T11:23:09.247844+0000","last_unstale":"2024-09-19T11:27:09.925177+0000","last_undegraded":"2024-09-19T11:27:09.925177+0000","last_fullsized":"2024-09-19T11:27:09.925177+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:27:09.925027+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:27:09.925027+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.925027+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:13:01.267066+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.902039+0000","last_change":"2024-09-19T11:27:13.901960+0000","last_active":"2024-09-19T11:27:13.902039+0000","last_peered":"2024-09-19T11:27:13.902039+0000","last_clean":"2024-09-19T11:27:13.902039+0000","last_became_active":"2024-09-19T11:23:11.263131+0000","last_became_peered":"2024-09-19T11:23:11.263131+0000","last_unstale":"2024-09-19T11:27:13.902039+0000","last_undegraded":"2024-09-19T11:27:13.902039+0000","last_fullsized":"2024-09-19T11:27:13.902039+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:13.901857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:13.901857+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.901857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:59.640755+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"31'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:21.839133+0000","last_change":"2024-09-19T11:27:21.839133+0000","last_active":"2024-09-19T11:27:21.839133+0000","last_peered":"2024-09-19T11:27:21.839133+0000","last_clean":"2024-09-19T11:27:21.839133+0000","last_became_active":"2024-09-19T11:23:13.266550+0000","last_became_peered":"2024-09-19T11:23:13.266550+0000","last_unstale":"2024-09-19T11:27:21.839133+0000","last_undegraded":"2024-09-19T11:27:21.839133+0000","last_fullsized":"2024-09-19T11:27:21.839133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:21.839044+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:21.839044+0000","last_clean_scrub_stamp":"2024-09-19T11:27:21.839044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:28:49.430324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.434656+0000","last_change":"2024-09-19T11:26:59.434595+0000","last_active":"2024-09-19T11:26:59.434656+0000","last_peered":"2024-09-19T11:26:59.434656+0000","last_clean":"2024-09-19T11:26:59.434656+0000","last_became_active":"2024-09-19T11:23:05.236271+0000","last_became_peered":"2024-09-19T11:23:05.236271+0000","last_unstale":"2024-09-19T11:26:59.434656+0000","last_undegraded":"2024-09-19T11:26:59.434656+0000","last_fullsized":"2024-09-19T11:26:59.434656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:59.434505+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.434505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:29.091390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"33'4","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.378497+0000","last_change":"2024-09-19T11:26:35.378428+0000","last_active":"2024-09-19T11:26:35.378497+0000","last_peered":"2024-09-19T11:26:35.378497+0000","last_clean":"2024-09-19T11:26:35.378497+0000","last_became_active":"2024-09-19T11:23:03.254808+0000","last_became_peered":"2024-09-19T11:23:03.254808+0000","last_unstale":"2024-09-19T11:26:35.378497+0000","last_undegraded":"2024-09-19T11:26:35.378497+0000","last_fullsized":"2024-09-19T11:26:35.378497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:35.378338+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.378338+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:26.014662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":6,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.815608+0000","last_change":"2024-09-19T11:26:54.815608+0000","last_active":"2024-09-19T11:26:54.815608+0000","last_peered":"2024-09-19T11:26:54.815608+0000","last_clean":"2024-09-19T11:26:54.815608+0000","last_became_active":"2024-09-19T11:23:42.283320+0000","last_became_peered":"2024-09-19T11:23:42.283320+0000","last_unstale":"2024-09-19T11:26:54.815608+0000","last_undegraded":"2024-09-19T11:26:54.815608+0000","last_fullsized":"2024-09-19T11:26:54.815608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.815526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.815526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:36.935748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"33'15","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.846772+0000","last_change":"2024-09-19T11:27:04.846706+0000","last_active":"2024-09-19T11:27:04.846772+0000","last_peered":"2024-09-19T11:27:04.846772+0000","last_clean":"2024-09-19T11:27:04.846772+0000","last_became_active":"2024-09-19T11:23:09.253192+0000","last_became_peered":"2024-09-19T11:23:09.253192+0000","last_unstale":"2024-09-19T11:27:04.846772+0000","last_undegraded":"2024-09-19T11:27:04.846772+0000","last_fullsized":"2024-09-19T11:27:04.846772+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'15","last_scrub_stamp":"2024-09-19T11:27:04.846607+0000","last_deep_scrub":"33'15","last_deep_scrub_stamp":"2024-09-19T11:27:04.846607+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.846607+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:56.058206+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"33'286","reported_seq":399,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:35.677364+0000","last_change":"2024-09-19T11:27:35.677247+0000","last_active":"2024-09-19T11:27:35.677364+0000","last_peered":"2024-09-19T11:27:35.677364+0000","last_clean":"2024-09-19T11:27:35.677364+0000","last_became_active":"2024-09-19T11:23:11.270670+0000","last_became_peered":"2024-09-19T11:23:11.270670+0000","last_unstale":"2024-09-19T11:27:35.677364+0000","last_undegraded":"2024-09-19T11:27:35.677364+0000","last_fullsized":"2024-09-19T11:27:35.677364+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:35.677118+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:35.677118+0000","last_clean_scrub_stamp":"2024-09-19T11:27:35.677118+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:04.854093+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"33'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.970806+0000","last_change":"2024-09-19T11:27:17.970806+0000","last_active":"2024-09-19T11:27:17.970806+0000","last_peered":"2024-09-19T11:27:17.970806+0000","last_clean":"2024-09-19T11:27:17.970806+0000","last_became_active":"2024-09-19T11:23:13.269835+0000","last_became_peered":"2024-09-19T11:23:13.269835+0000","last_unstale":"2024-09-19T11:27:17.970806+0000","last_undegraded":"2024-09-19T11:27:17.970806+0000","last_fullsized":"2024-09-19T11:27:17.970806+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:27:17.970666+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:27:17.970666+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.970666+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:52.045636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"33'2","reported_seq":45,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.025068+0000","last_change":"2024-09-19T11:26:42.025004+0000","last_active":"2024-09-19T11:26:42.025068+0000","last_peered":"2024-09-19T11:26:42.025068+0000","last_clean":"2024-09-19T11:26:42.025068+0000","last_became_active":"2024-09-19T11:23:05.248416+0000","last_became_peered":"2024-09-19T11:23:05.248416+0000","last_unstale":"2024-09-19T11:26:42.025068+0000","last_undegraded":"2024-09-19T11:26:42.025068+0000","last_fullsized":"2024-09-19T11:26:42.025068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.024919+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.024919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:31.267851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.871050+0000","last_change":"2024-09-19T11:26:24.870975+0000","last_active":"2024-09-19T11:26:24.871050+0000","last_peered":"2024-09-19T11:26:24.871050+0000","last_clean":"2024-09-19T11:26:24.871050+0000","last_became_active":"2024-09-19T11:23:03.248263+0000","last_became_peered":"2024-09-19T11:23:03.248263+0000","last_unstale":"2024-09-19T11:26:24.871050+0000","last_undegraded":"2024-09-19T11:26:24.871050+0000","last_fullsized":"2024-09-19T11:26:24.871050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.870884+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.870884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:14.567909+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.529065+0000","last_change":"2024-09-19T11:27:17.528989+0000","last_active":"2024-09-19T11:27:17.529065+0000","last_peered":"2024-09-19T11:27:17.529065+0000","last_clean":"2024-09-19T11:27:17.529065+0000","last_became_active":"2024-09-19T11:23:07.247679+0000","last_became_peered":"2024-09-19T11:23:07.247679+0000","last_unstale":"2024-09-19T11:27:17.529065+0000","last_undegraded":"2024-09-19T11:27:17.529065+0000","last_fullsized":"2024-09-19T11:27:17.529065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:17.528899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:17.528899+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.528899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:52:06.431179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"33'38","reported_seq":94,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.841652+0000","last_change":"2024-09-19T11:27:05.841593+0000","last_active":"2024-09-19T11:27:05.841652+0000","last_peered":"2024-09-19T11:27:05.841652+0000","last_clean":"2024-09-19T11:27:05.841652+0000","last_became_active":"2024-09-19T11:23:09.260565+0000","last_became_peered":"2024-09-19T11:23:09.260565+0000","last_unstale":"2024-09-19T11:27:05.841652+0000","last_undegraded":"2024-09-19T11:27:05.841652+0000","last_fullsized":"2024-09-19T11:27:05.841652+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:05.841496+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:05.841496+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.841496+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:33.086820+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"33'286","reported_seq":415,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.828685+0000","last_change":"2024-09-19T11:27:13.828626+0000","last_active":"2024-09-19T11:27:13.828685+0000","last_peered":"2024-09-19T11:27:13.828685+0000","last_clean":"2024-09-19T11:27:13.828685+0000","last_became_active":"2024-09-19T11:23:11.260217+0000","last_became_peered":"2024-09-19T11:23:11.260217+0000","last_unstale":"2024-09-19T11:27:13.828685+0000","last_undegraded":"2024-09-19T11:27:13.828685+0000","last_fullsized":"2024-09-19T11:27:13.828685+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:13.828521+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:13.828521+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.828521+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:34.119736+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"33'4","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.381390+0000","last_change":"2024-09-19T11:26:55.381320+0000","last_active":"2024-09-19T11:26:55.381390+0000","last_peered":"2024-09-19T11:26:55.381390+0000","last_clean":"2024-09-19T11:26:55.381390+0000","last_became_active":"2024-09-19T11:23:05.253023+0000","last_became_peered":"2024-09-19T11:23:05.253023+0000","last_unstale":"2024-09-19T11:26:55.381390+0000","last_undegraded":"2024-09-19T11:26:55.381390+0000","last_fullsized":"2024-09-19T11:26:55.381390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:55.381228+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.381228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:24:06.217118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.956228+0000","last_change":"2024-09-19T11:26:27.956170+0000","last_active":"2024-09-19T11:26:27.956228+0000","last_peered":"2024-09-19T11:26:27.956228+0000","last_clean":"2024-09-19T11:26:27.956228+0000","last_became_active":"2024-09-19T11:23:03.256040+0000","last_became_peered":"2024-09-19T11:23:03.256040+0000","last_unstale":"2024-09-19T11:26:27.956228+0000","last_undegraded":"2024-09-19T11:26:27.956228+0000","last_fullsized":"2024-09-19T11:26:27.956228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.956081+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.956081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:59.956862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.969564+0000","last_change":"2024-09-19T11:26:57.969498+0000","last_active":"2024-09-19T11:26:57.969564+0000","last_peered":"2024-09-19T11:26:57.969564+0000","last_clean":"2024-09-19T11:26:57.969564+0000","last_became_active":"2024-09-19T11:23:07.244049+0000","last_became_peered":"2024-09-19T11:23:07.244049+0000","last_unstale":"2024-09-19T11:26:57.969564+0000","last_undegraded":"2024-09-19T11:26:57.969564+0000","last_fullsized":"2024-09-19T11:26:57.969564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.969412+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.969412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.712881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"33'52","reported_seq":113,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.849298+0000","last_change":"2024-09-19T11:27:01.849236+0000","last_active":"2024-09-19T11:27:01.849298+0000","last_peered":"2024-09-19T11:27:01.849298+0000","last_clean":"2024-09-19T11:27:01.849298+0000","last_became_active":"2024-09-19T11:23:09.248680+0000","last_became_peered":"2024-09-19T11:23:09.248680+0000","last_unstale":"2024-09-19T11:27:01.849298+0000","last_undegraded":"2024-09-19T11:27:01.849298+0000","last_fullsized":"2024-09-19T11:27:01.849298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'52","last_scrub_stamp":"2024-09-19T11:27:01.849143+0000","last_deep_scrub":"33'52","last_deep_scrub_stamp":"2024-09-19T11:27:01.849143+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.849143+0000","objects_scrubbed":9,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:24.838105+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.808259+0000","last_change":"2024-09-19T11:27:12.808200+0000","last_active":"2024-09-19T11:27:12.808259+0000","last_peered":"2024-09-19T11:27:12.808259+0000","last_clean":"2024-09-19T11:27:12.808259+0000","last_became_active":"2024-09-19T11:23:11.257996+0000","last_became_peered":"2024-09-19T11:23:11.257996+0000","last_unstale":"2024-09-19T11:27:12.808259+0000","last_undegraded":"2024-09-19T11:27:12.808259+0000","last_fullsized":"2024-09-19T11:27:12.808259+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:12.808111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:12.808111+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.808111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:53.887293+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"33'3","reported_seq":32,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.942123+0000","last_change":"2024-09-19T11:27:14.942063+0000","last_active":"2024-09-19T11:27:14.942123+0000","last_peered":"2024-09-19T11:27:14.942123+0000","last_clean":"2024-09-19T11:27:14.942123+0000","last_became_active":"2024-09-19T11:23:13.274312+0000","last_became_peered":"2024-09-19T11:23:13.274312+0000","last_unstale":"2024-09-19T11:27:14.942123+0000","last_undegraded":"2024-09-19T11:27:14.942123+0000","last_fullsized":"2024-09-19T11:27:14.942123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:14.941960+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:14.941960+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.941960+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:20:48.525210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.014137+0000","last_change":"2024-09-19T11:26:39.014072+0000","last_active":"2024-09-19T11:26:39.014137+0000","last_peered":"2024-09-19T11:26:39.014137+0000","last_clean":"2024-09-19T11:26:39.014137+0000","last_became_active":"2024-09-19T11:23:05.245055+0000","last_became_peered":"2024-09-19T11:23:05.245055+0000","last_unstale":"2024-09-19T11:26:39.014137+0000","last_undegraded":"2024-09-19T11:26:39.014137+0000","last_fullsized":"2024-09-19T11:26:39.014137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:39.013977+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.013977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:33.627023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.962177+0000","last_change":"2024-09-19T11:26:26.962098+0000","last_active":"2024-09-19T11:26:26.962177+0000","last_peered":"2024-09-19T11:26:26.962177+0000","last_clean":"2024-09-19T11:26:26.962177+0000","last_became_active":"2024-09-19T11:23:03.256938+0000","last_became_peered":"2024-09-19T11:23:03.256938+0000","last_unstale":"2024-09-19T11:26:26.962177+0000","last_undegraded":"2024-09-19T11:26:26.962177+0000","last_fullsized":"2024-09-19T11:26:26.962177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.962016+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.962016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:19.643957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.852350+0000","last_change":"2024-09-19T11:26:53.852269+0000","last_active":"2024-09-19T11:26:53.852350+0000","last_peered":"2024-09-19T11:26:53.852350+0000","last_clean":"2024-09-19T11:26:53.852350+0000","last_became_active":"2024-09-19T11:23:07.246934+0000","last_became_peered":"2024-09-19T11:23:07.246934+0000","last_unstale":"2024-09-19T11:26:53.852350+0000","last_undegraded":"2024-09-19T11:26:53.852350+0000","last_fullsized":"2024-09-19T11:26:53.852350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:53.852186+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.852186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:52.995416+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"33'37","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:28.621224+0000","last_change":"2024-09-19T11:27:28.621154+0000","last_active":"2024-09-19T11:27:28.621224+0000","last_peered":"2024-09-19T11:27:28.621224+0000","last_clean":"2024-09-19T11:27:28.621224+0000","last_became_active":"2024-09-19T11:23:09.262202+0000","last_became_peered":"2024-09-19T11:23:09.262202+0000","last_unstale":"2024-09-19T11:27:28.621224+0000","last_undegraded":"2024-09-19T11:27:28.621224+0000","last_fullsized":"2024-09-19T11:27:28.621224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'37","last_scrub_stamp":"2024-09-19T11:27:28.621059+0000","last_deep_scrub":"33'37","last_deep_scrub_stamp":"2024-09-19T11:27:28.621059+0000","last_clean_scrub_stamp":"2024-09-19T11:27:28.621059+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:17.237043+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.801081+0000","last_change":"2024-09-19T11:27:11.801023+0000","last_active":"2024-09-19T11:27:11.801081+0000","last_peered":"2024-09-19T11:27:11.801081+0000","last_clean":"2024-09-19T11:27:11.801081+0000","last_became_active":"2024-09-19T11:23:11.265264+0000","last_became_peered":"2024-09-19T11:23:11.265264+0000","last_unstale":"2024-09-19T11:27:11.801081+0000","last_undegraded":"2024-09-19T11:27:11.801081+0000","last_fullsized":"2024-09-19T11:27:11.801081+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.800931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.800931+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.800931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:52:07.495727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"33'6","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.973097+0000","last_change":"2024-09-19T11:27:17.973033+0000","last_active":"2024-09-19T11:27:17.973097+0000","last_peered":"2024-09-19T11:27:17.973097+0000","last_clean":"2024-09-19T11:27:17.973097+0000","last_became_active":"2024-09-19T11:23:13.268458+0000","last_became_peered":"2024-09-19T11:23:13.268458+0000","last_unstale":"2024-09-19T11:27:17.973097+0000","last_undegraded":"2024-09-19T11:27:17.973097+0000","last_fullsized":"2024-09-19T11:27:17.973097+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:27:17.972934+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:27:17.972934+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.972934+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:31.636218+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.889558+0000","last_change":"2024-09-19T11:26:38.889489+0000","last_active":"2024-09-19T11:26:38.889558+0000","last_peered":"2024-09-19T11:26:38.889558+0000","last_clean":"2024-09-19T11:26:38.889558+0000","last_became_active":"2024-09-19T11:23:05.250469+0000","last_became_peered":"2024-09-19T11:23:05.250469+0000","last_unstale":"2024-09-19T11:26:38.889558+0000","last_undegraded":"2024-09-19T11:26:38.889558+0000","last_fullsized":"2024-09-19T11:26:38.889558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:38.889400+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.889400+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:14.595110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":26,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.412920+0000","last_change":"2024-09-19T11:26:36.412815+0000","last_active":"2024-09-19T11:26:36.412920+0000","last_peered":"2024-09-19T11:26:36.412920+0000","last_clean":"2024-09-19T11:26:36.412920+0000","last_became_active":"2024-09-19T11:23:03.242192+0000","last_became_peered":"2024-09-19T11:23:03.242192+0000","last_unstale":"2024-09-19T11:26:36.412920+0000","last_undegraded":"2024-09-19T11:26:36.412920+0000","last_fullsized":"2024-09-19T11:26:36.412920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:36.412703+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.412703+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:59.726639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":12,"num_write_kb":626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.839182+0000","last_change":"2024-09-19T11:26:55.839126+0000","last_active":"2024-09-19T11:26:55.839182+0000","last_peered":"2024-09-19T11:26:55.839182+0000","last_clean":"2024-09-19T11:26:55.839182+0000","last_became_active":"2024-09-19T11:23:07.246550+0000","last_became_peered":"2024-09-19T11:23:07.246550+0000","last_unstale":"2024-09-19T11:26:55.839182+0000","last_undegraded":"2024-09-19T11:26:55.839182+0000","last_fullsized":"2024-09-19T11:26:55.839182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839039+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:08.800957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"33'46","reported_seq":105,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:21.581544+0000","last_change":"2024-09-19T11:27:21.581442+0000","last_active":"2024-09-19T11:27:21.581544+0000","last_peered":"2024-09-19T11:27:21.581544+0000","last_clean":"2024-09-19T11:27:21.581544+0000","last_became_active":"2024-09-19T11:23:09.259122+0000","last_became_peered":"2024-09-19T11:23:09.259122+0000","last_unstale":"2024-09-19T11:27:21.581544+0000","last_undegraded":"2024-09-19T11:27:21.581544+0000","last_fullsized":"2024-09-19T11:27:21.581544+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:21.581324+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:21.581324+0000","last_clean_scrub_stamp":"2024-09-19T11:27:21.581324+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:02.933515+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"33'286","reported_seq":362,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.777546+0000","last_change":"2024-09-19T11:27:10.777433+0000","last_active":"2024-09-19T11:27:10.777546+0000","last_peered":"2024-09-19T11:27:10.777546+0000","last_clean":"2024-09-19T11:27:10.777546+0000","last_became_active":"2024-09-19T11:23:11.266688+0000","last_became_peered":"2024-09-19T11:23:11.266688+0000","last_unstale":"2024-09-19T11:27:10.777546+0000","last_undegraded":"2024-09-19T11:27:10.777546+0000","last_fullsized":"2024-09-19T11:27:10.777546+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:10.777329+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:10.777329+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.777329+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:36:10.736169+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"33'8","reported_seq":36,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:17.854886+0000","last_change":"2024-09-19T11:27:17.854808+0000","last_active":"2024-09-19T11:27:17.854886+0000","last_peered":"2024-09-19T11:27:17.854886+0000","last_clean":"2024-09-19T11:27:17.854886+0000","last_became_active":"2024-09-19T11:23:13.269391+0000","last_became_peered":"2024-09-19T11:23:13.269391+0000","last_unstale":"2024-09-19T11:27:17.854886+0000","last_undegraded":"2024-09-19T11:27:17.854886+0000","last_fullsized":"2024-09-19T11:27:17.854886+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:17.854703+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:17.854703+0000","last_clean_scrub_stamp":"2024-09-19T11:27:17.854703+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:13:23.432636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":11,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"33'12","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.810432+0000","last_change":"2024-09-19T11:26:40.810354+0000","last_active":"2024-09-19T11:26:40.810432+0000","last_peered":"2024-09-19T11:26:40.810432+0000","last_clean":"2024-09-19T11:26:40.810432+0000","last_became_active":"2024-09-19T11:23:05.234920+0000","last_became_peered":"2024-09-19T11:23:05.234920+0000","last_unstale":"2024-09-19T11:26:40.810432+0000","last_undegraded":"2024-09-19T11:26:40.810432+0000","last_fullsized":"2024-09-19T11:26:40.810432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:40.810272+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.810272+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:37.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:34.328610+0000","last_change":"2024-09-19T11:26:34.328550+0000","last_active":"2024-09-19T11:26:34.328610+0000","last_peered":"2024-09-19T11:26:34.328610+0000","last_clean":"2024-09-19T11:26:34.328610+0000","last_became_active":"2024-09-19T11:23:03.256943+0000","last_became_peered":"2024-09-19T11:23:03.256943+0000","last_unstale":"2024-09-19T11:26:34.328610+0000","last_undegraded":"2024-09-19T11:26:34.328610+0000","last_fullsized":"2024-09-19T11:26:34.328610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:34.328463+0000","last_clean_scrub_stamp":"2024-09-19T11:26:34.328463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:54.619678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.868804+0000","last_change":"2024-09-19T11:26:54.868748+0000","last_active":"2024-09-19T11:26:54.868804+0000","last_peered":"2024-09-19T11:26:54.868804+0000","last_clean":"2024-09-19T11:26:54.868804+0000","last_became_active":"2024-09-19T11:23:07.246648+0000","last_became_peered":"2024-09-19T11:23:07.246648+0000","last_unstale":"2024-09-19T11:26:54.868804+0000","last_undegraded":"2024-09-19T11:26:54.868804+0000","last_fullsized":"2024-09-19T11:26:54.868804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:54.868649+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.868649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:17.358870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"33'66","reported_seq":140,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:22.556698+0000","last_change":"2024-09-19T11:27:22.556636+0000","last_active":"2024-09-19T11:27:22.556698+0000","last_peered":"2024-09-19T11:27:22.556698+0000","last_clean":"2024-09-19T11:27:22.556698+0000","last_became_active":"2024-09-19T11:23:09.258797+0000","last_became_peered":"2024-09-19T11:23:09.258797+0000","last_unstale":"2024-09-19T11:27:22.556698+0000","last_undegraded":"2024-09-19T11:27:22.556698+0000","last_fullsized":"2024-09-19T11:27:22.556698+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'66","last_scrub_stamp":"2024-09-19T11:27:22.556541+0000","last_deep_scrub":"33'66","last_deep_scrub_stamp":"2024-09-19T11:27:22.556541+0000","last_clean_scrub_stamp":"2024-09-19T11:27:22.556541+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:40.992302+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.744390+0000","last_change":"2024-09-19T11:27:08.744324+0000","last_active":"2024-09-19T11:27:08.744390+0000","last_peered":"2024-09-19T11:27:08.744390+0000","last_clean":"2024-09-19T11:27:08.744390+0000","last_became_active":"2024-09-19T11:23:11.260117+0000","last_became_peered":"2024-09-19T11:23:11.260117+0000","last_unstale":"2024-09-19T11:27:08.744390+0000","last_undegraded":"2024-09-19T11:27:08.744390+0000","last_fullsized":"2024-09-19T11:27:08.744390+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:08.744222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:08.744222+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.744222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:48:52.725769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"33'21","reported_seq":57,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.840237+0000","last_change":"2024-09-19T11:27:18.840174+0000","last_active":"2024-09-19T11:27:18.840237+0000","last_peered":"2024-09-19T11:27:18.840237+0000","last_clean":"2024-09-19T11:27:18.840237+0000","last_became_active":"2024-09-19T11:23:13.273413+0000","last_became_peered":"2024-09-19T11:23:13.273413+0000","last_unstale":"2024-09-19T11:27:18.840237+0000","last_undegraded":"2024-09-19T11:27:18.840237+0000","last_fullsized":"2024-09-19T11:27:18.840237+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'21","last_scrub_stamp":"2024-09-19T11:27:18.840078+0000","last_deep_scrub":"33'21","last_deep_scrub_stamp":"2024-09-19T11:27:18.840078+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.840078+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:42.094194+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"33'17","reported_seq":93,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.048396+0000","last_change":"2024-09-19T11:26:40.048324+0000","last_active":"2024-09-19T11:26:40.048396+0000","last_peered":"2024-09-19T11:26:40.048396+0000","last_clean":"2024-09-19T11:26:40.048396+0000","last_became_active":"2024-09-19T11:23:05.246570+0000","last_became_peered":"2024-09-19T11:23:05.246570+0000","last_unstale":"2024-09-19T11:26:40.048396+0000","last_undegraded":"2024-09-19T11:26:40.048396+0000","last_fullsized":"2024-09-19T11:26:40.048396+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:26:40.048219+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.048219+0000","objects_scrubbed":1,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:04:18.644192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":62,"num_write":19,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.937593+0000","last_change":"2024-09-19T11:26:28.937533+0000","last_active":"2024-09-19T11:26:28.937593+0000","last_peered":"2024-09-19T11:26:28.937593+0000","last_clean":"2024-09-19T11:26:28.937593+0000","last_became_active":"2024-09-19T11:23:03.233952+0000","last_became_peered":"2024-09-19T11:23:03.233952+0000","last_unstale":"2024-09-19T11:26:28.937593+0000","last_undegraded":"2024-09-19T11:26:28.937593+0000","last_fullsized":"2024-09-19T11:26:28.937593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.937444+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.937444+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:51.513741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.828733+0000","last_change":"2024-09-19T11:26:52.828665+0000","last_active":"2024-09-19T11:26:52.828733+0000","last_peered":"2024-09-19T11:26:52.828733+0000","last_clean":"2024-09-19T11:26:52.828733+0000","last_became_active":"2024-09-19T11:23:07.244565+0000","last_became_peered":"2024-09-19T11:23:07.244565+0000","last_unstale":"2024-09-19T11:26:52.828733+0000","last_undegraded":"2024-09-19T11:26:52.828733+0000","last_fullsized":"2024-09-19T11:26:52.828733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:52.828574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.828574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:12.675425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"33'41","reported_seq":98,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.915307+0000","last_change":"2024-09-19T11:27:07.915209+0000","last_active":"2024-09-19T11:27:07.915307+0000","last_peered":"2024-09-19T11:27:07.915307+0000","last_clean":"2024-09-19T11:27:07.915307+0000","last_became_active":"2024-09-19T11:23:09.247846+0000","last_became_peered":"2024-09-19T11:23:09.247846+0000","last_unstale":"2024-09-19T11:27:07.915307+0000","last_undegraded":"2024-09-19T11:27:07.915307+0000","last_fullsized":"2024-09-19T11:27:07.915307+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'41","last_scrub_stamp":"2024-09-19T11:27:07.915099+0000","last_deep_scrub":"33'41","last_deep_scrub_stamp":"2024-09-19T11:27:07.915099+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.915099+0000","objects_scrubbed":5,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:28.937882+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:33.712113+0000","last_change":"2024-09-19T11:27:33.712035+0000","last_active":"2024-09-19T11:27:33.712113+0000","last_peered":"2024-09-19T11:27:33.712113+0000","last_clean":"2024-09-19T11:27:33.712113+0000","last_became_active":"2024-09-19T11:23:11.270343+0000","last_became_peered":"2024-09-19T11:23:11.270343+0000","last_unstale":"2024-09-19T11:27:33.712113+0000","last_undegraded":"2024-09-19T11:27:33.712113+0000","last_fullsized":"2024-09-19T11:27:33.712113+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:33.711945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:33.711945+0000","last_clean_scrub_stamp":"2024-09-19T11:27:33.711945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:16.723573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"33'4","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:38.649606+0000","last_change":"2024-09-19T11:27:38.649606+0000","last_active":"2024-09-19T11:27:38.649606+0000","last_peered":"2024-09-19T11:27:38.649606+0000","last_clean":"2024-09-19T11:27:38.649606+0000","last_became_active":"2024-09-19T11:23:13.272247+0000","last_became_peered":"2024-09-19T11:23:13.272247+0000","last_unstale":"2024-09-19T11:27:38.649606+0000","last_undegraded":"2024-09-19T11:27:38.649606+0000","last_fullsized":"2024-09-19T11:27:38.649606+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:38.649512+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:38.649512+0000","last_clean_scrub_stamp":"2024-09-19T11:27:38.649512+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:50:52.845888+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"33'8","reported_seq":71,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.393784+0000","last_change":"2024-09-19T11:26:56.393655+0000","last_active":"2024-09-19T11:26:56.393784+0000","last_peered":"2024-09-19T11:26:56.393784+0000","last_clean":"2024-09-19T11:26:56.393784+0000","last_became_active":"2024-09-19T11:23:05.253470+0000","last_became_peered":"2024-09-19T11:23:05.253470+0000","last_unstale":"2024-09-19T11:26:56.393784+0000","last_undegraded":"2024-09-19T11:26:56.393784+0000","last_fullsized":"2024-09-19T11:26:56.393784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:56.393557+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.393557+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:58:06.327682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":42,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.308945+0000","last_change":"2024-09-19T11:26:33.308889+0000","last_active":"2024-09-19T11:26:33.308945+0000","last_peered":"2024-09-19T11:26:33.308945+0000","last_clean":"2024-09-19T11:26:33.308945+0000","last_became_active":"2024-09-19T11:23:03.241366+0000","last_became_peered":"2024-09-19T11:23:03.241366+0000","last_unstale":"2024-09-19T11:26:33.308945+0000","last_undegraded":"2024-09-19T11:26:33.308945+0000","last_fullsized":"2024-09-19T11:26:33.308945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.308801+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.308801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:45:19.525046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.829879+0000","last_change":"2024-09-19T11:26:57.829806+0000","last_active":"2024-09-19T11:26:57.829879+0000","last_peered":"2024-09-19T11:26:57.829879+0000","last_clean":"2024-09-19T11:26:57.829879+0000","last_became_active":"2024-09-19T11:23:07.246681+0000","last_became_peered":"2024-09-19T11:23:07.246681+0000","last_unstale":"2024-09-19T11:26:57.829879+0000","last_undegraded":"2024-09-19T11:26:57.829879+0000","last_fullsized":"2024-09-19T11:26:57.829879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.829722+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.829722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:47.614275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"33'46","reported_seq":105,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.818978+0000","last_change":"2024-09-19T11:27:02.818874+0000","last_active":"2024-09-19T11:27:02.818978+0000","last_peered":"2024-09-19T11:27:02.818978+0000","last_clean":"2024-09-19T11:27:02.818978+0000","last_became_active":"2024-09-19T11:23:09.246211+0000","last_became_peered":"2024-09-19T11:23:09.246211+0000","last_unstale":"2024-09-19T11:27:02.818978+0000","last_undegraded":"2024-09-19T11:27:02.818978+0000","last_fullsized":"2024-09-19T11:27:02.818978+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:02.818754+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:02.818754+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.818754+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:54.439165+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"33'286","reported_seq":417,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.815155+0000","last_change":"2024-09-19T11:27:06.815074+0000","last_active":"2024-09-19T11:27:06.815155+0000","last_peered":"2024-09-19T11:27:06.815155+0000","last_clean":"2024-09-19T11:27:06.815155+0000","last_became_active":"2024-09-19T11:23:11.263350+0000","last_became_peered":"2024-09-19T11:23:11.263350+0000","last_unstale":"2024-09-19T11:27:06.815155+0000","last_undegraded":"2024-09-19T11:27:06.815155+0000","last_fullsized":"2024-09-19T11:27:06.815155+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:06.814991+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:06.814991+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.814991+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:24.746148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"33'16","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:24.811986+0000","last_change":"2024-09-19T11:27:24.811936+0000","last_active":"2024-09-19T11:27:24.811986+0000","last_peered":"2024-09-19T11:27:24.811986+0000","last_clean":"2024-09-19T11:27:24.811986+0000","last_became_active":"2024-09-19T11:23:13.264770+0000","last_became_peered":"2024-09-19T11:23:13.264770+0000","last_unstale":"2024-09-19T11:27:24.811986+0000","last_undegraded":"2024-09-19T11:27:24.811986+0000","last_fullsized":"2024-09-19T11:27:24.811986+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:27:24.811864+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:27:24.811864+0000","last_clean_scrub_stamp":"2024-09-19T11:27:24.811864+0000","objects_scrubbed":2,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:58.503153+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":26,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"33'6","reported_seq":58,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.810939+0000","last_change":"2024-09-19T11:26:41.810833+0000","last_active":"2024-09-19T11:26:41.810939+0000","last_peered":"2024-09-19T11:26:41.810939+0000","last_clean":"2024-09-19T11:26:41.810939+0000","last_became_active":"2024-09-19T11:23:05.249924+0000","last_became_peered":"2024-09-19T11:23:05.249924+0000","last_unstale":"2024-09-19T11:26:41.810939+0000","last_undegraded":"2024-09-19T11:26:41.810939+0000","last_fullsized":"2024-09-19T11:26:41.810939+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:41.810721+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.810721+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:31.441015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.269065+0000","last_change":"2024-09-19T11:26:31.268981+0000","last_active":"2024-09-19T11:26:31.269065+0000","last_peered":"2024-09-19T11:26:31.269065+0000","last_clean":"2024-09-19T11:26:31.269065+0000","last_became_active":"2024-09-19T11:23:03.254929+0000","last_became_peered":"2024-09-19T11:23:03.254929+0000","last_unstale":"2024-09-19T11:26:31.269065+0000","last_undegraded":"2024-09-19T11:26:31.269065+0000","last_fullsized":"2024-09-19T11:26:31.269065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:31.268893+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.268893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:31.227733+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'5","reported_seq":429,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.945365+0000","last_change":"2024-09-19T11:26:58.945300+0000","last_active":"2024-09-19T11:26:58.945365+0000","last_peered":"2024-09-19T11:26:58.945365+0000","last_clean":"2024-09-19T11:26:58.945365+0000","last_became_active":"2024-09-19T11:23:07.241808+0000","last_became_peered":"2024-09-19T11:23:07.241808+0000","last_unstale":"2024-09-19T11:26:58.945365+0000","last_undegraded":"2024-09-19T11:26:58.945365+0000","last_fullsized":"2024-09-19T11:26:58.945365+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:58.945214+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.945214+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:11.607957+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"33'15","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:23.597656+0000","last_change":"2024-09-19T11:27:23.597594+0000","last_active":"2024-09-19T11:27:23.597656+0000","last_peered":"2024-09-19T11:27:23.597656+0000","last_clean":"2024-09-19T11:27:23.597656+0000","last_became_active":"2024-09-19T11:23:09.255503+0000","last_became_peered":"2024-09-19T11:23:09.255503+0000","last_unstale":"2024-09-19T11:27:23.597656+0000","last_undegraded":"2024-09-19T11:27:23.597656+0000","last_fullsized":"2024-09-19T11:27:23.597656+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'15","last_scrub_stamp":"2024-09-19T11:27:23.597495+0000","last_deep_scrub":"33'15","last_deep_scrub_stamp":"2024-09-19T11:27:23.597495+0000","last_clean_scrub_stamp":"2024-09-19T11:27:23.597495+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:50:04.826176+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"33'286","reported_seq":479,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.877425+0000","last_change":"2024-09-19T11:27:12.877361+0000","last_active":"2024-09-19T11:27:12.877425+0000","last_peered":"2024-09-19T11:27:12.877425+0000","last_clean":"2024-09-19T11:27:12.877425+0000","last_became_active":"2024-09-19T11:23:11.255598+0000","last_became_peered":"2024-09-19T11:23:11.255598+0000","last_unstale":"2024-09-19T11:27:12.877425+0000","last_undegraded":"2024-09-19T11:27:12.877425+0000","last_fullsized":"2024-09-19T11:27:12.877425+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:12.877271+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:12.877271+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.877271+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:29.229183+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"33'6","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.902199+0000","last_change":"2024-09-19T11:27:13.902118+0000","last_active":"2024-09-19T11:27:13.902199+0000","last_peered":"2024-09-19T11:27:13.902199+0000","last_clean":"2024-09-19T11:27:13.902199+0000","last_became_active":"2024-09-19T11:23:13.266481+0000","last_became_peered":"2024-09-19T11:23:13.266481+0000","last_unstale":"2024-09-19T11:27:13.902199+0000","last_undegraded":"2024-09-19T11:27:13.902199+0000","last_fullsized":"2024-09-19T11:27:13.902199+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:27:13.902022+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:27:13.902022+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.902022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:10.843373+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"33'23","reported_seq":110,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.028259+0000","last_change":"2024-09-19T11:26:42.028189+0000","last_active":"2024-09-19T11:26:42.028259+0000","last_peered":"2024-09-19T11:26:42.028259+0000","last_clean":"2024-09-19T11:26:42.028259+0000","last_became_active":"2024-09-19T11:23:05.255064+0000","last_became_peered":"2024-09-19T11:23:05.255064+0000","last_unstale":"2024-09-19T11:26:42.028259+0000","last_undegraded":"2024-09-19T11:26:42.028259+0000","last_fullsized":"2024-09-19T11:26:42.028259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'23","last_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_deep_scrub":"33'23","last_deep_scrub_stamp":"2024-09-19T11:26:42.028101+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.028101+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.215685+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":99,"num_read_kb":91,"num_write":27,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.225710+0000","last_change":"2024-09-19T11:26:29.225653+0000","last_active":"2024-09-19T11:26:29.225710+0000","last_peered":"2024-09-19T11:26:29.225710+0000","last_clean":"2024-09-19T11:26:29.225710+0000","last_became_active":"2024-09-19T11:23:03.253831+0000","last_became_peered":"2024-09-19T11:23:03.253831+0000","last_unstale":"2024-09-19T11:26:29.225710+0000","last_undegraded":"2024-09-19T11:26:29.225710+0000","last_fullsized":"2024-09-19T11:26:29.225710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:29.225549+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.225549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:47.310948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"33'7","reported_seq":431,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.565109+0000","last_change":"2024-09-19T11:27:16.565040+0000","last_active":"2024-09-19T11:27:16.565109+0000","last_peered":"2024-09-19T11:27:16.565109+0000","last_clean":"2024-09-19T11:27:16.565109+0000","last_became_active":"2024-09-19T11:23:07.250044+0000","last_became_peered":"2024-09-19T11:23:07.250044+0000","last_unstale":"2024-09-19T11:27:16.565109+0000","last_undegraded":"2024-09-19T11:27:16.565109+0000","last_fullsized":"2024-09-19T11:27:16.565109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:27:16.564928+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:27:16.564928+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.564928+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:52:13.697680+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1429,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":586,"num_read_kb":586,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"33'49","reported_seq":109,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.884490+0000","last_change":"2024-09-19T11:27:06.884406+0000","last_active":"2024-09-19T11:27:06.884490+0000","last_peered":"2024-09-19T11:27:06.884490+0000","last_clean":"2024-09-19T11:27:06.884490+0000","last_became_active":"2024-09-19T11:23:09.247340+0000","last_became_peered":"2024-09-19T11:23:09.247340+0000","last_unstale":"2024-09-19T11:27:06.884490+0000","last_undegraded":"2024-09-19T11:27:06.884490+0000","last_fullsized":"2024-09-19T11:27:06.884490+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'49","last_scrub_stamp":"2024-09-19T11:27:06.884296+0000","last_deep_scrub":"33'49","last_deep_scrub_stamp":"2024-09-19T11:27:06.884296+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.884296+0000","objects_scrubbed":8,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:44:16.451590+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.819681+0000","last_change":"2024-09-19T11:27:08.819603+0000","last_active":"2024-09-19T11:27:08.819681+0000","last_peered":"2024-09-19T11:27:08.819681+0000","last_clean":"2024-09-19T11:27:08.819681+0000","last_became_active":"2024-09-19T11:23:11.265654+0000","last_became_peered":"2024-09-19T11:23:11.265654+0000","last_unstale":"2024-09-19T11:27:08.819681+0000","last_undegraded":"2024-09-19T11:27:08.819681+0000","last_fullsized":"2024-09-19T11:27:08.819681+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:08.819506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:08.819506+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.819506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:54.217393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"33'7","reported_seq":35,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.938535+0000","last_change":"2024-09-19T11:27:16.938455+0000","last_active":"2024-09-19T11:27:16.938535+0000","last_peered":"2024-09-19T11:27:16.938535+0000","last_clean":"2024-09-19T11:27:16.938535+0000","last_became_active":"2024-09-19T11:23:13.263593+0000","last_became_peered":"2024-09-19T11:23:13.263593+0000","last_unstale":"2024-09-19T11:27:16.938535+0000","last_undegraded":"2024-09-19T11:27:16.938535+0000","last_fullsized":"2024-09-19T11:27:16.938535+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:27:16.938366+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:27:16.938366+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.938366+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:08.561118+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"33'8","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.369733+0000","last_change":"2024-09-19T11:26:52.369670+0000","last_active":"2024-09-19T11:26:52.369733+0000","last_peered":"2024-09-19T11:26:52.369733+0000","last_clean":"2024-09-19T11:26:52.369733+0000","last_became_active":"2024-09-19T11:23:05.252882+0000","last_became_peered":"2024-09-19T11:23:05.252882+0000","last_unstale":"2024-09-19T11:26:52.369733+0000","last_undegraded":"2024-09-19T11:26:52.369733+0000","last_fullsized":"2024-09-19T11:26:52.369733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:52.369580+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.369580+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:20:43.275474+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"33'74","reported_seq":157,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.856490+0000","last_change":"2024-09-19T11:26:23.856409+0000","last_active":"2024-09-19T11:26:23.856490+0000","last_peered":"2024-09-19T11:26:23.856490+0000","last_clean":"2024-09-19T11:26:23.856490+0000","last_became_active":"2024-09-19T11:22:46.831334+0000","last_became_peered":"2024-09-19T11:22:46.831334+0000","last_unstale":"2024-09-19T11:26:23.856490+0000","last_undegraded":"2024-09-19T11:26:23.856490+0000","last_fullsized":"2024-09-19T11:26:23.856490+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'74","last_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_deep_scrub":"33'74","last_deep_scrub_stamp":"2024-09-19T11:26:23.856290+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.856290+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:02.254545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"33'2","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.143039+0000","last_change":"2024-09-19T11:26:27.142962+0000","last_active":"2024-09-19T11:26:27.143039+0000","last_peered":"2024-09-19T11:26:27.143039+0000","last_clean":"2024-09-19T11:26:27.143039+0000","last_became_active":"2024-09-19T11:22:53.810818+0000","last_became_peered":"2024-09-19T11:22:53.810818+0000","last_unstale":"2024-09-19T11:26:27.143039+0000","last_undegraded":"2024-09-19T11:26:27.143039+0000","last_fullsized":"2024-09-19T11:26:27.143039+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:27.142873+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.142873+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.914383+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00093365500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.274051+0000","last_change":"2024-09-19T11:26:30.273990+0000","last_active":"2024-09-19T11:26:30.274051+0000","last_peered":"2024-09-19T11:26:30.274051+0000","last_clean":"2024-09-19T11:26:30.274051+0000","last_became_active":"2024-09-19T11:23:03.254697+0000","last_became_peered":"2024-09-19T11:23:03.254697+0000","last_unstale":"2024-09-19T11:26:30.274051+0000","last_undegraded":"2024-09-19T11:26:30.274051+0000","last_fullsized":"2024-09-19T11:26:30.274051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.273902+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.273902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:16:45.300652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.985765+0000","last_change":"2024-09-19T11:26:56.985644+0000","last_active":"2024-09-19T11:26:56.985765+0000","last_peered":"2024-09-19T11:26:56.985765+0000","last_clean":"2024-09-19T11:26:56.985765+0000","last_became_active":"2024-09-19T11:23:07.238492+0000","last_became_peered":"2024-09-19T11:23:07.238492+0000","last_unstale":"2024-09-19T11:26:56.985765+0000","last_undegraded":"2024-09-19T11:26:56.985765+0000","last_fullsized":"2024-09-19T11:26:56.985765+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.985544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.985544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:47.501191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"33'90","reported_seq":181,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.543157+0000","last_change":"2024-09-19T11:27:20.543099+0000","last_active":"2024-09-19T11:27:20.543157+0000","last_peered":"2024-09-19T11:27:20.543157+0000","last_clean":"2024-09-19T11:27:20.543157+0000","last_became_active":"2024-09-19T11:23:09.260548+0000","last_became_peered":"2024-09-19T11:23:09.260548+0000","last_unstale":"2024-09-19T11:27:20.543157+0000","last_undegraded":"2024-09-19T11:27:20.543157+0000","last_fullsized":"2024-09-19T11:27:20.543157+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'90","last_scrub_stamp":"2024-09-19T11:27:20.542999+0000","last_deep_scrub":"33'90","last_deep_scrub_stamp":"2024-09-19T11:27:20.542999+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.542999+0000","objects_scrubbed":5,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:14.545112+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.736754+0000","last_change":"2024-09-19T11:27:07.736634+0000","last_active":"2024-09-19T11:27:07.736754+0000","last_peered":"2024-09-19T11:27:07.736754+0000","last_clean":"2024-09-19T11:27:07.736754+0000","last_became_active":"2024-09-19T11:23:11.265394+0000","last_became_peered":"2024-09-19T11:23:11.265394+0000","last_unstale":"2024-09-19T11:27:07.736754+0000","last_undegraded":"2024-09-19T11:27:07.736754+0000","last_fullsized":"2024-09-19T11:27:07.736754+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.736538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.736538+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.736538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:27.768987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"33'81","reported_seq":172,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.953984+0000","last_change":"2024-09-19T11:27:15.953879+0000","last_active":"2024-09-19T11:27:15.953984+0000","last_peered":"2024-09-19T11:27:15.953984+0000","last_clean":"2024-09-19T11:27:15.953984+0000","last_became_active":"2024-09-19T11:23:13.266566+0000","last_became_peered":"2024-09-19T11:23:13.266566+0000","last_unstale":"2024-09-19T11:27:15.953984+0000","last_undegraded":"2024-09-19T11:27:15.953984+0000","last_fullsized":"2024-09-19T11:27:15.953984+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'81","last_scrub_stamp":"2024-09-19T11:27:15.953765+0000","last_deep_scrub":"33'81","last_deep_scrub_stamp":"2024-09-19T11:27:15.953765+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.953765+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:43.808777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":223,"num_read_kb":221,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"33'6","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.415571+0000","last_change":"2024-09-19T11:26:54.415515+0000","last_active":"2024-09-19T11:26:54.415571+0000","last_peered":"2024-09-19T11:26:54.415571+0000","last_clean":"2024-09-19T11:26:54.415571+0000","last_became_active":"2024-09-19T11:23:05.233332+0000","last_became_peered":"2024-09-19T11:23:05.233332+0000","last_unstale":"2024-09-19T11:26:54.415571+0000","last_undegraded":"2024-09-19T11:26:54.415571+0000","last_fullsized":"2024-09-19T11:26:54.415571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:54.415426+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.415426+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:18:50.714030+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.079877+0000","last_change":"2024-09-19T11:26:24.079800+0000","last_active":"2024-09-19T11:26:24.079877+0000","last_peered":"2024-09-19T11:26:24.079877+0000","last_clean":"2024-09-19T11:26:24.079877+0000","last_became_active":"2024-09-19T11:22:53.812027+0000","last_became_peered":"2024-09-19T11:22:53.812027+0000","last_unstale":"2024-09-19T11:26:24.079877+0000","last_undegraded":"2024-09-19T11:26:24.079877+0000","last_fullsized":"2024-09-19T11:26:24.079877+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.079699+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.079699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:48:18.789176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068199700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.923570+0000","last_change":"2024-09-19T11:26:25.923493+0000","last_active":"2024-09-19T11:26:25.923570+0000","last_peered":"2024-09-19T11:26:25.923570+0000","last_clean":"2024-09-19T11:26:25.923570+0000","last_became_active":"2024-09-19T11:23:03.236305+0000","last_became_peered":"2024-09-19T11:23:03.236305+0000","last_unstale":"2024-09-19T11:26:25.923570+0000","last_undegraded":"2024-09-19T11:26:25.923570+0000","last_fullsized":"2024-09-19T11:26:25.923570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.923407+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.923407+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:39:56.735995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:13.613109+0000","last_change":"2024-09-19T11:27:13.613049+0000","last_active":"2024-09-19T11:27:13.613109+0000","last_peered":"2024-09-19T11:27:13.613109+0000","last_clean":"2024-09-19T11:27:13.613109+0000","last_became_active":"2024-09-19T11:23:07.245798+0000","last_became_peered":"2024-09-19T11:23:07.245798+0000","last_unstale":"2024-09-19T11:27:13.613109+0000","last_undegraded":"2024-09-19T11:27:13.613109+0000","last_fullsized":"2024-09-19T11:27:13.613109+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:13.612955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:13.612955+0000","last_clean_scrub_stamp":"2024-09-19T11:27:13.612955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:02.028356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"33'294","reported_seq":520,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.901143+0000","last_change":"2024-09-19T11:27:05.901072+0000","last_active":"2024-09-19T11:27:05.901143+0000","last_peered":"2024-09-19T11:27:05.901143+0000","last_clean":"2024-09-19T11:27:05.901143+0000","last_became_active":"2024-09-19T11:23:09.256950+0000","last_became_peered":"2024-09-19T11:23:09.256950+0000","last_unstale":"2024-09-19T11:27:05.901143+0000","last_undegraded":"2024-09-19T11:27:05.901143+0000","last_fullsized":"2024-09-19T11:27:05.901143+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'294","last_scrub_stamp":"2024-09-19T11:27:05.900953+0000","last_deep_scrub":"33'294","last_deep_scrub_stamp":"2024-09-19T11:27:05.900953+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.900953+0000","objects_scrubbed":10,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:53.378733+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:30.677490+0000","last_change":"2024-09-19T11:27:30.677490+0000","last_active":"2024-09-19T11:27:30.677490+0000","last_peered":"2024-09-19T11:27:30.677490+0000","last_clean":"2024-09-19T11:27:30.677490+0000","last_became_active":"2024-09-19T11:23:11.270394+0000","last_became_peered":"2024-09-19T11:23:11.270394+0000","last_unstale":"2024-09-19T11:27:30.677490+0000","last_undegraded":"2024-09-19T11:27:30.677490+0000","last_fullsized":"2024-09-19T11:27:30.677490+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:30.677403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:30.677403+0000","last_clean_scrub_stamp":"2024-09-19T11:27:30.677403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:32.369255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"33'5","reported_seq":33,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.945274+0000","last_change":"2024-09-19T11:27:19.945189+0000","last_active":"2024-09-19T11:27:19.945274+0000","last_peered":"2024-09-19T11:27:19.945274+0000","last_clean":"2024-09-19T11:27:19.945274+0000","last_became_active":"2024-09-19T11:23:13.269627+0000","last_became_peered":"2024-09-19T11:23:13.269627+0000","last_unstale":"2024-09-19T11:27:19.945274+0000","last_undegraded":"2024-09-19T11:27:19.945274+0000","last_fullsized":"2024-09-19T11:27:19.945274+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:27:19.945103+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:27:19.945103+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.945103+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:34.266283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.967905+0000","last_change":"2024-09-19T11:26:39.967834+0000","last_active":"2024-09-19T11:26:39.967905+0000","last_peered":"2024-09-19T11:26:39.967905+0000","last_clean":"2024-09-19T11:26:39.967905+0000","last_became_active":"2024-09-19T11:23:05.247992+0000","last_became_peered":"2024-09-19T11:23:05.247992+0000","last_unstale":"2024-09-19T11:26:39.967905+0000","last_undegraded":"2024-09-19T11:26:39.967905+0000","last_fullsized":"2024-09-19T11:26:39.967905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.967748+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.967748+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:48.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.952580+0000","last_change":"2024-09-19T11:26:23.952524+0000","last_active":"2024-09-19T11:26:23.952580+0000","last_peered":"2024-09-19T11:26:23.952580+0000","last_clean":"2024-09-19T11:26:23.952580+0000","last_became_active":"2024-09-19T11:22:53.811822+0000","last_became_peered":"2024-09-19T11:22:53.811822+0000","last_unstale":"2024-09-19T11:26:23.952580+0000","last_undegraded":"2024-09-19T11:26:23.952580+0000","last_fullsized":"2024-09-19T11:26:23.952580+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.952436+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.952436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:33.569870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052487699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.922891+0000","last_change":"2024-09-19T11:26:24.922822+0000","last_active":"2024-09-19T11:26:24.922891+0000","last_peered":"2024-09-19T11:26:24.922891+0000","last_clean":"2024-09-19T11:26:24.922891+0000","last_became_active":"2024-09-19T11:23:03.243146+0000","last_became_peered":"2024-09-19T11:23:03.243146+0000","last_unstale":"2024-09-19T11:26:24.922891+0000","last_undegraded":"2024-09-19T11:26:24.922891+0000","last_fullsized":"2024-09-19T11:26:24.922891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:24.922736+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.922736+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:47.473806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"33'2","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.833301+0000","last_change":"2024-09-19T11:26:53.833236+0000","last_active":"2024-09-19T11:26:53.833301+0000","last_peered":"2024-09-19T11:26:53.833301+0000","last_clean":"2024-09-19T11:26:53.833301+0000","last_became_active":"2024-09-19T11:23:07.244155+0000","last_became_peered":"2024-09-19T11:23:07.244155+0000","last_unstale":"2024-09-19T11:26:53.833301+0000","last_undegraded":"2024-09-19T11:26:53.833301+0000","last_fullsized":"2024-09-19T11:26:53.833301+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:53.833148+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.833148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:15:46.323076+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"33'24","reported_seq":102,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.835137+0000","last_change":"2024-09-19T11:26:59.835062+0000","last_active":"2024-09-19T11:26:59.835137+0000","last_peered":"2024-09-19T11:26:59.835137+0000","last_clean":"2024-09-19T11:26:59.835137+0000","last_became_active":"2024-09-19T11:23:09.249351+0000","last_became_peered":"2024-09-19T11:23:09.249351+0000","last_unstale":"2024-09-19T11:26:59.835137+0000","last_undegraded":"2024-09-19T11:26:59.835137+0000","last_fullsized":"2024-09-19T11:26:59.835137+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'24","last_scrub_stamp":"2024-09-19T11:26:59.834946+0000","last_deep_scrub":"33'24","last_deep_scrub_stamp":"2024-09-19T11:26:59.834946+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.834946+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:40.523084+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:06.753997+0000","last_change":"2024-09-19T11:27:06.753941+0000","last_active":"2024-09-19T11:27:06.753997+0000","last_peered":"2024-09-19T11:27:06.753997+0000","last_clean":"2024-09-19T11:27:06.753997+0000","last_became_active":"2024-09-19T11:23:11.265280+0000","last_became_peered":"2024-09-19T11:23:11.265280+0000","last_unstale":"2024-09-19T11:27:06.753997+0000","last_undegraded":"2024-09-19T11:27:06.753997+0000","last_fullsized":"2024-09-19T11:27:06.753997+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:06.753841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:06.753841+0000","last_clean_scrub_stamp":"2024-09-19T11:27:06.753841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:57.677312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:39.657252+0000","last_change":"2024-09-19T11:27:39.657173+0000","last_active":"2024-09-19T11:27:39.657252+0000","last_peered":"2024-09-19T11:27:39.657252+0000","last_clean":"2024-09-19T11:27:39.657252+0000","last_became_active":"2024-09-19T11:23:13.273047+0000","last_became_peered":"2024-09-19T11:23:13.273047+0000","last_unstale":"2024-09-19T11:27:39.657252+0000","last_undegraded":"2024-09-19T11:27:39.657252+0000","last_fullsized":"2024-09-19T11:27:39.657252+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:39.657088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:39.657088+0000","last_clean_scrub_stamp":"2024-09-19T11:27:39.657088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:33.294178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"33'16","reported_seq":68,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:37.977214+0000","last_change":"2024-09-19T11:26:37.977150+0000","last_active":"2024-09-19T11:26:37.977214+0000","last_peered":"2024-09-19T11:26:37.977214+0000","last_clean":"2024-09-19T11:26:37.977214+0000","last_became_active":"2024-09-19T11:23:05.237439+0000","last_became_peered":"2024-09-19T11:23:05.237439+0000","last_unstale":"2024-09-19T11:26:37.977214+0000","last_undegraded":"2024-09-19T11:26:37.977214+0000","last_fullsized":"2024-09-19T11:26:37.977214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:26:37.977048+0000","last_clean_scrub_stamp":"2024-09-19T11:26:37.977048+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:18:20.097270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"33'3","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.109984+0000","last_change":"2024-09-19T11:26:25.109916+0000","last_active":"2024-09-19T11:26:25.109984+0000","last_peered":"2024-09-19T11:26:25.109984+0000","last_clean":"2024-09-19T11:26:25.109984+0000","last_became_active":"2024-09-19T11:22:53.811927+0000","last_became_peered":"2024-09-19T11:22:53.811927+0000","last_unstale":"2024-09-19T11:26:25.109984+0000","last_undegraded":"2024-09-19T11:26:25.109984+0000","last_fullsized":"2024-09-19T11:26:25.109984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:26:25.109824+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.109824+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:48.020763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011895390000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.938101+0000","last_change":"2024-09-19T11:26:25.938045+0000","last_active":"2024-09-19T11:26:25.938101+0000","last_peered":"2024-09-19T11:26:25.938101+0000","last_clean":"2024-09-19T11:26:25.938101+0000","last_became_active":"2024-09-19T11:23:03.247141+0000","last_became_peered":"2024-09-19T11:23:03.247141+0000","last_unstale":"2024-09-19T11:26:25.938101+0000","last_undegraded":"2024-09-19T11:26:25.938101+0000","last_fullsized":"2024-09-19T11:26:25.938101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:25.937963+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.937963+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:58:57.848583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.801326+0000","last_change":"2024-09-19T11:26:56.801257+0000","last_active":"2024-09-19T11:26:56.801326+0000","last_peered":"2024-09-19T11:26:56.801326+0000","last_clean":"2024-09-19T11:26:56.801326+0000","last_became_active":"2024-09-19T11:23:07.243500+0000","last_became_peered":"2024-09-19T11:23:07.243500+0000","last_unstale":"2024-09-19T11:26:56.801326+0000","last_undegraded":"2024-09-19T11:26:56.801326+0000","last_fullsized":"2024-09-19T11:26:56.801326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.801170+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.801170+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:30:01.667240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"33'20","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.527738+0000","last_change":"2024-09-19T11:27:19.527663+0000","last_active":"2024-09-19T11:27:19.527738+0000","last_peered":"2024-09-19T11:27:19.527738+0000","last_clean":"2024-09-19T11:27:19.527738+0000","last_became_active":"2024-09-19T11:23:09.254141+0000","last_became_peered":"2024-09-19T11:23:09.254141+0000","last_unstale":"2024-09-19T11:27:19.527738+0000","last_undegraded":"2024-09-19T11:27:19.527738+0000","last_fullsized":"2024-09-19T11:27:19.527738+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'20","last_scrub_stamp":"2024-09-19T11:27:19.527564+0000","last_deep_scrub":"33'20","last_deep_scrub_stamp":"2024-09-19T11:27:19.527564+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.527564+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:21.167261+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:31.653248+0000","last_change":"2024-09-19T11:27:31.653190+0000","last_active":"2024-09-19T11:27:31.653248+0000","last_peered":"2024-09-19T11:27:31.653248+0000","last_clean":"2024-09-19T11:27:31.653248+0000","last_became_active":"2024-09-19T11:23:11.270501+0000","last_became_peered":"2024-09-19T11:23:11.270501+0000","last_unstale":"2024-09-19T11:27:31.653248+0000","last_undegraded":"2024-09-19T11:27:31.653248+0000","last_fullsized":"2024-09-19T11:27:31.653248+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:31.653102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:31.653102+0000","last_clean_scrub_stamp":"2024-09-19T11:27:31.653102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:07.186773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"33'4","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.934985+0000","last_change":"2024-09-19T11:27:18.934889+0000","last_active":"2024-09-19T11:27:18.934985+0000","last_peered":"2024-09-19T11:27:18.934985+0000","last_clean":"2024-09-19T11:27:18.934985+0000","last_became_active":"2024-09-19T11:23:13.273516+0000","last_became_peered":"2024-09-19T11:23:13.273516+0000","last_unstale":"2024-09-19T11:27:18.934985+0000","last_undegraded":"2024-09-19T11:27:18.934985+0000","last_fullsized":"2024-09-19T11:27:18.934985+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:18.934732+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:18.934732+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.934732+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:57:23.688618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"33'14","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:35.916131+0000","last_change":"2024-09-19T11:26:35.916064+0000","last_active":"2024-09-19T11:26:35.916131+0000","last_peered":"2024-09-19T11:26:35.916131+0000","last_clean":"2024-09-19T11:26:35.916131+0000","last_became_active":"2024-09-19T11:23:05.246412+0000","last_became_peered":"2024-09-19T11:23:05.246412+0000","last_unstale":"2024-09-19T11:26:35.916131+0000","last_undegraded":"2024-09-19T11:26:35.916131+0000","last_fullsized":"2024-09-19T11:26:35.916131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:35.915972+0000","last_clean_scrub_stamp":"2024-09-19T11:26:35.915972+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:44:33.729135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":39,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:24.933807+0000","last_change":"2024-09-19T11:26:24.933741+0000","last_active":"2024-09-19T11:26:24.933807+0000","last_peered":"2024-09-19T11:26:24.933807+0000","last_clean":"2024-09-19T11:26:24.933807+0000","last_became_active":"2024-09-19T11:23:03.250857+0000","last_became_peered":"2024-09-19T11:23:03.250857+0000","last_unstale":"2024-09-19T11:26:24.933807+0000","last_undegraded":"2024-09-19T11:26:24.933807+0000","last_fullsized":"2024-09-19T11:26:24.933807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:24.933645+0000","last_clean_scrub_stamp":"2024-09-19T11:26:24.933645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:29.620054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.910643+0000","last_change":"2024-09-19T11:26:22.910571+0000","last_active":"2024-09-19T11:26:22.910643+0000","last_peered":"2024-09-19T11:26:22.910643+0000","last_clean":"2024-09-19T11:26:22.910643+0000","last_became_active":"2024-09-19T11:22:53.811613+0000","last_became_peered":"2024-09-19T11:22:53.811613+0000","last_unstale":"2024-09-19T11:26:22.910643+0000","last_undegraded":"2024-09-19T11:26:22.910643+0000","last_fullsized":"2024-09-19T11:26:22.910643+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.910497+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.910497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:20.068073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059092200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"33'2","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:52.866238+0000","last_change":"2024-09-19T11:26:52.866167+0000","last_active":"2024-09-19T11:26:52.866238+0000","last_peered":"2024-09-19T11:26:52.866238+0000","last_clean":"2024-09-19T11:26:52.866238+0000","last_became_active":"2024-09-19T11:23:07.247598+0000","last_became_peered":"2024-09-19T11:23:07.247598+0000","last_unstale":"2024-09-19T11:26:52.866238+0000","last_undegraded":"2024-09-19T11:26:52.866238+0000","last_fullsized":"2024-09-19T11:26:52.866238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:52.866082+0000","last_clean_scrub_stamp":"2024-09-19T11:26:52.866082+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:47.631328+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"33'77","reported_seq":154,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.785719+0000","last_change":"2024-09-19T11:26:58.785636+0000","last_active":"2024-09-19T11:26:58.785719+0000","last_peered":"2024-09-19T11:26:58.785719+0000","last_clean":"2024-09-19T11:26:58.785719+0000","last_became_active":"2024-09-19T11:23:09.257470+0000","last_became_peered":"2024-09-19T11:23:09.257470+0000","last_unstale":"2024-09-19T11:26:58.785719+0000","last_undegraded":"2024-09-19T11:26:58.785719+0000","last_fullsized":"2024-09-19T11:26:58.785719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'77","last_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_deep_scrub":"33'77","last_deep_scrub_stamp":"2024-09-19T11:26:58.785526+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.785526+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:40.190489+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:07.829172+0000","last_change":"2024-09-19T11:27:07.829111+0000","last_active":"2024-09-19T11:27:07.829172+0000","last_peered":"2024-09-19T11:27:07.829172+0000","last_clean":"2024-09-19T11:27:07.829172+0000","last_became_active":"2024-09-19T11:23:11.268327+0000","last_became_peered":"2024-09-19T11:23:11.268327+0000","last_unstale":"2024-09-19T11:27:07.829172+0000","last_undegraded":"2024-09-19T11:27:07.829172+0000","last_fullsized":"2024-09-19T11:27:07.829172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:07.829015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:07.829015+0000","last_clean_scrub_stamp":"2024-09-19T11:27:07.829015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:13.610649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"33'17","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:19.843438+0000","last_change":"2024-09-19T11:27:19.843376+0000","last_active":"2024-09-19T11:27:19.843438+0000","last_peered":"2024-09-19T11:27:19.843438+0000","last_clean":"2024-09-19T11:27:19.843438+0000","last_became_active":"2024-09-19T11:23:13.267271+0000","last_became_peered":"2024-09-19T11:23:13.267271+0000","last_unstale":"2024-09-19T11:27:19.843438+0000","last_undegraded":"2024-09-19T11:27:19.843438+0000","last_fullsized":"2024-09-19T11:27:19.843438+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'17","last_scrub_stamp":"2024-09-19T11:27:19.843296+0000","last_deep_scrub":"33'17","last_deep_scrub_stamp":"2024-09-19T11:27:19.843296+0000","last_clean_scrub_stamp":"2024-09-19T11:27:19.843296+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:43.970345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":86,"num_write":42,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"33'5","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:36.952834+0000","last_change":"2024-09-19T11:26:36.952752+0000","last_active":"2024-09-19T11:26:36.952834+0000","last_peered":"2024-09-19T11:26:36.952834+0000","last_clean":"2024-09-19T11:26:36.952834+0000","last_became_active":"2024-09-19T11:23:05.246035+0000","last_became_peered":"2024-09-19T11:23:05.246035+0000","last_unstale":"2024-09-19T11:26:36.952834+0000","last_undegraded":"2024-09-19T11:26:36.952834+0000","last_fullsized":"2024-09-19T11:26:36.952834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'5","last_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_deep_scrub":"33'5","last_deep_scrub_stamp":"2024-09-19T11:26:36.952660+0000","last_clean_scrub_stamp":"2024-09-19T11:26:36.952660+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:15.496448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.969573+0000","last_change":"2024-09-19T11:26:38.969511+0000","last_active":"2024-09-19T11:26:38.969573+0000","last_peered":"2024-09-19T11:26:38.969573+0000","last_clean":"2024-09-19T11:26:38.969573+0000","last_became_active":"2024-09-19T11:23:03.236292+0000","last_became_peered":"2024-09-19T11:23:03.236292+0000","last_unstale":"2024-09-19T11:26:38.969573+0000","last_undegraded":"2024-09-19T11:26:38.969573+0000","last_fullsized":"2024-09-19T11:26:38.969573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.969423+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.969423+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:49.370950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.293541+0000","last_change":"2024-09-19T11:26:32.293468+0000","last_active":"2024-09-19T11:26:32.293541+0000","last_peered":"2024-09-19T11:26:32.293541+0000","last_clean":"2024-09-19T11:26:32.293541+0000","last_became_active":"2024-09-19T11:22:53.812121+0000","last_became_peered":"2024-09-19T11:22:53.812121+0000","last_unstale":"2024-09-19T11:26:32.293541+0000","last_undegraded":"2024-09-19T11:26:32.293541+0000","last_fullsized":"2024-09-19T11:26:32.293541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.293378+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.293378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:10:48.996006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001444482,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.538549+0000","last_change":"2024-09-19T11:27:10.538492+0000","last_active":"2024-09-19T11:27:10.538549+0000","last_peered":"2024-09-19T11:27:10.538549+0000","last_clean":"2024-09-19T11:27:10.538549+0000","last_became_active":"2024-09-19T11:23:07.242859+0000","last_became_peered":"2024-09-19T11:23:07.242859+0000","last_unstale":"2024-09-19T11:27:10.538549+0000","last_undegraded":"2024-09-19T11:27:10.538549+0000","last_fullsized":"2024-09-19T11:27:10.538549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:10.538407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:10.538407+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.538407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:33.331328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"33'22","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.936170+0000","last_change":"2024-09-19T11:27:04.936063+0000","last_active":"2024-09-19T11:27:04.936170+0000","last_peered":"2024-09-19T11:27:04.936170+0000","last_clean":"2024-09-19T11:27:04.936170+0000","last_became_active":"2024-09-19T11:23:09.260749+0000","last_became_peered":"2024-09-19T11:23:09.260749+0000","last_unstale":"2024-09-19T11:27:04.936170+0000","last_undegraded":"2024-09-19T11:27:04.936170+0000","last_fullsized":"2024-09-19T11:27:04.936170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'22","last_scrub_stamp":"2024-09-19T11:27:04.935933+0000","last_deep_scrub":"33'22","last_deep_scrub_stamp":"2024-09-19T11:27:04.935933+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.935933+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.148853+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"33'286","reported_seq":369,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.915549+0000","last_change":"2024-09-19T11:27:10.915484+0000","last_active":"2024-09-19T11:27:10.915549+0000","last_peered":"2024-09-19T11:27:10.915549+0000","last_clean":"2024-09-19T11:27:10.915549+0000","last_became_active":"2024-09-19T11:23:11.262010+0000","last_became_peered":"2024-09-19T11:23:11.262010+0000","last_unstale":"2024-09-19T11:27:10.915549+0000","last_undegraded":"2024-09-19T11:27:10.915549+0000","last_fullsized":"2024-09-19T11:27:10.915549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:10.915397+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:10.915397+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.915397+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:07.096881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:40.651947+0000","last_change":"2024-09-19T11:27:40.651836+0000","last_active":"2024-09-19T11:27:40.651947+0000","last_peered":"2024-09-19T11:27:40.651947+0000","last_clean":"2024-09-19T11:27:40.651947+0000","last_became_active":"2024-09-19T11:23:13.269450+0000","last_became_peered":"2024-09-19T11:23:13.269450+0000","last_unstale":"2024-09-19T11:27:40.651947+0000","last_undegraded":"2024-09-19T11:27:40.651947+0000","last_fullsized":"2024-09-19T11:27:40.651947+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:40.651729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:40.651729+0000","last_clean_scrub_stamp":"2024-09-19T11:27:40.651729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:01.744367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"33'12","reported_seq":79,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:53.381797+0000","last_change":"2024-09-19T11:26:53.381726+0000","last_active":"2024-09-19T11:26:53.381797+0000","last_peered":"2024-09-19T11:26:53.381797+0000","last_clean":"2024-09-19T11:26:53.381797+0000","last_became_active":"2024-09-19T11:23:05.253544+0000","last_became_peered":"2024-09-19T11:23:05.253544+0000","last_unstale":"2024-09-19T11:26:53.381797+0000","last_undegraded":"2024-09-19T11:26:53.381797+0000","last_fullsized":"2024-09-19T11:26:53.381797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:53.381618+0000","last_clean_scrub_stamp":"2024-09-19T11:26:53.381618+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:56.071102+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":49,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.185454+0000","last_change":"2024-09-19T11:26:28.185387+0000","last_active":"2024-09-19T11:26:28.185454+0000","last_peered":"2024-09-19T11:26:28.185454+0000","last_clean":"2024-09-19T11:26:28.185454+0000","last_became_active":"2024-09-19T11:23:03.241106+0000","last_became_peered":"2024-09-19T11:23:03.241106+0000","last_unstale":"2024-09-19T11:26:28.185454+0000","last_undegraded":"2024-09-19T11:26:28.185454+0000","last_fullsized":"2024-09-19T11:26:28.185454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.185291+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.185291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:48.217740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:23.897506+0000","last_change":"2024-09-19T11:26:23.897455+0000","last_active":"2024-09-19T11:26:23.897506+0000","last_peered":"2024-09-19T11:26:23.897506+0000","last_clean":"2024-09-19T11:26:23.897506+0000","last_became_active":"2024-09-19T11:22:53.811705+0000","last_became_peered":"2024-09-19T11:22:53.811705+0000","last_unstale":"2024-09-19T11:26:23.897506+0000","last_undegraded":"2024-09-19T11:26:23.897506+0000","last_fullsized":"2024-09-19T11:26:23.897506+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:23.897373+0000","last_clean_scrub_stamp":"2024-09-19T11:26:23.897373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:13.527667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000530442,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.503825+0000","last_change":"2024-09-19T11:27:09.503759+0000","last_active":"2024-09-19T11:27:09.503825+0000","last_peered":"2024-09-19T11:27:09.503825+0000","last_clean":"2024-09-19T11:27:09.503825+0000","last_became_active":"2024-09-19T11:23:07.244718+0000","last_became_peered":"2024-09-19T11:23:07.244718+0000","last_unstale":"2024-09-19T11:27:09.503825+0000","last_undegraded":"2024-09-19T11:27:09.503825+0000","last_fullsized":"2024-09-19T11:27:09.503825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:09.503672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:09.503672+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.503672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:53.319658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"33'16","reported_seq":56,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.512281+0000","last_change":"2024-09-19T11:27:18.512211+0000","last_active":"2024-09-19T11:27:18.512281+0000","last_peered":"2024-09-19T11:27:18.512281+0000","last_clean":"2024-09-19T11:27:18.512281+0000","last_became_active":"2024-09-19T11:23:09.255771+0000","last_became_peered":"2024-09-19T11:23:09.255771+0000","last_unstale":"2024-09-19T11:27:18.512281+0000","last_undegraded":"2024-09-19T11:27:18.512281+0000","last_fullsized":"2024-09-19T11:27:18.512281+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'16","last_scrub_stamp":"2024-09-19T11:27:18.512110+0000","last_deep_scrub":"33'16","last_deep_scrub_stamp":"2024-09-19T11:27:18.512110+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.512110+0000","objects_scrubbed":5,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:13.848124+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"33'286","reported_seq":443,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.791846+0000","last_change":"2024-09-19T11:27:09.791720+0000","last_active":"2024-09-19T11:27:09.791846+0000","last_peered":"2024-09-19T11:27:09.791846+0000","last_clean":"2024-09-19T11:27:09.791846+0000","last_became_active":"2024-09-19T11:23:11.257946+0000","last_became_peered":"2024-09-19T11:23:11.257946+0000","last_unstale":"2024-09-19T11:27:09.791846+0000","last_undegraded":"2024-09-19T11:27:09.791846+0000","last_fullsized":"2024-09-19T11:27:09.791846+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:09.791625+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:09.791625+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.791625+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:24.818374+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"31'3","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.916837+0000","last_change":"2024-09-19T11:27:20.916756+0000","last_active":"2024-09-19T11:27:20.916837+0000","last_peered":"2024-09-19T11:27:20.916837+0000","last_clean":"2024-09-19T11:27:20.916837+0000","last_became_active":"2024-09-19T11:23:13.274180+0000","last_became_peered":"2024-09-19T11:23:13.274180+0000","last_unstale":"2024-09-19T11:27:20.916837+0000","last_undegraded":"2024-09-19T11:27:20.916837+0000","last_fullsized":"2024-09-19T11:27:20.916837+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3","last_scrub_stamp":"2024-09-19T11:27:20.916668+0000","last_deep_scrub":"31'3","last_deep_scrub_stamp":"2024-09-19T11:27:20.916668+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.916668+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:56.260631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"33'4","reported_seq":54,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.850294+0000","last_change":"2024-09-19T11:26:39.850188+0000","last_active":"2024-09-19T11:26:39.850294+0000","last_peered":"2024-09-19T11:26:39.850294+0000","last_clean":"2024-09-19T11:26:39.850294+0000","last_became_active":"2024-09-19T11:23:05.249906+0000","last_became_peered":"2024-09-19T11:23:05.249906+0000","last_unstale":"2024-09-19T11:26:39.850294+0000","last_undegraded":"2024-09-19T11:26:39.850294+0000","last_fullsized":"2024-09-19T11:26:39.850294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.850077+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.850077+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:05.006871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:22.886575+0000","last_change":"2024-09-19T11:26:22.886518+0000","last_active":"2024-09-19T11:26:22.886575+0000","last_peered":"2024-09-19T11:26:22.886575+0000","last_clean":"2024-09-19T11:26:22.886575+0000","last_became_active":"2024-09-19T11:23:03.243450+0000","last_became_peered":"2024-09-19T11:23:03.243450+0000","last_unstale":"2024-09-19T11:26:22.886575+0000","last_undegraded":"2024-09-19T11:26:22.886575+0000","last_fullsized":"2024-09-19T11:26:22.886575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:22.886419+0000","last_clean_scrub_stamp":"2024-09-19T11:26:22.886419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:07.618282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":63,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.146522+0000","last_change":"2024-09-19T11:26:26.146468+0000","last_active":"2024-09-19T11:26:26.146522+0000","last_peered":"2024-09-19T11:26:26.146522+0000","last_clean":"2024-09-19T11:26:26.146522+0000","last_became_active":"2024-09-19T11:22:53.997643+0000","last_became_peered":"2024-09-19T11:22:53.997643+0000","last_unstale":"2024-09-19T11:26:26.146522+0000","last_undegraded":"2024-09-19T11:26:26.146522+0000","last_fullsized":"2024-09-19T11:26:26.146522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.146376+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.146376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:37.768766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001225554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.572672+0000","last_change":"2024-09-19T11:27:11.572672+0000","last_active":"2024-09-19T11:27:11.572672+0000","last_peered":"2024-09-19T11:27:11.572672+0000","last_clean":"2024-09-19T11:27:11.572672+0000","last_became_active":"2024-09-19T11:23:07.247078+0000","last_became_peered":"2024-09-19T11:23:07.247078+0000","last_unstale":"2024-09-19T11:27:11.572672+0000","last_undegraded":"2024-09-19T11:27:11.572672+0000","last_fullsized":"2024-09-19T11:27:11.572672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.572582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.572582+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.572582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:30:13.389084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"33'99","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.848493+0000","last_change":"2024-09-19T11:26:58.848423+0000","last_active":"2024-09-19T11:26:58.848493+0000","last_peered":"2024-09-19T11:26:58.848493+0000","last_clean":"2024-09-19T11:26:58.848493+0000","last_became_active":"2024-09-19T11:23:09.257998+0000","last_became_peered":"2024-09-19T11:23:09.257998+0000","last_unstale":"2024-09-19T11:26:58.848493+0000","last_undegraded":"2024-09-19T11:26:58.848493+0000","last_fullsized":"2024-09-19T11:26:58.848493+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'99","last_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_deep_scrub":"33'99","last_deep_scrub_stamp":"2024-09-19T11:26:58.848330+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.848330+0000","objects_scrubbed":8,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:48.091262+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:05.742808+0000","last_change":"2024-09-19T11:27:05.742747+0000","last_active":"2024-09-19T11:27:05.742808+0000","last_peered":"2024-09-19T11:27:05.742808+0000","last_clean":"2024-09-19T11:27:05.742808+0000","last_became_active":"2024-09-19T11:23:11.263333+0000","last_became_peered":"2024-09-19T11:23:11.263333+0000","last_unstale":"2024-09-19T11:27:05.742808+0000","last_undegraded":"2024-09-19T11:27:05.742808+0000","last_fullsized":"2024-09-19T11:27:05.742808+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:05.742667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:05.742667+0000","last_clean_scrub_stamp":"2024-09-19T11:27:05.742667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:19.395498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.964661+0000","last_change":"2024-09-19T11:27:16.964584+0000","last_active":"2024-09-19T11:27:16.964661+0000","last_peered":"2024-09-19T11:27:16.964661+0000","last_clean":"2024-09-19T11:27:16.964661+0000","last_became_active":"2024-09-19T11:23:13.265643+0000","last_became_peered":"2024-09-19T11:23:13.265643+0000","last_unstale":"2024-09-19T11:27:16.964661+0000","last_undegraded":"2024-09-19T11:27:16.964661+0000","last_fullsized":"2024-09-19T11:27:16.964661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:16.964498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:16.964498+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.964498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:49:58.369461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"31'2","reported_seq":29,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:18.936328+0000","last_change":"2024-09-19T11:27:18.936226+0000","last_active":"2024-09-19T11:27:18.936328+0000","last_peered":"2024-09-19T11:27:18.936328+0000","last_clean":"2024-09-19T11:27:18.936328+0000","last_became_active":"2024-09-19T11:23:13.270732+0000","last_became_peered":"2024-09-19T11:23:13.270732+0000","last_unstale":"2024-09-19T11:27:18.936328+0000","last_undegraded":"2024-09-19T11:27:18.936328+0000","last_fullsized":"2024-09-19T11:27:18.936328+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:27:18.936125+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:27:18.936125+0000","last_clean_scrub_stamp":"2024-09-19T11:27:18.936125+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:50.662583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.771752+0000","last_change":"2024-09-19T11:26:42.771683+0000","last_active":"2024-09-19T11:26:42.771752+0000","last_peered":"2024-09-19T11:26:42.771752+0000","last_clean":"2024-09-19T11:26:42.771752+0000","last_became_active":"2024-09-19T11:23:05.245342+0000","last_became_peered":"2024-09-19T11:23:05.245342+0000","last_unstale":"2024-09-19T11:26:42.771752+0000","last_undegraded":"2024-09-19T11:26:42.771752+0000","last_fullsized":"2024-09-19T11:26:42.771752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.771596+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.771596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:48:05.322666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:38.396229+0000","last_change":"2024-09-19T11:26:38.396108+0000","last_active":"2024-09-19T11:26:38.396229+0000","last_peered":"2024-09-19T11:26:38.396229+0000","last_clean":"2024-09-19T11:26:38.396229+0000","last_became_active":"2024-09-19T11:23:03.254040+0000","last_became_peered":"2024-09-19T11:23:03.254040+0000","last_unstale":"2024-09-19T11:26:38.396229+0000","last_undegraded":"2024-09-19T11:26:38.396229+0000","last_fullsized":"2024-09-19T11:26:38.396229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:38.396006+0000","last_clean_scrub_stamp":"2024-09-19T11:26:38.396006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:14.129195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.922218+0000","last_change":"2024-09-19T11:27:00.922151+0000","last_active":"2024-09-19T11:27:00.922218+0000","last_peered":"2024-09-19T11:27:00.922218+0000","last_clean":"2024-09-19T11:27:00.922218+0000","last_became_active":"2024-09-19T11:23:07.240385+0000","last_became_peered":"2024-09-19T11:23:07.240385+0000","last_unstale":"2024-09-19T11:27:00.922218+0000","last_undegraded":"2024-09-19T11:27:00.922218+0000","last_fullsized":"2024-09-19T11:27:00.922218+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.922068+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.922068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:51.324340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"33'40","reported_seq":97,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:24.595028+0000","last_change":"2024-09-19T11:27:24.594970+0000","last_active":"2024-09-19T11:27:24.595028+0000","last_peered":"2024-09-19T11:27:24.595028+0000","last_clean":"2024-09-19T11:27:24.595028+0000","last_became_active":"2024-09-19T11:23:09.257402+0000","last_became_peered":"2024-09-19T11:23:09.257402+0000","last_unstale":"2024-09-19T11:27:24.595028+0000","last_undegraded":"2024-09-19T11:27:24.595028+0000","last_fullsized":"2024-09-19T11:27:24.595028+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'40","last_scrub_stamp":"2024-09-19T11:27:24.594878+0000","last_deep_scrub":"33'40","last_deep_scrub_stamp":"2024-09-19T11:27:24.594878+0000","last_clean_scrub_stamp":"2024-09-19T11:27:24.594878+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:48:58.860644+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:34.693711+0000","last_change":"2024-09-19T11:27:34.693641+0000","last_active":"2024-09-19T11:27:34.693711+0000","last_peered":"2024-09-19T11:27:34.693711+0000","last_clean":"2024-09-19T11:27:34.693711+0000","last_became_active":"2024-09-19T11:23:11.270739+0000","last_became_peered":"2024-09-19T11:23:11.270739+0000","last_unstale":"2024-09-19T11:27:34.693711+0000","last_undegraded":"2024-09-19T11:27:34.693711+0000","last_fullsized":"2024-09-19T11:27:34.693711+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:34.693536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:34.693536+0000","last_clean_scrub_stamp":"2024-09-19T11:27:34.693536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:09.572432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"33'3","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:23.851220+0000","last_change":"2024-09-19T11:27:23.851220+0000","last_active":"2024-09-19T11:27:23.851220+0000","last_peered":"2024-09-19T11:27:23.851220+0000","last_clean":"2024-09-19T11:27:23.851220+0000","last_became_active":"2024-09-19T11:23:13.273227+0000","last_became_peered":"2024-09-19T11:23:13.273227+0000","last_unstale":"2024-09-19T11:27:23.851220+0000","last_undegraded":"2024-09-19T11:27:23.851220+0000","last_fullsized":"2024-09-19T11:27:23.851220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:23.851126+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:23.851126+0000","last_clean_scrub_stamp":"2024-09-19T11:27:23.851126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:40.839787+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.036356+0000","last_change":"2024-09-19T11:26:41.036281+0000","last_active":"2024-09-19T11:26:41.036356+0000","last_peered":"2024-09-19T11:26:41.036356+0000","last_clean":"2024-09-19T11:26:41.036356+0000","last_became_active":"2024-09-19T11:23:42.285391+0000","last_became_peered":"2024-09-19T11:23:42.285391+0000","last_unstale":"2024-09-19T11:26:41.036356+0000","last_undegraded":"2024-09-19T11:26:41.036356+0000","last_fullsized":"2024-09-19T11:26:41.036356+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:41.036169+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.036169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:34.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.922067+0000","last_change":"2024-09-19T11:26:28.921998+0000","last_active":"2024-09-19T11:26:28.922067+0000","last_peered":"2024-09-19T11:26:28.922067+0000","last_clean":"2024-09-19T11:26:28.922067+0000","last_became_active":"2024-09-19T11:23:03.255559+0000","last_became_peered":"2024-09-19T11:23:03.255559+0000","last_unstale":"2024-09-19T11:26:28.922067+0000","last_undegraded":"2024-09-19T11:26:28.922067+0000","last_fullsized":"2024-09-19T11:26:28.922067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'2","last_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_deep_scrub":"31'2","last_deep_scrub_stamp":"2024-09-19T11:26:28.921906+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.921906+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:00.637589+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.583223+0000","last_change":"2024-09-19T11:27:14.583167+0000","last_active":"2024-09-19T11:27:14.583223+0000","last_peered":"2024-09-19T11:27:14.583223+0000","last_clean":"2024-09-19T11:27:14.583223+0000","last_became_active":"2024-09-19T11:23:07.248489+0000","last_became_peered":"2024-09-19T11:23:07.248489+0000","last_unstale":"2024-09-19T11:27:14.583223+0000","last_undegraded":"2024-09-19T11:27:14.583223+0000","last_fullsized":"2024-09-19T11:27:14.583223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.583070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.583070+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.583070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:47.425828+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"33'8","reported_seq":49,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.727384+0000","last_change":"2024-09-19T11:27:01.727321+0000","last_active":"2024-09-19T11:27:01.727384+0000","last_peered":"2024-09-19T11:27:01.727384+0000","last_clean":"2024-09-19T11:27:01.727384+0000","last_became_active":"2024-09-19T11:23:09.244654+0000","last_became_peered":"2024-09-19T11:23:09.244654+0000","last_unstale":"2024-09-19T11:27:01.727384+0000","last_undegraded":"2024-09-19T11:27:01.727384+0000","last_fullsized":"2024-09-19T11:27:01.727384+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:01.727234+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:01.727234+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.727234+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:43:28.373856+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:09.813816+0000","last_change":"2024-09-19T11:27:09.813741+0000","last_active":"2024-09-19T11:27:09.813816+0000","last_peered":"2024-09-19T11:27:09.813816+0000","last_clean":"2024-09-19T11:27:09.813816+0000","last_became_active":"2024-09-19T11:23:11.267737+0000","last_became_peered":"2024-09-19T11:23:11.267737+0000","last_unstale":"2024-09-19T11:27:09.813816+0000","last_undegraded":"2024-09-19T11:27:09.813816+0000","last_fullsized":"2024-09-19T11:27:09.813816+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:09.813641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:09.813641+0000","last_clean_scrub_stamp":"2024-09-19T11:27:09.813641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:52:43.359127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:22.823406+0000","last_change":"2024-09-19T11:27:22.823406+0000","last_active":"2024-09-19T11:27:22.823406+0000","last_peered":"2024-09-19T11:27:22.823406+0000","last_clean":"2024-09-19T11:27:22.823406+0000","last_became_active":"2024-09-19T11:23:13.266870+0000","last_became_peered":"2024-09-19T11:23:13.266870+0000","last_unstale":"2024-09-19T11:27:22.823406+0000","last_undegraded":"2024-09-19T11:27:22.823406+0000","last_fullsized":"2024-09-19T11:27:22.823406+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:22.823310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:22.823310+0000","last_clean_scrub_stamp":"2024-09-19T11:27:22.823310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:50.964355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"33'8","reported_seq":72,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.002077+0000","last_change":"2024-09-19T11:26:41.001963+0000","last_active":"2024-09-19T11:26:41.002077+0000","last_peered":"2024-09-19T11:26:41.002077+0000","last_clean":"2024-09-19T11:26:41.002077+0000","last_became_active":"2024-09-19T11:23:05.239876+0000","last_became_peered":"2024-09-19T11:23:05.239876+0000","last_unstale":"2024-09-19T11:26:41.002077+0000","last_undegraded":"2024-09-19T11:26:41.002077+0000","last_fullsized":"2024-09-19T11:26:41.002077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:41.001858+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.001858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:06.599582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"33'6","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.938101+0000","last_change":"2024-09-19T11:26:29.937976+0000","last_active":"2024-09-19T11:26:29.938101+0000","last_peered":"2024-09-19T11:26:29.938101+0000","last_clean":"2024-09-19T11:26:29.938101+0000","last_became_active":"2024-09-19T11:23:03.257214+0000","last_became_peered":"2024-09-19T11:23:03.257214+0000","last_unstale":"2024-09-19T11:26:29.938101+0000","last_undegraded":"2024-09-19T11:26:29.938101+0000","last_fullsized":"2024-09-19T11:26:29.938101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:29.937882+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.937882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:14.058304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":8,"num_write":30,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:51.813328+0000","last_change":"2024-09-19T11:26:51.813274+0000","last_active":"2024-09-19T11:26:51.813328+0000","last_peered":"2024-09-19T11:26:51.813328+0000","last_clean":"2024-09-19T11:26:51.813328+0000","last_became_active":"2024-09-19T11:23:07.243173+0000","last_became_peered":"2024-09-19T11:23:07.243173+0000","last_unstale":"2024-09-19T11:26:51.813328+0000","last_undegraded":"2024-09-19T11:26:51.813328+0000","last_fullsized":"2024-09-19T11:26:51.813328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:51.813187+0000","last_clean_scrub_stamp":"2024-09-19T11:26:51.813187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:32.845364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"33'49","reported_seq":122,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.759170+0000","last_change":"2024-09-19T11:27:00.759116+0000","last_active":"2024-09-19T11:27:00.759170+0000","last_peered":"2024-09-19T11:27:00.759170+0000","last_clean":"2024-09-19T11:27:00.759170+0000","last_became_active":"2024-09-19T11:23:09.248372+0000","last_became_peered":"2024-09-19T11:23:09.248372+0000","last_unstale":"2024-09-19T11:27:00.759170+0000","last_undegraded":"2024-09-19T11:27:00.759170+0000","last_fullsized":"2024-09-19T11:27:00.759170+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'49","last_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_deep_scrub":"33'49","last_deep_scrub_stamp":"2024-09-19T11:27:00.759010+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.759010+0000","objects_scrubbed":6,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:16.347162+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.892563+0000","last_change":"2024-09-19T11:27:11.892506+0000","last_active":"2024-09-19T11:27:11.892563+0000","last_peered":"2024-09-19T11:27:11.892563+0000","last_clean":"2024-09-19T11:27:11.892563+0000","last_became_active":"2024-09-19T11:23:11.262377+0000","last_became_peered":"2024-09-19T11:23:11.262377+0000","last_unstale":"2024-09-19T11:27:11.892563+0000","last_undegraded":"2024-09-19T11:27:11.892563+0000","last_fullsized":"2024-09-19T11:27:11.892563+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.892416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.892416+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.892416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:56:17.528967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:46.606568+0000","last_change":"2024-09-19T11:27:46.606568+0000","last_active":"2024-09-19T11:27:46.606568+0000","last_peered":"2024-09-19T11:27:46.606568+0000","last_clean":"2024-09-19T11:27:46.606568+0000","last_became_active":"2024-09-19T11:23:13.274414+0000","last_became_peered":"2024-09-19T11:23:13.274414+0000","last_unstale":"2024-09-19T11:27:46.606568+0000","last_undegraded":"2024-09-19T11:27:46.606568+0000","last_fullsized":"2024-09-19T11:27:46.606568+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:46.606466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:46.606466+0000","last_clean_scrub_stamp":"2024-09-19T11:27:46.606466+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:40:07.359161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'10","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:58.392292+0000","last_change":"2024-09-19T11:26:58.392197+0000","last_active":"2024-09-19T11:26:58.392292+0000","last_peered":"2024-09-19T11:26:58.392292+0000","last_clean":"2024-09-19T11:26:58.392292+0000","last_became_active":"2024-09-19T11:23:05.252398+0000","last_became_peered":"2024-09-19T11:23:05.252398+0000","last_unstale":"2024-09-19T11:26:58.392292+0000","last_undegraded":"2024-09-19T11:26:58.392292+0000","last_fullsized":"2024-09-19T11:26:58.392292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:58.392102+0000","last_clean_scrub_stamp":"2024-09-19T11:26:58.392102+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:10.174526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:25.886564+0000","last_change":"2024-09-19T11:26:25.886507+0000","last_active":"2024-09-19T11:26:25.886564+0000","last_peered":"2024-09-19T11:26:25.886564+0000","last_clean":"2024-09-19T11:26:25.886564+0000","last_became_active":"2024-09-19T11:23:03.245903+0000","last_became_peered":"2024-09-19T11:23:03.245903+0000","last_unstale":"2024-09-19T11:26:25.886564+0000","last_undegraded":"2024-09-19T11:26:25.886564+0000","last_fullsized":"2024-09-19T11:26:25.886564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:25.886414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:25.886414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:38.573945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.585069+0000","last_change":"2024-09-19T11:27:12.584986+0000","last_active":"2024-09-19T11:27:12.585069+0000","last_peered":"2024-09-19T11:27:12.585069+0000","last_clean":"2024-09-19T11:27:12.585069+0000","last_became_active":"2024-09-19T11:23:07.245026+0000","last_became_peered":"2024-09-19T11:23:07.245026+0000","last_unstale":"2024-09-19T11:27:12.585069+0000","last_undegraded":"2024-09-19T11:27:12.585069+0000","last_fullsized":"2024-09-19T11:27:12.585069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:12.584897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:12.584897+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.584897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:55.169943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"33'38","reported_seq":94,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.810289+0000","last_change":"2024-09-19T11:27:03.810233+0000","last_active":"2024-09-19T11:27:03.810289+0000","last_peered":"2024-09-19T11:27:03.810289+0000","last_clean":"2024-09-19T11:27:03.810289+0000","last_became_active":"2024-09-19T11:23:09.245974+0000","last_became_peered":"2024-09-19T11:23:09.245974+0000","last_unstale":"2024-09-19T11:27:03.810289+0000","last_undegraded":"2024-09-19T11:27:03.810289+0000","last_fullsized":"2024-09-19T11:27:03.810289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:03.810143+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:03.810143+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.810143+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:31.289316+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:11.822301+0000","last_change":"2024-09-19T11:27:11.822230+0000","last_active":"2024-09-19T11:27:11.822301+0000","last_peered":"2024-09-19T11:27:11.822301+0000","last_clean":"2024-09-19T11:27:11.822301+0000","last_became_active":"2024-09-19T11:23:11.253401+0000","last_became_peered":"2024-09-19T11:23:11.253401+0000","last_unstale":"2024-09-19T11:27:11.822301+0000","last_undegraded":"2024-09-19T11:27:11.822301+0000","last_fullsized":"2024-09-19T11:27:11.822301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:11.822138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:11.822138+0000","last_clean_scrub_stamp":"2024-09-19T11:27:11.822138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:55:55.083098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.486672+0000","last_change":"2024-09-19T11:27:01.486612+0000","last_active":"2024-09-19T11:27:01.486672+0000","last_peered":"2024-09-19T11:27:01.486672+0000","last_clean":"2024-09-19T11:27:01.486672+0000","last_became_active":"2024-09-19T11:23:05.236328+0000","last_became_peered":"2024-09-19T11:23:05.236328+0000","last_unstale":"2024-09-19T11:27:01.486672+0000","last_undegraded":"2024-09-19T11:27:01.486672+0000","last_fullsized":"2024-09-19T11:27:01.486672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:01.486507+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.486507+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:11.419750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"33'4","reported_seq":52,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:39.386822+0000","last_change":"2024-09-19T11:26:39.386729+0000","last_active":"2024-09-19T11:26:39.386822+0000","last_peered":"2024-09-19T11:26:39.386822+0000","last_clean":"2024-09-19T11:26:39.386822+0000","last_became_active":"2024-09-19T11:23:03.253821+0000","last_became_peered":"2024-09-19T11:23:03.253821+0000","last_unstale":"2024-09-19T11:26:39.386822+0000","last_undegraded":"2024-09-19T11:26:39.386822+0000","last_fullsized":"2024-09-19T11:26:39.386822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:39.386636+0000","last_clean_scrub_stamp":"2024-09-19T11:26:39.386636+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:26.742196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:55.840087+0000","last_change":"2024-09-19T11:26:55.840003+0000","last_active":"2024-09-19T11:26:55.840087+0000","last_peered":"2024-09-19T11:26:55.840087+0000","last_clean":"2024-09-19T11:26:55.840087+0000","last_became_active":"2024-09-19T11:23:42.944695+0000","last_became_peered":"2024-09-19T11:23:42.944695+0000","last_unstale":"2024-09-19T11:26:55.840087+0000","last_undegraded":"2024-09-19T11:26:55.840087+0000","last_fullsized":"2024-09-19T11:26:55.840087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:55.839922+0000","last_clean_scrub_stamp":"2024-09-19T11:26:55.839922+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:06.241449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"33'67","reported_seq":140,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.770359+0000","last_change":"2024-09-19T11:27:03.770359+0000","last_active":"2024-09-19T11:27:03.770359+0000","last_peered":"2024-09-19T11:27:03.770359+0000","last_clean":"2024-09-19T11:27:03.770359+0000","last_became_active":"2024-09-19T11:23:09.246299+0000","last_became_peered":"2024-09-19T11:23:09.246299+0000","last_unstale":"2024-09-19T11:27:03.770359+0000","last_undegraded":"2024-09-19T11:27:03.770359+0000","last_fullsized":"2024-09-19T11:27:03.770359+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'67","last_scrub_stamp":"2024-09-19T11:27:03.770268+0000","last_deep_scrub":"33'67","last_deep_scrub_stamp":"2024-09-19T11:27:03.770268+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.770268+0000","objects_scrubbed":5,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:59.209518+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4206,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":65,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:10.859547+0000","last_change":"2024-09-19T11:27:10.859547+0000","last_active":"2024-09-19T11:27:10.859547+0000","last_peered":"2024-09-19T11:27:10.859547+0000","last_clean":"2024-09-19T11:27:10.859547+0000","last_became_active":"2024-09-19T11:23:11.260049+0000","last_became_peered":"2024-09-19T11:23:11.260049+0000","last_unstale":"2024-09-19T11:27:10.859547+0000","last_undegraded":"2024-09-19T11:27:10.859547+0000","last_fullsized":"2024-09-19T11:27:10.859547+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:10.859446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:10.859446+0000","last_clean_scrub_stamp":"2024-09-19T11:27:10.859446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:44.106754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"33'14","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:20.861372+0000","last_change":"2024-09-19T11:27:20.861251+0000","last_active":"2024-09-19T11:27:20.861372+0000","last_peered":"2024-09-19T11:27:20.861372+0000","last_clean":"2024-09-19T11:27:20.861372+0000","last_became_active":"2024-09-19T11:23:13.267785+0000","last_became_peered":"2024-09-19T11:23:13.267785+0000","last_unstale":"2024-09-19T11:27:20.861372+0000","last_undegraded":"2024-09-19T11:27:20.861372+0000","last_fullsized":"2024-09-19T11:27:20.861372+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:27:20.861155+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:27:20.861155+0000","last_clean_scrub_stamp":"2024-09-19T11:27:20.861155+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:45.440786+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"33'10","reported_seq":80,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.954173+0000","last_change":"2024-09-19T11:26:43.954113+0000","last_active":"2024-09-19T11:26:43.954173+0000","last_peered":"2024-09-19T11:26:43.954173+0000","last_clean":"2024-09-19T11:26:43.954173+0000","last_became_active":"2024-09-19T11:23:05.246055+0000","last_became_peered":"2024-09-19T11:23:05.246055+0000","last_unstale":"2024-09-19T11:26:43.954173+0000","last_undegraded":"2024-09-19T11:26:43.954173+0000","last_fullsized":"2024-09-19T11:26:43.954173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'10","last_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_deep_scrub":"33'10","last_deep_scrub_stamp":"2024-09-19T11:26:43.954028+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.954028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:24.870979+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":51,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.929576+0000","last_change":"2024-09-19T11:26:26.929509+0000","last_active":"2024-09-19T11:26:26.929576+0000","last_peered":"2024-09-19T11:26:26.929576+0000","last_clean":"2024-09-19T11:26:26.929576+0000","last_became_active":"2024-09-19T11:23:03.246373+0000","last_became_peered":"2024-09-19T11:23:03.246373+0000","last_unstale":"2024-09-19T11:26:26.929576+0000","last_undegraded":"2024-09-19T11:26:26.929576+0000","last_fullsized":"2024-09-19T11:26:26.929576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:26.929422+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.929422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:53.997533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.861748+0000","last_change":"2024-09-19T11:27:00.861682+0000","last_active":"2024-09-19T11:27:00.861748+0000","last_peered":"2024-09-19T11:27:00.861748+0000","last_clean":"2024-09-19T11:27:00.861748+0000","last_became_active":"2024-09-19T11:23:07.242864+0000","last_became_peered":"2024-09-19T11:23:07.242864+0000","last_unstale":"2024-09-19T11:27:00.861748+0000","last_undegraded":"2024-09-19T11:27:00.861748+0000","last_fullsized":"2024-09-19T11:27:00.861748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.861599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.861599+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.861599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:41.778551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"33'46","reported_seq":104,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:29.645354+0000","last_change":"2024-09-19T11:27:29.645354+0000","last_active":"2024-09-19T11:27:29.645354+0000","last_peered":"2024-09-19T11:27:29.645354+0000","last_clean":"2024-09-19T11:27:29.645354+0000","last_became_active":"2024-09-19T11:23:09.255623+0000","last_became_peered":"2024-09-19T11:23:09.255623+0000","last_unstale":"2024-09-19T11:27:29.645354+0000","last_undegraded":"2024-09-19T11:27:29.645354+0000","last_fullsized":"2024-09-19T11:27:29.645354+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:29.645252+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:29.645252+0000","last_clean_scrub_stamp":"2024-09-19T11:27:29.645252+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:55:18.923250+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.953402+0000","last_change":"2024-09-19T11:27:15.953402+0000","last_active":"2024-09-19T11:27:15.953402+0000","last_peered":"2024-09-19T11:27:15.953402+0000","last_clean":"2024-09-19T11:27:15.953402+0000","last_became_active":"2024-09-19T11:23:11.266458+0000","last_became_peered":"2024-09-19T11:23:11.266458+0000","last_unstale":"2024-09-19T11:27:15.953402+0000","last_undegraded":"2024-09-19T11:27:15.953402+0000","last_fullsized":"2024-09-19T11:27:15.953402+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.953297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.953297+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.953297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:28.014945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:43.639691+0000","last_change":"2024-09-19T11:27:43.639591+0000","last_active":"2024-09-19T11:27:43.639691+0000","last_peered":"2024-09-19T11:27:43.639691+0000","last_clean":"2024-09-19T11:27:43.639691+0000","last_became_active":"2024-09-19T11:23:13.273004+0000","last_became_peered":"2024-09-19T11:23:13.273004+0000","last_unstale":"2024-09-19T11:27:43.639691+0000","last_undegraded":"2024-09-19T11:27:43.639691+0000","last_fullsized":"2024-09-19T11:27:43.639691+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:43.639490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:43.639490+0000","last_clean_scrub_stamp":"2024-09-19T11:27:43.639490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:51.685106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:00.474660+0000","last_change":"2024-09-19T11:27:00.474660+0000","last_active":"2024-09-19T11:27:00.474660+0000","last_peered":"2024-09-19T11:27:00.474660+0000","last_clean":"2024-09-19T11:27:00.474660+0000","last_became_active":"2024-09-19T11:23:05.253825+0000","last_became_peered":"2024-09-19T11:23:05.253825+0000","last_unstale":"2024-09-19T11:27:00.474660+0000","last_undegraded":"2024-09-19T11:27:00.474660+0000","last_fullsized":"2024-09-19T11:27:00.474660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:00.474573+0000","last_clean_scrub_stamp":"2024-09-19T11:27:00.474573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:59.322724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:28.877728+0000","last_change":"2024-09-19T11:26:28.877728+0000","last_active":"2024-09-19T11:26:28.877728+0000","last_peered":"2024-09-19T11:26:28.877728+0000","last_clean":"2024-09-19T11:26:28.877728+0000","last_became_active":"2024-09-19T11:23:03.243721+0000","last_became_peered":"2024-09-19T11:23:03.243721+0000","last_unstale":"2024-09-19T11:26:28.877728+0000","last_undegraded":"2024-09-19T11:26:28.877728+0000","last_fullsized":"2024-09-19T11:26:28.877728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:28.877640+0000","last_clean_scrub_stamp":"2024-09-19T11:26:28.877640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:09.086406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:57.801547+0000","last_change":"2024-09-19T11:26:57.801547+0000","last_active":"2024-09-19T11:26:57.801547+0000","last_peered":"2024-09-19T11:26:57.801547+0000","last_clean":"2024-09-19T11:26:57.801547+0000","last_became_active":"2024-09-19T11:23:07.241446+0000","last_became_peered":"2024-09-19T11:23:07.241446+0000","last_unstale":"2024-09-19T11:26:57.801547+0000","last_undegraded":"2024-09-19T11:26:57.801547+0000","last_fullsized":"2024-09-19T11:26:57.801547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:57.801457+0000","last_clean_scrub_stamp":"2024-09-19T11:26:57.801457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:23.399181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"33'42","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.773714+0000","last_change":"2024-09-19T11:27:04.773660+0000","last_active":"2024-09-19T11:27:04.773714+0000","last_peered":"2024-09-19T11:27:04.773714+0000","last_clean":"2024-09-19T11:27:04.773714+0000","last_became_active":"2024-09-19T11:23:09.257859+0000","last_became_peered":"2024-09-19T11:23:09.257859+0000","last_unstale":"2024-09-19T11:27:04.773714+0000","last_undegraded":"2024-09-19T11:27:04.773714+0000","last_fullsized":"2024-09-19T11:27:04.773714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'42","last_scrub_stamp":"2024-09-19T11:27:04.773575+0000","last_deep_scrub":"33'42","last_deep_scrub_stamp":"2024-09-19T11:27:04.773575+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.773575+0000","objects_scrubbed":5,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:44.880940+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":55,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:16.829408+0000","last_change":"2024-09-19T11:27:16.829408+0000","last_active":"2024-09-19T11:27:16.829408+0000","last_peered":"2024-09-19T11:27:16.829408+0000","last_clean":"2024-09-19T11:27:16.829408+0000","last_became_active":"2024-09-19T11:23:11.261454+0000","last_became_peered":"2024-09-19T11:23:11.261454+0000","last_unstale":"2024-09-19T11:27:16.829408+0000","last_undegraded":"2024-09-19T11:27:16.829408+0000","last_fullsized":"2024-09-19T11:27:16.829408+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:16.829319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:16.829319+0000","last_clean_scrub_stamp":"2024-09-19T11:27:16.829319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:55:59.496334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"33'3","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:42.675081+0000","last_change":"2024-09-19T11:27:42.675004+0000","last_active":"2024-09-19T11:27:42.675081+0000","last_peered":"2024-09-19T11:27:42.675081+0000","last_clean":"2024-09-19T11:27:42.675081+0000","last_became_active":"2024-09-19T11:23:13.273073+0000","last_became_peered":"2024-09-19T11:23:13.273073+0000","last_unstale":"2024-09-19T11:27:42.675081+0000","last_undegraded":"2024-09-19T11:27:42.675081+0000","last_fullsized":"2024-09-19T11:27:42.675081+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:42.674916+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:42.674916+0000","last_clean_scrub_stamp":"2024-09-19T11:27:42.674916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:02:22.322376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"33'8","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.000020+0000","last_change":"2024-09-19T11:26:43.000020+0000","last_active":"2024-09-19T11:26:43.000020+0000","last_peered":"2024-09-19T11:26:43.000020+0000","last_clean":"2024-09-19T11:26:43.000020+0000","last_became_active":"2024-09-19T11:23:05.237905+0000","last_became_peered":"2024-09-19T11:23:05.237905+0000","last_unstale":"2024-09-19T11:26:43.000020+0000","last_undegraded":"2024-09-19T11:26:43.000020+0000","last_fullsized":"2024-09-19T11:26:43.000020+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:42.999915+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.999915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:23.688053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:40.342895+0000","last_change":"2024-09-19T11:26:40.342895+0000","last_active":"2024-09-19T11:26:40.342895+0000","last_peered":"2024-09-19T11:26:40.342895+0000","last_clean":"2024-09-19T11:26:40.342895+0000","last_became_active":"2024-09-19T11:23:03.253172+0000","last_became_peered":"2024-09-19T11:23:03.253172+0000","last_unstale":"2024-09-19T11:26:40.342895+0000","last_undegraded":"2024-09-19T11:26:40.342895+0000","last_fullsized":"2024-09-19T11:26:40.342895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:40.342808+0000","last_clean_scrub_stamp":"2024-09-19T11:26:40.342808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:52.126397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":44,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.896501+0000","last_change":"2024-09-19T11:27:03.896501+0000","last_active":"2024-09-19T11:27:03.896501+0000","last_peered":"2024-09-19T11:27:03.896501+0000","last_clean":"2024-09-19T11:27:03.896501+0000","last_became_active":"2024-09-19T11:23:42.285788+0000","last_became_peered":"2024-09-19T11:23:42.285788+0000","last_unstale":"2024-09-19T11:27:03.896501+0000","last_undegraded":"2024-09-19T11:27:03.896501+0000","last_fullsized":"2024-09-19T11:27:03.896501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:03.896413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:03.896413+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.896413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:17.149571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"33'15","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:32.675558+0000","last_change":"2024-09-19T11:27:32.675558+0000","last_active":"2024-09-19T11:27:32.675558+0000","last_peered":"2024-09-19T11:27:32.675558+0000","last_clean":"2024-09-19T11:27:32.675558+0000","last_became_active":"2024-09-19T11:23:09.256872+0000","last_became_peered":"2024-09-19T11:23:09.256872+0000","last_unstale":"2024-09-19T11:27:32.675558+0000","last_undegraded":"2024-09-19T11:27:32.675558+0000","last_fullsized":"2024-09-19T11:27:32.675558+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'15","last_scrub_stamp":"2024-09-19T11:27:32.675456+0000","last_deep_scrub":"33'15","last_deep_scrub_stamp":"2024-09-19T11:27:32.675456+0000","last_clean_scrub_stamp":"2024-09-19T11:27:32.675456+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:30:09.848845+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"33'286","reported_seq":431,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:12.871742+0000","last_change":"2024-09-19T11:27:12.871624+0000","last_active":"2024-09-19T11:27:12.871742+0000","last_peered":"2024-09-19T11:27:12.871742+0000","last_clean":"2024-09-19T11:27:12.871742+0000","last_became_active":"2024-09-19T11:23:11.267659+0000","last_became_peered":"2024-09-19T11:23:11.267659+0000","last_unstale":"2024-09-19T11:27:12.871742+0000","last_undegraded":"2024-09-19T11:27:12.871742+0000","last_fullsized":"2024-09-19T11:27:12.871742+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'286","last_scrub_stamp":"2024-09-19T11:27:12.871525+0000","last_deep_scrub":"33'286","last_deep_scrub_stamp":"2024-09-19T11:27:12.871525+0000","last_clean_scrub_stamp":"2024-09-19T11:27:12.871525+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:18.160862+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:41.641664+0000","last_change":"2024-09-19T11:27:41.641595+0000","last_active":"2024-09-19T11:27:41.641664+0000","last_peered":"2024-09-19T11:27:41.641664+0000","last_clean":"2024-09-19T11:27:41.641664+0000","last_became_active":"2024-09-19T11:23:13.272000+0000","last_became_peered":"2024-09-19T11:23:13.272000+0000","last_unstale":"2024-09-19T11:27:41.641664+0000","last_undegraded":"2024-09-19T11:27:41.641664+0000","last_fullsized":"2024-09-19T11:27:41.641664+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:41.641507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:41.641507+0000","last_clean_scrub_stamp":"2024-09-19T11:27:41.641507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:55:08.499395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"33'4","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.773518+0000","last_change":"2024-09-19T11:26:47.773400+0000","last_active":"2024-09-19T11:26:47.773518+0000","last_peered":"2024-09-19T11:26:47.773518+0000","last_clean":"2024-09-19T11:26:47.773518+0000","last_became_active":"2024-09-19T11:23:05.235292+0000","last_became_peered":"2024-09-19T11:23:05.235292+0000","last_unstale":"2024-09-19T11:26:47.773518+0000","last_undegraded":"2024-09-19T11:26:47.773518+0000","last_fullsized":"2024-09-19T11:26:47.773518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:47.773301+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.773301+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:48.002135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.907839+0000","last_change":"2024-09-19T11:26:27.907839+0000","last_active":"2024-09-19T11:26:27.907839+0000","last_peered":"2024-09-19T11:26:27.907839+0000","last_clean":"2024-09-19T11:26:27.907839+0000","last_became_active":"2024-09-19T11:23:03.244985+0000","last_became_peered":"2024-09-19T11:23:03.244985+0000","last_unstale":"2024-09-19T11:26:27.907839+0000","last_undegraded":"2024-09-19T11:26:27.907839+0000","last_fullsized":"2024-09-19T11:26:27.907839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.907726+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.907726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:48.160182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.884954+0000","last_change":"2024-09-19T11:27:02.884954+0000","last_active":"2024-09-19T11:27:02.884954+0000","last_peered":"2024-09-19T11:27:02.884954+0000","last_clean":"2024-09-19T11:27:02.884954+0000","last_became_active":"2024-09-19T11:23:07.246371+0000","last_became_peered":"2024-09-19T11:23:07.246371+0000","last_unstale":"2024-09-19T11:27:02.884954+0000","last_undegraded":"2024-09-19T11:27:02.884954+0000","last_fullsized":"2024-09-19T11:27:02.884954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:02.884869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:02.884869+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.884869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:00.122518+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"33'46","reported_seq":105,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:08.960022+0000","last_change":"2024-09-19T11:27:08.959960+0000","last_active":"2024-09-19T11:27:08.960022+0000","last_peered":"2024-09-19T11:27:08.960022+0000","last_clean":"2024-09-19T11:27:08.960022+0000","last_became_active":"2024-09-19T11:23:09.246478+0000","last_became_peered":"2024-09-19T11:23:09.246478+0000","last_unstale":"2024-09-19T11:27:08.960022+0000","last_undegraded":"2024-09-19T11:27:08.960022+0000","last_fullsized":"2024-09-19T11:27:08.960022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'46","last_scrub_stamp":"2024-09-19T11:27:08.959862+0000","last_deep_scrub":"33'46","last_deep_scrub_stamp":"2024-09-19T11:27:08.959862+0000","last_clean_scrub_stamp":"2024-09-19T11:27:08.959862+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:37.904449+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:14.836678+0000","last_change":"2024-09-19T11:27:14.836563+0000","last_active":"2024-09-19T11:27:14.836678+0000","last_peered":"2024-09-19T11:27:14.836678+0000","last_clean":"2024-09-19T11:27:14.836678+0000","last_became_active":"2024-09-19T11:23:11.266176+0000","last_became_peered":"2024-09-19T11:23:11.266176+0000","last_unstale":"2024-09-19T11:27:14.836678+0000","last_undegraded":"2024-09-19T11:27:14.836678+0000","last_fullsized":"2024-09-19T11:27:14.836678+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:14.836468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:14.836468+0000","last_clean_scrub_stamp":"2024-09-19T11:27:14.836468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:05.472792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:21.916333+0000","last_change":"2024-09-19T11:27:21.916333+0000","last_active":"2024-09-19T11:27:21.916333+0000","last_peered":"2024-09-19T11:27:21.916333+0000","last_clean":"2024-09-19T11:27:21.916333+0000","last_became_active":"2024-09-19T11:23:13.268478+0000","last_became_peered":"2024-09-19T11:23:13.268478+0000","last_unstale":"2024-09-19T11:27:21.916333+0000","last_undegraded":"2024-09-19T11:27:21.916333+0000","last_fullsized":"2024-09-19T11:27:21.916333+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:21.916240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:21.916240+0000","last_clean_scrub_stamp":"2024-09-19T11:27:21.916240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:27:07.280285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"33'6","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.724920+0000","last_change":"2024-09-19T11:26:46.724840+0000","last_active":"2024-09-19T11:26:46.724920+0000","last_peered":"2024-09-19T11:26:46.724920+0000","last_clean":"2024-09-19T11:26:46.724920+0000","last_became_active":"2024-09-19T11:23:05.236336+0000","last_became_peered":"2024-09-19T11:23:05.236336+0000","last_unstale":"2024-09-19T11:26:46.724920+0000","last_undegraded":"2024-09-19T11:26:46.724920+0000","last_fullsized":"2024-09-19T11:26:46.724920+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:46.724740+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.724740+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:47.953525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:41.382940+0000","last_change":"2024-09-19T11:26:41.382940+0000","last_active":"2024-09-19T11:26:41.382940+0000","last_peered":"2024-09-19T11:26:41.382940+0000","last_clean":"2024-09-19T11:26:41.382940+0000","last_became_active":"2024-09-19T11:23:03.257631+0000","last_became_peered":"2024-09-19T11:23:03.257631+0000","last_unstale":"2024-09-19T11:26:41.382940+0000","last_undegraded":"2024-09-19T11:26:41.382940+0000","last_fullsized":"2024-09-19T11:26:41.382940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:41.382839+0000","last_clean_scrub_stamp":"2024-09-19T11:26:41.382839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:55.560428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:56.798773+0000","last_change":"2024-09-19T11:26:56.798773+0000","last_active":"2024-09-19T11:26:56.798773+0000","last_peered":"2024-09-19T11:26:56.798773+0000","last_clean":"2024-09-19T11:26:56.798773+0000","last_became_active":"2024-09-19T11:23:07.247306+0000","last_became_peered":"2024-09-19T11:23:07.247306+0000","last_unstale":"2024-09-19T11:26:56.798773+0000","last_undegraded":"2024-09-19T11:26:56.798773+0000","last_fullsized":"2024-09-19T11:26:56.798773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:56.798674+0000","last_clean_scrub_stamp":"2024-09-19T11:26:56.798674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:05.764652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"33'38","reported_seq":94,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.766830+0000","last_change":"2024-09-19T11:27:02.766754+0000","last_active":"2024-09-19T11:27:02.766830+0000","last_peered":"2024-09-19T11:27:02.766830+0000","last_clean":"2024-09-19T11:27:02.766830+0000","last_became_active":"2024-09-19T11:23:09.257463+0000","last_became_peered":"2024-09-19T11:23:09.257463+0000","last_unstale":"2024-09-19T11:27:02.766830+0000","last_undegraded":"2024-09-19T11:27:02.766830+0000","last_fullsized":"2024-09-19T11:27:02.766830+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'38","last_scrub_stamp":"2024-09-19T11:27:02.766665+0000","last_deep_scrub":"33'38","last_deep_scrub_stamp":"2024-09-19T11:27:02.766665+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.766665+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:23.822170+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":31,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:36.641947+0000","last_change":"2024-09-19T11:27:36.641873+0000","last_active":"2024-09-19T11:27:36.641947+0000","last_peered":"2024-09-19T11:27:36.641947+0000","last_clean":"2024-09-19T11:27:36.641947+0000","last_became_active":"2024-09-19T11:23:11.271198+0000","last_became_peered":"2024-09-19T11:23:11.271198+0000","last_unstale":"2024-09-19T11:27:36.641947+0000","last_undegraded":"2024-09-19T11:27:36.641947+0000","last_fullsized":"2024-09-19T11:27:36.641947+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:36.641782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:36.641782+0000","last_clean_scrub_stamp":"2024-09-19T11:27:36.641782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:48.099365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":26,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:45.641034+0000","last_change":"2024-09-19T11:27:45.641034+0000","last_active":"2024-09-19T11:27:45.641034+0000","last_peered":"2024-09-19T11:27:45.641034+0000","last_clean":"2024-09-19T11:27:45.641034+0000","last_became_active":"2024-09-19T11:23:13.273416+0000","last_became_peered":"2024-09-19T11:23:13.273416+0000","last_unstale":"2024-09-19T11:27:45.641034+0000","last_undegraded":"2024-09-19T11:27:45.641034+0000","last_fullsized":"2024-09-19T11:27:45.641034+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:45.640937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:45.640937+0000","last_clean_scrub_stamp":"2024-09-19T11:27:45.640937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:12:28.210787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"33'4","reported_seq":27,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.728999+0000","last_change":"2024-09-19T11:26:45.728999+0000","last_active":"2024-09-19T11:26:45.728999+0000","last_peered":"2024-09-19T11:26:45.728999+0000","last_clean":"2024-09-19T11:26:45.728999+0000","last_became_active":"2024-09-19T11:23:42.281743+0000","last_became_peered":"2024-09-19T11:23:42.281743+0000","last_unstale":"2024-09-19T11:26:45.728999+0000","last_undegraded":"2024-09-19T11:26:45.728999+0000","last_fullsized":"2024-09-19T11:26:45.728999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.728900+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.728900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:26.277302+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:26.898920+0000","last_change":"2024-09-19T11:26:26.898841+0000","last_active":"2024-09-19T11:26:26.898920+0000","last_peered":"2024-09-19T11:26:26.898920+0000","last_clean":"2024-09-19T11:26:26.898920+0000","last_became_active":"2024-09-19T11:23:03.237008+0000","last_became_peered":"2024-09-19T11:23:03.237008+0000","last_unstale":"2024-09-19T11:26:26.898920+0000","last_undegraded":"2024-09-19T11:26:26.898920+0000","last_fullsized":"2024-09-19T11:26:26.898920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:26.898758+0000","last_clean_scrub_stamp":"2024-09-19T11:26:26.898758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:04.348903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:59.929697+0000","last_change":"2024-09-19T11:26:59.929630+0000","last_active":"2024-09-19T11:26:59.929697+0000","last_peered":"2024-09-19T11:26:59.929697+0000","last_clean":"2024-09-19T11:26:59.929697+0000","last_became_active":"2024-09-19T11:23:07.244782+0000","last_became_peered":"2024-09-19T11:23:07.244782+0000","last_unstale":"2024-09-19T11:26:59.929697+0000","last_undegraded":"2024-09-19T11:26:59.929697+0000","last_fullsized":"2024-09-19T11:26:59.929697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:59.929544+0000","last_clean_scrub_stamp":"2024-09-19T11:26:59.929544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:14.552058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"33'31","reported_seq":84,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:27.654733+0000","last_change":"2024-09-19T11:27:27.654733+0000","last_active":"2024-09-19T11:27:27.654733+0000","last_peered":"2024-09-19T11:27:27.654733+0000","last_clean":"2024-09-19T11:27:27.654733+0000","last_became_active":"2024-09-19T11:23:09.257172+0000","last_became_peered":"2024-09-19T11:23:09.257172+0000","last_unstale":"2024-09-19T11:27:27.654733+0000","last_undegraded":"2024-09-19T11:27:27.654733+0000","last_fullsized":"2024-09-19T11:27:27.654733+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'31","last_scrub_stamp":"2024-09-19T11:27:27.654639+0000","last_deep_scrub":"33'31","last_deep_scrub_stamp":"2024-09-19T11:27:27.654639+0000","last_clean_scrub_stamp":"2024-09-19T11:27:27.654639+0000","objects_scrubbed":2,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:43.311480+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:37.662963+0000","last_change":"2024-09-19T11:27:37.662963+0000","last_active":"2024-09-19T11:27:37.662963+0000","last_peered":"2024-09-19T11:27:37.662963+0000","last_clean":"2024-09-19T11:27:37.662963+0000","last_became_active":"2024-09-19T11:23:11.267555+0000","last_became_peered":"2024-09-19T11:23:11.267555+0000","last_unstale":"2024-09-19T11:27:37.662963+0000","last_undegraded":"2024-09-19T11:27:37.662963+0000","last_fullsized":"2024-09-19T11:27:37.662963+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:37.662870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:37.662870+0000","last_clean_scrub_stamp":"2024-09-19T11:27:37.662870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:11:05.648114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"33'4","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:47.561172+0000","last_change":"2024-09-19T11:27:47.561172+0000","last_active":"2024-09-19T11:27:47.561172+0000","last_peered":"2024-09-19T11:27:47.561172+0000","last_clean":"2024-09-19T11:27:47.561172+0000","last_became_active":"2024-09-19T11:23:13.274498+0000","last_became_peered":"2024-09-19T11:23:13.274498+0000","last_unstale":"2024-09-19T11:27:47.561172+0000","last_undegraded":"2024-09-19T11:27:47.561172+0000","last_fullsized":"2024-09-19T11:27:47.561172+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:47.561071+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:47.561071+0000","last_clean_scrub_stamp":"2024-09-19T11:27:47.561071+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:50:20.106927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"33'6","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.773266+0000","last_change":"2024-09-19T11:26:44.773207+0000","last_active":"2024-09-19T11:26:44.773266+0000","last_peered":"2024-09-19T11:26:44.773266+0000","last_clean":"2024-09-19T11:26:44.773266+0000","last_became_active":"2024-09-19T11:23:05.234441+0000","last_became_peered":"2024-09-19T11:23:05.234441+0000","last_unstale":"2024-09-19T11:26:44.773266+0000","last_undegraded":"2024-09-19T11:26:44.773266+0000","last_fullsized":"2024-09-19T11:26:44.773266+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.773116+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.773116+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:30.345346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"33'4","reported_seq":55,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.916369+0000","last_change":"2024-09-19T11:26:29.916292+0000","last_active":"2024-09-19T11:26:29.916369+0000","last_peered":"2024-09-19T11:26:29.916369+0000","last_clean":"2024-09-19T11:26:29.916369+0000","last_became_active":"2024-09-19T11:23:03.246772+0000","last_became_peered":"2024-09-19T11:23:03.246772+0000","last_unstale":"2024-09-19T11:26:29.916369+0000","last_undegraded":"2024-09-19T11:26:29.916369+0000","last_fullsized":"2024-09-19T11:26:29.916369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:29.916202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.916202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:50.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":7,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"33'3","reported_seq":42,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:01.924117+0000","last_change":"2024-09-19T11:27:01.924054+0000","last_active":"2024-09-19T11:27:01.924117+0000","last_peered":"2024-09-19T11:27:01.924117+0000","last_clean":"2024-09-19T11:27:01.924117+0000","last_became_active":"2024-09-19T11:23:07.247161+0000","last_became_peered":"2024-09-19T11:23:07.247161+0000","last_unstale":"2024-09-19T11:27:01.924117+0000","last_undegraded":"2024-09-19T11:27:01.924117+0000","last_fullsized":"2024-09-19T11:27:01.924117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'3","last_scrub_stamp":"2024-09-19T11:27:01.923971+0000","last_deep_scrub":"33'3","last_deep_scrub_stamp":"2024-09-19T11:27:01.923971+0000","last_clean_scrub_stamp":"2024-09-19T11:27:01.923971+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:26:44.012364+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"33'42","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:26.635927+0000","last_change":"2024-09-19T11:27:26.635927+0000","last_active":"2024-09-19T11:27:26.635927+0000","last_peered":"2024-09-19T11:27:26.635927+0000","last_clean":"2024-09-19T11:27:26.635927+0000","last_became_active":"2024-09-19T11:23:09.255261+0000","last_became_peered":"2024-09-19T11:23:09.255261+0000","last_unstale":"2024-09-19T11:27:26.635927+0000","last_undegraded":"2024-09-19T11:27:26.635927+0000","last_fullsized":"2024-09-19T11:27:26.635927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'42","last_scrub_stamp":"2024-09-19T11:27:26.635831+0000","last_deep_scrub":"33'42","last_deep_scrub_stamp":"2024-09-19T11:27:26.635831+0000","last_clean_scrub_stamp":"2024-09-19T11:27:26.635831+0000","objects_scrubbed":5,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:23.640501+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:15.837684+0000","last_change":"2024-09-19T11:27:15.837684+0000","last_active":"2024-09-19T11:27:15.837684+0000","last_peered":"2024-09-19T11:27:15.837684+0000","last_clean":"2024-09-19T11:27:15.837684+0000","last_became_active":"2024-09-19T11:23:11.265822+0000","last_became_peered":"2024-09-19T11:23:11.265822+0000","last_unstale":"2024-09-19T11:27:15.837684+0000","last_undegraded":"2024-09-19T11:27:15.837684+0000","last_fullsized":"2024-09-19T11:27:15.837684+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:27:15.837602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:27:15.837602+0000","last_clean_scrub_stamp":"2024-09-19T11:27:15.837602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:39.151539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"33'8","reported_seq":39,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:44.633073+0000","last_change":"2024-09-19T11:27:44.633003+0000","last_active":"2024-09-19T11:27:44.633073+0000","last_peered":"2024-09-19T11:27:44.633073+0000","last_clean":"2024-09-19T11:27:44.633073+0000","last_became_active":"2024-09-19T11:23:13.273437+0000","last_became_peered":"2024-09-19T11:23:13.273437+0000","last_unstale":"2024-09-19T11:27:44.633073+0000","last_undegraded":"2024-09-19T11:27:44.633073+0000","last_fullsized":"2024-09-19T11:27:44.633073+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:44.632915+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:44.632915+0000","last_clean_scrub_stamp":"2024-09-19T11:27:44.632915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:56.951566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":20,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"33'4","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.120490+0000","last_change":"2024-09-19T11:26:45.120385+0000","last_active":"2024-09-19T11:26:45.120490+0000","last_peered":"2024-09-19T11:26:45.120490+0000","last_clean":"2024-09-19T11:26:45.120490+0000","last_became_active":"2024-09-19T11:23:05.251911+0000","last_became_peered":"2024-09-19T11:23:05.251911+0000","last_unstale":"2024-09-19T11:26:45.120490+0000","last_undegraded":"2024-09-19T11:26:45.120490+0000","last_fullsized":"2024-09-19T11:26:45.120490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:45.120277+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.120277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:55.577999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.949943+0000","last_change":"2024-09-19T11:26:31.949825+0000","last_active":"2024-09-19T11:26:31.949943+0000","last_peered":"2024-09-19T11:26:31.949943+0000","last_clean":"2024-09-19T11:26:31.949943+0000","last_became_active":"2024-09-19T11:23:03.256846+0000","last_became_peered":"2024-09-19T11:23:03.256846+0000","last_unstale":"2024-09-19T11:26:31.949943+0000","last_undegraded":"2024-09-19T11:26:31.949943+0000","last_fullsized":"2024-09-19T11:26:31.949943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.949702+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.949702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:38.587259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"33'18","reported_seq":76,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.879892+0000","last_change":"2024-09-19T11:26:46.879819+0000","last_active":"2024-09-19T11:26:46.879892+0000","last_peered":"2024-09-19T11:26:46.879892+0000","last_clean":"2024-09-19T11:26:46.879892+0000","last_became_active":"2024-09-19T11:23:05.237947+0000","last_became_peered":"2024-09-19T11:23:05.237947+0000","last_unstale":"2024-09-19T11:26:46.879892+0000","last_undegraded":"2024-09-19T11:26:46.879892+0000","last_fullsized":"2024-09-19T11:26:46.879892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'18","last_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_deep_scrub":"33'18","last_deep_scrub_stamp":"2024-09-19T11:26:46.879729+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.879729+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:14.358902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":58,"num_write":23,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:27.905354+0000","last_change":"2024-09-19T11:26:27.905294+0000","last_active":"2024-09-19T11:26:27.905354+0000","last_peered":"2024-09-19T11:26:27.905354+0000","last_clean":"2024-09-19T11:26:27.905354+0000","last_became_active":"2024-09-19T11:23:03.237679+0000","last_became_peered":"2024-09-19T11:23:03.237679+0000","last_unstale":"2024-09-19T11:26:27.905354+0000","last_undegraded":"2024-09-19T11:26:27.905354+0000","last_fullsized":"2024-09-19T11:26:27.905354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:27.905202+0000","last_clean_scrub_stamp":"2024-09-19T11:26:27.905202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:43:38.666093+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:04.503655+0000","last_change":"2024-09-19T11:27:04.503595+0000","last_active":"2024-09-19T11:27:04.503655+0000","last_peered":"2024-09-19T11:27:04.503655+0000","last_clean":"2024-09-19T11:27:04.503655+0000","last_became_active":"2024-09-19T11:23:05.237082+0000","last_became_peered":"2024-09-19T11:23:05.237082+0000","last_unstale":"2024-09-19T11:27:04.503655+0000","last_undegraded":"2024-09-19T11:27:04.503655+0000","last_fullsized":"2024-09-19T11:27:04.503655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:27:04.503510+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:27:04.503510+0000","last_clean_scrub_stamp":"2024-09-19T11:27:04.503510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:01:27.895549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:42.388722+0000","last_change":"2024-09-19T11:26:42.388646+0000","last_active":"2024-09-19T11:26:42.388722+0000","last_peered":"2024-09-19T11:26:42.388722+0000","last_clean":"2024-09-19T11:26:42.388722+0000","last_became_active":"2024-09-19T11:23:03.256293+0000","last_became_peered":"2024-09-19T11:23:03.256293+0000","last_unstale":"2024-09-19T11:26:42.388722+0000","last_undegraded":"2024-09-19T11:26:42.388722+0000","last_fullsized":"2024-09-19T11:26:42.388722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:42.388560+0000","last_clean_scrub_stamp":"2024-09-19T11:26:42.388560+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:15:26.548057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"33'8","reported_seq":60,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.897022+0000","last_change":"2024-09-19T11:26:45.896965+0000","last_active":"2024-09-19T11:26:45.897022+0000","last_peered":"2024-09-19T11:26:45.897022+0000","last_clean":"2024-09-19T11:26:45.897022+0000","last_became_active":"2024-09-19T11:23:05.237307+0000","last_became_peered":"2024-09-19T11:23:05.237307+0000","last_unstale":"2024-09-19T11:26:45.897022+0000","last_undegraded":"2024-09-19T11:26:45.897022+0000","last_fullsized":"2024-09-19T11:26:45.897022+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:26:45.896872+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.896872+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:37.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.953844+0000","last_change":"2024-09-19T11:26:30.953783+0000","last_active":"2024-09-19T11:26:30.953844+0000","last_peered":"2024-09-19T11:26:30.953844+0000","last_clean":"2024-09-19T11:26:30.953844+0000","last_became_active":"2024-09-19T11:23:03.243025+0000","last_became_peered":"2024-09-19T11:23:03.243025+0000","last_unstale":"2024-09-19T11:26:30.953844+0000","last_undegraded":"2024-09-19T11:26:30.953844+0000","last_fullsized":"2024-09-19T11:26:30.953844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.953693+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.953693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:14:32.144901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"33'6","reported_seq":53,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.913054+0000","last_change":"2024-09-19T11:26:49.912991+0000","last_active":"2024-09-19T11:26:49.913054+0000","last_peered":"2024-09-19T11:26:49.913054+0000","last_clean":"2024-09-19T11:26:49.913054+0000","last_became_active":"2024-09-19T11:23:05.240001+0000","last_became_peered":"2024-09-19T11:23:05.240001+0000","last_unstale":"2024-09-19T11:26:49.913054+0000","last_undegraded":"2024-09-19T11:26:49.913054+0000","last_fullsized":"2024-09-19T11:26:49.913054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.912905+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.912905+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:20:07.844062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"33'7","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:29.952205+0000","last_change":"2024-09-19T11:26:29.952079+0000","last_active":"2024-09-19T11:26:29.952205+0000","last_peered":"2024-09-19T11:26:29.952205+0000","last_clean":"2024-09-19T11:26:29.952205+0000","last_became_active":"2024-09-19T11:23:03.233910+0000","last_became_peered":"2024-09-19T11:23:03.233910+0000","last_unstale":"2024-09-19T11:26:29.952205+0000","last_undegraded":"2024-09-19T11:26:29.952205+0000","last_fullsized":"2024-09-19T11:26:29.952205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'7","last_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_deep_scrub":"33'7","last_deep_scrub_stamp":"2024-09-19T11:26:29.951980+0000","last_clean_scrub_stamp":"2024-09-19T11:26:29.951980+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:27:30.180814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":16,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"33'6","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.104057+0000","last_change":"2024-09-19T11:26:44.103981+0000","last_active":"2024-09-19T11:26:44.104057+0000","last_peered":"2024-09-19T11:26:44.104057+0000","last_clean":"2024-09-19T11:26:44.104057+0000","last_became_active":"2024-09-19T11:23:05.251447+0000","last_became_peered":"2024-09-19T11:23:05.251447+0000","last_unstale":"2024-09-19T11:26:44.104057+0000","last_undegraded":"2024-09-19T11:26:44.104057+0000","last_fullsized":"2024-09-19T11:26:44.104057+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:44.103889+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.103889+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:22:14.482038+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:30.867345+0000","last_change":"2024-09-19T11:26:30.867275+0000","last_active":"2024-09-19T11:26:30.867345+0000","last_peered":"2024-09-19T11:26:30.867345+0000","last_clean":"2024-09-19T11:26:30.867345+0000","last_became_active":"2024-09-19T11:23:03.247834+0000","last_became_peered":"2024-09-19T11:23:03.247834+0000","last_unstale":"2024-09-19T11:26:30.867345+0000","last_undegraded":"2024-09-19T11:26:30.867345+0000","last_fullsized":"2024-09-19T11:26:30.867345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:30.867184+0000","last_clean_scrub_stamp":"2024-09-19T11:26:30.867184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:59:38.055814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":43,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.932867+0000","last_change":"2024-09-19T11:26:44.932807+0000","last_active":"2024-09-19T11:26:44.932867+0000","last_peered":"2024-09-19T11:26:44.932867+0000","last_clean":"2024-09-19T11:26:44.932867+0000","last_became_active":"2024-09-19T11:23:05.248041+0000","last_became_peered":"2024-09-19T11:23:05.248041+0000","last_unstale":"2024-09-19T11:26:44.932867+0000","last_undegraded":"2024-09-19T11:26:44.932867+0000","last_fullsized":"2024-09-19T11:26:44.932867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.932715+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.932715+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:07.921677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.978046+0000","last_change":"2024-09-19T11:26:32.977981+0000","last_active":"2024-09-19T11:26:32.978046+0000","last_peered":"2024-09-19T11:26:32.978046+0000","last_clean":"2024-09-19T11:26:32.978046+0000","last_became_active":"2024-09-19T11:23:03.257391+0000","last_became_peered":"2024-09-19T11:23:03.257391+0000","last_unstale":"2024-09-19T11:26:32.978046+0000","last_undegraded":"2024-09-19T11:26:32.978046+0000","last_fullsized":"2024-09-19T11:26:32.978046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:32.977891+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.977891+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:36.076149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"33'14","reported_seq":89,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.063998+0000","last_change":"2024-09-19T11:26:43.063925+0000","last_active":"2024-09-19T11:26:43.063998+0000","last_peered":"2024-09-19T11:26:43.063998+0000","last_clean":"2024-09-19T11:26:43.063998+0000","last_became_active":"2024-09-19T11:23:05.248340+0000","last_became_peered":"2024-09-19T11:23:05.248340+0000","last_unstale":"2024-09-19T11:26:43.063998+0000","last_undegraded":"2024-09-19T11:26:43.063998+0000","last_fullsized":"2024-09-19T11:26:43.063998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:43.063836+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.063836+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:23.821074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":61,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.990936+0000","last_change":"2024-09-19T11:26:31.990845+0000","last_active":"2024-09-19T11:26:31.990936+0000","last_peered":"2024-09-19T11:26:31.990936+0000","last_clean":"2024-09-19T11:26:31.990936+0000","last_became_active":"2024-09-19T11:23:03.238263+0000","last_became_peered":"2024-09-19T11:23:03.238263+0000","last_unstale":"2024-09-19T11:26:31.990936+0000","last_undegraded":"2024-09-19T11:26:31.990936+0000","last_fullsized":"2024-09-19T11:26:31.990936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:31.990752+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.990752+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:33.522127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:50.044237+0000","last_change":"2024-09-19T11:26:50.044177+0000","last_active":"2024-09-19T11:26:50.044237+0000","last_peered":"2024-09-19T11:26:50.044237+0000","last_clean":"2024-09-19T11:26:50.044237+0000","last_became_active":"2024-09-19T11:23:05.247998+0000","last_became_peered":"2024-09-19T11:23:05.247998+0000","last_unstale":"2024-09-19T11:26:50.044237+0000","last_undegraded":"2024-09-19T11:26:50.044237+0000","last_fullsized":"2024-09-19T11:26:50.044237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:50.044090+0000","last_clean_scrub_stamp":"2024-09-19T11:26:50.044090+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:50.741427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:43.359821+0000","last_change":"2024-09-19T11:26:43.359747+0000","last_active":"2024-09-19T11:26:43.359821+0000","last_peered":"2024-09-19T11:26:43.359821+0000","last_clean":"2024-09-19T11:26:43.359821+0000","last_became_active":"2024-09-19T11:23:03.254375+0000","last_became_peered":"2024-09-19T11:23:03.254375+0000","last_unstale":"2024-09-19T11:26:43.359821+0000","last_undegraded":"2024-09-19T11:26:43.359821+0000","last_fullsized":"2024-09-19T11:26:43.359821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:43.359650+0000","last_clean_scrub_stamp":"2024-09-19T11:26:43.359650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:26.024659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"33'2","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:03.528672+0000","last_change":"2024-09-19T11:27:03.528597+0000","last_active":"2024-09-19T11:27:03.528672+0000","last_peered":"2024-09-19T11:27:03.528672+0000","last_clean":"2024-09-19T11:27:03.528672+0000","last_became_active":"2024-09-19T11:23:05.243184+0000","last_became_peered":"2024-09-19T11:23:05.243184+0000","last_unstale":"2024-09-19T11:27:03.528672+0000","last_undegraded":"2024-09-19T11:27:03.528672+0000","last_fullsized":"2024-09-19T11:27:03.528672+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:27:03.528502+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:27:03.528502+0000","last_clean_scrub_stamp":"2024-09-19T11:27:03.528502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:34.145386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.017153+0000","last_change":"2024-09-19T11:26:33.017041+0000","last_active":"2024-09-19T11:26:33.017153+0000","last_peered":"2024-09-19T11:26:33.017153+0000","last_clean":"2024-09-19T11:26:33.017153+0000","last_became_active":"2024-09-19T11:23:03.242867+0000","last_became_peered":"2024-09-19T11:23:03.242867+0000","last_unstale":"2024-09-19T11:26:33.017153+0000","last_undegraded":"2024-09-19T11:26:33.017153+0000","last_fullsized":"2024-09-19T11:26:33.017153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.016950+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.016950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:19:06.549550+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.883583+0000","last_change":"2024-09-19T11:26:47.883512+0000","last_active":"2024-09-19T11:26:47.883583+0000","last_peered":"2024-09-19T11:26:47.883583+0000","last_clean":"2024-09-19T11:26:47.883583+0000","last_became_active":"2024-09-19T11:23:05.241045+0000","last_became_peered":"2024-09-19T11:23:05.241045+0000","last_unstale":"2024-09-19T11:26:47.883583+0000","last_undegraded":"2024-09-19T11:26:47.883583+0000","last_fullsized":"2024-09-19T11:26:47.883583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.883414+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.883414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:39.095013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"33'4","reported_seq":59,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:31.853715+0000","last_change":"2024-09-19T11:26:31.853602+0000","last_active":"2024-09-19T11:26:31.853715+0000","last_peered":"2024-09-19T11:26:31.853715+0000","last_clean":"2024-09-19T11:26:31.853715+0000","last_became_active":"2024-09-19T11:23:03.246997+0000","last_became_peered":"2024-09-19T11:23:03.246997+0000","last_unstale":"2024-09-19T11:26:31.853715+0000","last_undegraded":"2024-09-19T11:26:31.853715+0000","last_fullsized":"2024-09-19T11:26:31.853715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:31.853479+0000","last_clean_scrub_stamp":"2024-09-19T11:26:31.853479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T21:16:07.826853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"33'6","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:49.073256+0000","last_change":"2024-09-19T11:26:49.073137+0000","last_active":"2024-09-19T11:26:49.073256+0000","last_peered":"2024-09-19T11:26:49.073256+0000","last_clean":"2024-09-19T11:26:49.073256+0000","last_became_active":"2024-09-19T11:23:05.246384+0000","last_became_peered":"2024-09-19T11:23:05.246384+0000","last_unstale":"2024-09-19T11:26:49.073256+0000","last_undegraded":"2024-09-19T11:26:49.073256+0000","last_fullsized":"2024-09-19T11:26:49.073256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'6","last_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_deep_scrub":"33'6","last_deep_scrub_stamp":"2024-09-19T11:26:49.073043+0000","last_clean_scrub_stamp":"2024-09-19T11:26:49.073043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:26.833881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:32.886986+0000","last_change":"2024-09-19T11:26:32.886892+0000","last_active":"2024-09-19T11:26:32.886986+0000","last_peered":"2024-09-19T11:26:32.886986+0000","last_clean":"2024-09-19T11:26:32.886986+0000","last_became_active":"2024-09-19T11:23:03.243156+0000","last_became_peered":"2024-09-19T11:23:03.243156+0000","last_unstale":"2024-09-19T11:26:32.886986+0000","last_undegraded":"2024-09-19T11:26:32.886986+0000","last_fullsized":"2024-09-19T11:26:32.886986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:32.886787+0000","last_clean_scrub_stamp":"2024-09-19T11:26:32.886787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:19.114834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"33'4","reported_seq":67,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.038331+0000","last_change":"2024-09-19T11:26:48.038260+0000","last_active":"2024-09-19T11:26:48.038331+0000","last_peered":"2024-09-19T11:26:48.038331+0000","last_clean":"2024-09-19T11:26:48.038331+0000","last_became_active":"2024-09-19T11:23:05.245455+0000","last_became_peered":"2024-09-19T11:23:05.245455+0000","last_unstale":"2024-09-19T11:26:48.038331+0000","last_undegraded":"2024-09-19T11:26:48.038331+0000","last_fullsized":"2024-09-19T11:26:48.038331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'4","last_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_deep_scrub":"33'4","last_deep_scrub_stamp":"2024-09-19T11:26:48.038149+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.038149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:16.505179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:44.342771+0000","last_change":"2024-09-19T11:26:44.342671+0000","last_active":"2024-09-19T11:26:44.342771+0000","last_peered":"2024-09-19T11:26:44.342771+0000","last_clean":"2024-09-19T11:26:44.342771+0000","last_became_active":"2024-09-19T11:23:03.241025+0000","last_became_peered":"2024-09-19T11:23:03.241025+0000","last_unstale":"2024-09-19T11:26:44.342771+0000","last_undegraded":"2024-09-19T11:26:44.342771+0000","last_fullsized":"2024-09-19T11:26:44.342771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:44.342574+0000","last_clean_scrub_stamp":"2024-09-19T11:26:44.342574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:43:38.309444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"33'14","reported_seq":75,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.082768+0000","last_change":"2024-09-19T11:26:47.082623+0000","last_active":"2024-09-19T11:26:47.082768+0000","last_peered":"2024-09-19T11:26:47.082768+0000","last_clean":"2024-09-19T11:26:47.082768+0000","last_became_active":"2024-09-19T11:23:05.247655+0000","last_became_peered":"2024-09-19T11:23:05.247655+0000","last_unstale":"2024-09-19T11:26:47.082768+0000","last_undegraded":"2024-09-19T11:26:47.082768+0000","last_fullsized":"2024-09-19T11:26:47.082768+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:47.082498+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.082498+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:22.862275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":41,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.395348+0000","last_change":"2024-09-19T11:26:46.395278+0000","last_active":"2024-09-19T11:26:46.395348+0000","last_peered":"2024-09-19T11:26:46.395348+0000","last_clean":"2024-09-19T11:26:46.395348+0000","last_became_active":"2024-09-19T11:23:03.254995+0000","last_became_peered":"2024-09-19T11:23:03.254995+0000","last_unstale":"2024-09-19T11:26:46.395348+0000","last_undegraded":"2024-09-19T11:26:46.395348+0000","last_fullsized":"2024-09-19T11:26:46.395348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:46.395181+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.395181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T23:17:36.015668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"33'8","reported_seq":69,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:27:02.516399+0000","last_change":"2024-09-19T11:27:02.516337+0000","last_active":"2024-09-19T11:27:02.516399+0000","last_peered":"2024-09-19T11:27:02.516399+0000","last_clean":"2024-09-19T11:27:02.516399+0000","last_became_active":"2024-09-19T11:23:05.250250+0000","last_became_peered":"2024-09-19T11:23:05.250250+0000","last_unstale":"2024-09-19T11:27:02.516399+0000","last_undegraded":"2024-09-19T11:27:02.516399+0000","last_fullsized":"2024-09-19T11:27:02.516399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'8","last_scrub_stamp":"2024-09-19T11:27:02.516249+0000","last_deep_scrub":"33'8","last_deep_scrub_stamp":"2024-09-19T11:27:02.516249+0000","last_clean_scrub_stamp":"2024-09-19T11:27:02.516249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:11.616188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"33'2","reported_seq":51,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.979209+0000","last_change":"2024-09-19T11:26:33.979119+0000","last_active":"2024-09-19T11:26:33.979209+0000","last_peered":"2024-09-19T11:26:33.979209+0000","last_clean":"2024-09-19T11:26:33.979209+0000","last_became_active":"2024-09-19T11:23:03.258368+0000","last_became_peered":"2024-09-19T11:23:03.258368+0000","last_unstale":"2024-09-19T11:26:33.979209+0000","last_undegraded":"2024-09-19T11:26:33.979209+0000","last_fullsized":"2024-09-19T11:26:33.979209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:33.979000+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.979000+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:24.446915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"33'26","reported_seq":87,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:46.075697+0000","last_change":"2024-09-19T11:26:46.075580+0000","last_active":"2024-09-19T11:26:46.075697+0000","last_peered":"2024-09-19T11:26:46.075697+0000","last_clean":"2024-09-19T11:26:46.075697+0000","last_became_active":"2024-09-19T11:23:05.247891+0000","last_became_peered":"2024-09-19T11:23:05.247891+0000","last_unstale":"2024-09-19T11:26:46.075697+0000","last_undegraded":"2024-09-19T11:26:46.075697+0000","last_fullsized":"2024-09-19T11:26:46.075697+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'26","last_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_deep_scrub":"33'26","last_deep_scrub_stamp":"2024-09-19T11:26:46.075475+0000","last_clean_scrub_stamp":"2024-09-19T11:26:46.075475+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:51.242922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":78,"num_write":34,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:45.388213+0000","last_change":"2024-09-19T11:26:45.388148+0000","last_active":"2024-09-19T11:26:45.388213+0000","last_peered":"2024-09-19T11:26:45.388213+0000","last_clean":"2024-09-19T11:26:45.388213+0000","last_became_active":"2024-09-19T11:23:03.252316+0000","last_became_peered":"2024-09-19T11:23:03.252316+0000","last_unstale":"2024-09-19T11:26:45.388213+0000","last_undegraded":"2024-09-19T11:26:45.388213+0000","last_fullsized":"2024-09-19T11:26:45.388213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:45.388045+0000","last_clean_scrub_stamp":"2024-09-19T11:26:45.388045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:47.891066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"33'2","reported_seq":50,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:47.428247+0000","last_change":"2024-09-19T11:26:47.428183+0000","last_active":"2024-09-19T11:26:47.428247+0000","last_peered":"2024-09-19T11:26:47.428247+0000","last_clean":"2024-09-19T11:26:47.428247+0000","last_became_active":"2024-09-19T11:23:03.254467+0000","last_became_peered":"2024-09-19T11:23:03.254467+0000","last_unstale":"2024-09-19T11:26:47.428247+0000","last_undegraded":"2024-09-19T11:26:47.428247+0000","last_fullsized":"2024-09-19T11:26:47.428247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'2","last_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_deep_scrub":"33'2","last_deep_scrub_stamp":"2024-09-19T11:26:47.428079+0000","last_clean_scrub_stamp":"2024-09-19T11:26:47.428079+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:59:02.198551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"33'14","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:48.933106+0000","last_change":"2024-09-19T11:26:48.933041+0000","last_active":"2024-09-19T11:26:48.933106+0000","last_peered":"2024-09-19T11:26:48.933106+0000","last_clean":"2024-09-19T11:26:48.933106+0000","last_became_active":"2024-09-19T11:23:05.248002+0000","last_became_peered":"2024-09-19T11:23:05.248002+0000","last_unstale":"2024-09-19T11:26:48.933106+0000","last_undegraded":"2024-09-19T11:26:48.933106+0000","last_fullsized":"2024-09-19T11:26:48.933106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'14","last_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_deep_scrub":"33'14","last_deep_scrub_stamp":"2024-09-19T11:26:48.932953+0000","last_clean_scrub_stamp":"2024-09-19T11:26:48.932953+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:16.178277+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":32,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":48,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:33.990768+0000","last_change":"2024-09-19T11:26:33.990707+0000","last_active":"2024-09-19T11:26:33.990768+0000","last_peered":"2024-09-19T11:26:33.990768+0000","last_clean":"2024-09-19T11:26:33.990768+0000","last_became_active":"2024-09-19T11:23:03.243226+0000","last_became_peered":"2024-09-19T11:23:03.243226+0000","last_unstale":"2024-09-19T11:26:33.990768+0000","last_undegraded":"2024-09-19T11:26:33.990768+0000","last_fullsized":"2024-09-19T11:26:33.990768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T11:26:33.990617+0000","last_clean_scrub_stamp":"2024-09-19T11:26:33.990617+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:35.040214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"33'12","reported_seq":61,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-19T11:26:54.051819+0000","last_change":"2024-09-19T11:26:54.051717+0000","last_active":"2024-09-19T11:26:54.051819+0000","last_peered":"2024-09-19T11:26:54.051819+0000","last_clean":"2024-09-19T11:26:54.051819+0000","last_became_active":"2024-09-19T11:23:05.245409+0000","last_became_peered":"2024-09-19T11:23:05.245409+0000","last_unstale":"2024-09-19T11:26:54.051819+0000","last_undegraded":"2024-09-19T11:26:54.051819+0000","last_fullsized":"2024-09-19T11:26:54.051819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'12","last_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_deep_scrub":"33'12","last_deep_scrub_stamp":"2024-09-19T11:26:54.051625+0000","last_clean_scrub_stamp":"2024-09-19T11:26:54.051625+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:56:27.912591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":22,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1549,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":690,"num_read_kb":658,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2288,"ondisk_log_size":2288,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1557,"ondisk_log_size":1557,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1672,"num_read_kb":1562,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":479,"ondisk_log_size":479,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":189,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":192,"acting":192,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673026,"num_pgs":167,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":479232,"data_stored":150572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673026,"num_pgs":136,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":737565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705731,"num_pgs":140,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":739071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705730,"num_pgs":151,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":442368,"data_stored":148560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T11:28:05.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-19T11:28:05.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.803+0000 7fb3fbfff640 1 Processor -- start 2024-09-19T11:28:05.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.803+0000 7fb3fbfff640 1 -- start start 2024-09-19T11:28:05.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.803+0000 7fb3fbfff640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb3fc12c9b0 0x7fb3fc131450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:28:05.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.803+0000 7fb3fbfff640 1 -- --> v1:172.21.15.22:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3fc06cc90 con 0x7fb3fc12bec0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.803+0000 7fb3fbfff640 1 -- --> v2:172.21.15.22:3300/0 -- mon_getmap magic: 0 -- 0x7fb3fc09d0d0 con 0x7fb3fc12c9b0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.804+0000 7fb3fa7fc640 1 --2- >> v2:172.21.15.22:3300/0 conn(0x7fb3fc12c9b0 0x7fb3fc131450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.804+0000 7fb3faffd640 1 --1- >> v1:172.21.15.22:6789/0 conn(0x7fb3fc12bec0 0x7fb3fc12c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.22:6789/0 says I am v1:172.21.15.22:36856/0 (socket says 172.21.15.22:36856) 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.804+0000 7fb3faffd640 1 -- 172.21.15.22:0/1354983744 learned_addr learned my addr 172.21.15.22:0/1354983744 (peer_addr_for_me v1:172.21.15.22:0/0) 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.804+0000 7fb3fa7fc640 1 -- 172.21.15.22:0/1354983744 >> v1:172.21.15.22:6789/0 conn(0x7fb3fc12bec0 legacy=0x7fb3fc12c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.804+0000 7fb3fa7fc640 1 -- 172.21.15.22:0/1354983744 --> v2:172.21.15.22:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc06f090 con 0x7fb3fc12c9b0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fa7fc640 1 --2- 172.21.15.22:0/1354983744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12c9b0 0x7fb3fc131450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009720 tx=0x7fb3f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a30aba899e986c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3f9ffb640 1 -- 172.21.15.22:0/1354983744 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f0037070 con 0x7fb3fc12c9b0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3f9ffb640 1 -- 172.21.15.22:0/1354983744 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f002ad90 con 0x7fb3fc12c9b0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3f9ffb640 1 -- 172.21.15.22:0/1354983744 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f0032590 con 0x7fb3fc12c9b0 2024-09-19T11:28:05.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fbfff640 1 -- 172.21.15.22:0/1354983744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12c9b0 msgr2=0x7fb3fc131450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:05.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fbfff640 1 --2- 172.21.15.22:0/1354983744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12c9b0 0x7fb3fc131450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009720 tx=0x7fb3f002a110 comp rx=0 tx=0).stop 2024-09-19T11:28:05.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fbfff640 1 -- 172.21.15.22:0/1354983744 shutdown_connections 2024-09-19T11:28:05.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fbfff640 1 --2- 172.21.15.22:0/1354983744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12c9b0 0x7fb3fc131450 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:28:05.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fbfff640 1 -- 172.21.15.22:0/1354983744 >> 172.21.15.22:0/1354983744 conn(0x7fb3fc09a970 msgr2=0x7fb3fc09b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:28:05.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fbfff640 1 -- 172.21.15.22:0/1354983744 shutdown_connections 2024-09-19T11:28:05.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.805+0000 7fb3fbfff640 1 -- 172.21.15.22:0/1354983744 wait complete. 2024-09-19T11:28:05.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.806+0000 7fb3fbfff640 1 Processor -- start 2024-09-19T11:28:05.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3fbfff640 1 -- start start 2024-09-19T11:28:05.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3fbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12bec0 0x7fb3fc126940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3fbfff640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc131990 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3faffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12bec0 0x7fb3fc126940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3faffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12bec0 0x7fb3fc126940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:39274/0 (socket says 172.21.15.22:39274) 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3faffd640 1 -- 172.21.15.22:0/256105231 learned_addr learned my addr 172.21.15.22:0/256105231 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3faffd640 1 -- 172.21.15.22:0/256105231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc128ea0 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.807+0000 7fb3faffd640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12bec0 0x7fb3fc126940 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c450 tx=0x7fb3ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.808+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3ec017070 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.808+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec005c10 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.808+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3fc128b90 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.808+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3ec0070d0 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.808+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3fc129140 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.809+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 51031+0+0 (secure 0 0 0) 0x7fb3ec01a070 con 0x7fb3fc12bec0 2024-09-19T11:28:05.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.809+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3fc12c2a0 con 0x7fb3fc12bec0 2024-09-19T11:28:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.809+0000 7fb3e3fff640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb3e803e6c0 0x7fb3e8040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T11:28:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.809+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6313+0+0 (secure 0 0 0) 0x7fb3ec052b80 con 0x7fb3fc12bec0 2024-09-19T11:28:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.809+0000 7fb3fa7fc640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb3e803e6c0 0x7fb3e8040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T11:28:05.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.810+0000 7fb3fa7fc640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb3e803e6c0 0x7fb3e8040b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0034000 tx=0x7fb3f002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T11:28:05.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.813+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196984 (secure 0 0 0) 0x7fb3ec01bdd0 con 0x7fb3fc12bec0 2024-09-19T11:28:06.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:05.999+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fb3fc050ee0 con 0x7fb3fc12bec0 2024-09-19T11:28:06.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.001+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7fb3ec01a880 con 0x7fb3fc12bec0 2024-09-19T11:28:06.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.001+0000 7fb3e3fff640 1 -- 172.21.15.22:0/256105231 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb3ec059080 con 0x7fb3fc12bec0 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb3e803e6c0 msgr2=0x7fb3e8040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb3e803e6c0 0x7fb3e8040b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0034000 tx=0x7fb3f002e000 comp rx=0 tx=0).stop 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12bec0 msgr2=0x7fb3fc126940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12bec0 0x7fb3fc126940 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c450 tx=0x7fb3ec00c910 comp rx=0 tx=0).stop 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 shutdown_connections 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.28:6824/2061368145,v1:172.21.15.28:6825/2061368145] conn(0x7fb3e803e6c0 0x7fb3e8040b80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 --2- 172.21.15.22:0/256105231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3fc12bec0 0x7fb3fc126940 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 >> 172.21.15.22:0/256105231 conn(0x7fb3fc09a970 msgr2=0x7fb3fc099100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T11:28:06.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 shutdown_connections 2024-09-19T11:28:06.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-19T11:28:06.003+0000 7fb3fbfff640 1 -- 172.21.15.22:0/256105231 wait complete. 2024-09-19T11:28:06.015 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-19T11:28:06.016 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-19T11:28:06.016 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-19T11:28:06.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T11:28:06.045 INFO:tasks.ceph.osd.0:Stopped 2024-09-19T11:28:06.046 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-19T11:28:06.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T11:28:06.085 INFO:tasks.ceph.osd.1:Stopped 2024-09-19T11:28:06.085 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-19T11:28:06.085 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T11:28:06.155 INFO:tasks.ceph.osd.2:Stopped 2024-09-19T11:28:06.155 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-19T11:28:06.155 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T11:28:06.228 INFO:tasks.ceph.osd.3:Stopped 2024-09-19T11:28:06.228 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-19T11:28:06.228 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-19T11:28:06.228 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T11:28:06.299 INFO:tasks.ceph.mgr.x:Stopped 2024-09-19T11:28:06.299 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-19T11:28:06.299 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-19T11:28:06.299 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T11:28:06.311 INFO:tasks.ceph.mon.a:Stopped 2024-09-19T11:28:06.311 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-19T11:28:06.311 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-09-19T11:28:06.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T11:28:06.382 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-19T11:28:06.544 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:28:06.545 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-19T11:28:06.690 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:28:06.690 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-19T11:28:06.754 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:28:06.754 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-19T11:28:06.794 INFO:tasks.ceph:Archiving mon data... 2024-09-19T11:28:06.795 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434/data/mon.a.tgz 2024-09-19T11:28:06.796 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-09-19T11:28:06.813 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.zkwSWFUgMI 2024-09-19T11:28:06.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zkwSWFUgMI 2024-09-19T11:28:06.955 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.zkwSWFUgMI 2024-09-19T11:28:07.042 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.zkwSWFUgMI is 281KB 2024-09-19T11:28:07.136 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.zkwSWFUgMI 2024-09-19T11:28:07.152 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-19T11:28:07.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T11:28:07.193 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T11:28:07.273 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T11:28:07.293 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T11:28:07.293 INFO:tasks.ceph:Archiving crash dumps... 2024-09-19T11:28:07.294 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/crash to /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434/remote/smithi022/crash 2024-09-19T11:28:07.295 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T11:28:07.338 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/crash to /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434/remote/smithi028/crash 2024-09-19T11:28:07.339 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T11:28:07.379 INFO:tasks.ceph:Compressing logs... 2024-09-19T11:28:07.379 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T11:28:07.382 DEBUG:teuthology.orchestra.run.smithi028:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31234.log 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30138.log 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31234.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27271.log 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31234.log.gz 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30441.log 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.30138.log.gz 2024-09-19T11:28:07.423 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30442.log 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.tmp-client.admin.27271.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27271.log.gz 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31024.log 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30441.log.gz 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31276.log 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.30442.log.gz 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31192.log 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31276.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31024.log.gz 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31740.log 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.31276.log.gz 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30298.log 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.31192.log.gz 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30440.log 2024-09-19T11:28:07.424 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30298.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31740.log.gz 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30610.log 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.30298.log.gz 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30340.log 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.30440.log.gz 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30439.log 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30340.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.30610.log.gz 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30340.log.gz 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31528.log 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.30439.log.gz 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31108.log 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30778.log 2024-09-19T11:28:07.425 INFO:teuthology.orchestra.run.smithi022.stderr: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.31528.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31780.log 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.31108.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31360.log 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30778.log: /var/log/ceph/ceph-client.admin.31780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30570.log 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.30778.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31780.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30970.log 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30570.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.31360.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30082.log 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.30570.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31697.log 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.30970.log.gz 2024-09-19T11:28:07.426 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30569.log 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31697.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30082.log.gz 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29135.log 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31697.log.gz 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31318.log 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30569.log: /var/log/ceph/ceph-client.admin.29135.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.30569.log.gz 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr: 84.9%/var/log/ceph/ceph-client.admin.31318.log: -- replaced with /var/log/ceph/ceph-client.admin.29135.log.gz 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31402.log 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31318.log.gz 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31066.log 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31150.log 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31066.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31402.log.gz 2024-09-19T11:28:07.427 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31655.log 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31066.log.gz 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31444.log 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31150.log: /var/log/ceph/ceph-client.admin.31655.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.31150.log.gz 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31444.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.31655.log.gz 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30025.log 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.31444.log.gz 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30163.log 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.30025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31612.log 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30163.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30025.log.gz 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29097.log 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.30163.log.gz 2024-09-19T11:28:07.428 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31570.log 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.31612.log: /var/log/ceph/ceph-client.admin.29097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30737.log 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.31612.log.gz 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 85.1%/var/log/ceph/ceph-client.admin.31570.log: -- replaced with /var/log/ceph/ceph-client.admin.29097.log.gz 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28979.log 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.31570.log.gz 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30928.log 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.30737.log.gz 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.28979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31486.log 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.28979.log.gz 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30256.log 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30928.log: /var/log/ceph/ceph-client.admin.31486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30886.log 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.30928.log.gz 2024-09-19T11:28:07.429 INFO:teuthology.orchestra.run.smithi022.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31486.log.gz 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30777.log 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30256.log.gz 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30214.log 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30886.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.30886.log.gz 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30609.log 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.30777.log.gz 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.30214.log: /var/log/ceph/ceph-client.admin.30738.log 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30609.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30214.log.gz 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.30609.log.gz 2024-09-19T11:28:07.430 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.30738.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.30738.log.gz 2024-09-19T11:28:07.460 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82785.log 2024-09-19T11:28:07.460 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44670.log 2024-09-19T11:28:07.461 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31339.log 2024-09-19T11:28:07.461 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.82785.log: /var/log/ceph/ceph-client.0.44670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67055.log 2024-09-19T11:28:07.462 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.31339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93645.log 2024-09-19T11:28:07.462 INFO:teuthology.orchestra.run.smithi028.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.31339.log.gz 2024-09-19T11:28:07.462 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73710.log 2024-09-19T11:28:07.462 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.67055.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.44670.log.gz 2024-09-19T11:28:07.462 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.93645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64635.log 2024-09-19T11:28:07.465 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.73710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66450.log 2024-09-19T11:28:07.465 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.64635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74920.log 2024-09-19T11:28:07.465 INFO:teuthology.orchestra.run.smithi028.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.82785.log.gz 2024-09-19T11:28:07.465 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.66450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34687.log 2024-09-19T11:28:07.465 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.67055.log.gz 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.93645.log.gz 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.74920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63425.log 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.73710.log.gz 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.34687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45880.log 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.34687.log.gz 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.63425.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.64635.log.gz 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.66450.log.gz 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70685.log 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.45880.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.74920.log.gz 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36811.log 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.70685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80970.log 2024-09-19T11:28:07.466 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.36811.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-19T11:28:07.500 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.63425.log.gz 2024-09-19T11:28:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.80970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77340.log 2024-09-19T11:28:07.500 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% 91.0% -- replaced with /var/log/ceph/ceph-client.0.45880.log.gz 2024-09-19T11:28:07.500 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.70685.log.gz 2024-09-19T11:28:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2024-09-19T11:28:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.0.77340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90620.log 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92435.log 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.36811.log.gz 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.90620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41645.log 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr: 91.8% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.0.92435.log: -5 --verbose -- /var/log/ceph/ceph-client.0.36206.log 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.77340.log.gz 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.80970.log.gz 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.41645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59795.log 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.36206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56165.log 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.90620.log.gz 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.59795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38626.log 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.92435.log.gz 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.41645.log.gz 2024-09-19T11:28:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.56165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44065.log 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.38626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54350.log 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.44065.log: 90.2% -- replaced with /var/log/ceph/ceph-client.0.38626.log.gz 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.36206.log.gz 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56770.log 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.59795.log.gz 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.54350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71290.log 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.56165.log.gz 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.56770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43460.log 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.44065.log.gz 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.71290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41040.log 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.43460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34647.log 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.54350.log.gz 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.41040.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.56770.log.gz 2024-09-19T11:28:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53745.log 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.34647.log: -5 --verbose -- /var/log/ceph/ceph-client.0.94855.log 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.53745.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.34647.log.gz 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.43460.log.gz 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71895.log 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.41040.log.gz 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.71290.log.gz 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91225.log 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.94855.log: /var/log/ceph/ceph-client.0.71895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39225.log 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.91225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40435.log 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr: 91.9%/var/log/ceph/ceph-client.0.39225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91830.log 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.53745.log.gz 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.94855.log.gz 2024-09-19T11:28:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.40435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35601.log 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.71895.log.gz 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.91830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51930.log 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.91225.log.gz 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.35601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95512.log 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.0.39225.log.gz 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.51930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34727.log 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.40435.log.gz 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.95512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62820.log 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.91830.log.gz 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.34727.log: 92.0% -- replaced with /var/log/ceph/ceph-client.0.35601.log.gz 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54955.log 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.51930.log.gz 2024-09-19T11:28:07.504 INFO:teuthology.orchestra.run.smithi028.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.34727.log.gz 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.62820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53140.log 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.54955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77945.log 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.53140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65240.log 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.77945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59190.log 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.62820.log.gz 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.65240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86990.log 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.54955.log.gz 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.53140.log.gz 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50115.log 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.59190.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.95512.log.gz 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.86990.log: 91.1%gzip -- replaced with /var/log/ceph/ceph-client.0.77945.log.gz -5 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.0.69475.log 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.65240.log.gz 2024-09-19T11:28:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.50115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88200.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78550.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.69475.log: /var/log/ceph/ceph-client.0.88200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75525.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1%/var/log/ceph/ceph-client.0.78550.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.59190.log.gz 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.86990.log.gz 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94250.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.50115.log.gz 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.75525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31299.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.0.94250.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34807.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% 91.2% -- replaced with /var/log/ceph/ceph-client.0.88200.log.gz 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.69475.log.gz 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.31299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67660.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.31299.log.gz 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.34807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51325.log 2024-09-19T11:28:07.506 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.78550.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.75525.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.34807.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.67660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79760.log 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.51325.log: gzip -5 --verbose -- 91.0% /var/log/ceph/ceph-client.0.76130.log 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.94250.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.79760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84594.log 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.76130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88805.log 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.84594.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.51325.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31379.log 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.79760.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.0.88805.log: -5 --verbose -- 92.5% -- replaced with /var/log/ceph/ceph-client.0.67660.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.0.46485.log 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.84594.log.gz 2024-09-19T11:28:07.507 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.76130.log.gz 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.31379.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.31379.log.gz 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50720.log 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.46485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73105.log 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.50720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83995.log 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.73105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68265.log 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.88805.log.gz 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.83995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42855.log 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.46485.log.gz 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.68265.log: gzip 91.0% -- replaced with /var/log/ceph/ceph-client.0.50720.log.gz 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose 90.6% -- replaced with /var/log/ceph/ceph-client.0.83995.log.gz 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-osd.2.log 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.42855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55560.log 2024-09-19T11:28:07.508 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.73105.log.gz 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60400.log 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57375.log 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.55560.log: /var/log/ceph/ceph-client.0.60400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34767.log 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.42855.log.gz 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.57375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48300.log 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38021.log 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.55560.log.gz/var/log/ceph/ceph-client.0.48300.log: 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.34767.log: 93.3% -- replaced with /var/log/ceph/ceph-client.0.68265.log.gz 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.60400.log.gz 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48905.log 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.34767.log.gz 91.0% 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.57375.log.gz 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.38021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90015.log 2024-09-19T11:28:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.48905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64030.log 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.90015.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.48300.log.gz 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83390.log 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.64030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89410.log 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.83390.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.48905.log.gz 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.38021.log.gz 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80365.log 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.89410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70080.log 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.80365.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.90015.log.gz 2024-09-19T11:28:07.510 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.64030.log.gz 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.83390.log.gz 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.70080.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.89410.log.gz 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62215.log 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61610.log 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.70080.log.gz 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.62215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49510.log 2024-09-19T11:28:07.511 INFO:teuthology.orchestra.run.smithi028.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.80365.log.gz 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.61610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79155.log 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.49510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85193.log 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.79155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93040.log 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% 91.1% -- replaced with /var/log/ceph/ceph-client.0.61610.log.gz 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.62215.log.gz 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.85193.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.49510.log.gz 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37416.log 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.0.85193.log.gz 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.93040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86391.log 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.79155.log.gz 2024-09-19T11:28:07.512 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.37416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57980.log 2024-09-19T11:28:07.524 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.86391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42250.log 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.57980.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.93040.log.gz 90.5% -- replaced with /var/log/ceph/ceph-client.0.86391.log.gz 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.42250.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.37416.log.gz 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.57980.log.gz 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47695.log 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82180.log 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/rgw.ceph.client.0.log: /var/log/ceph/ceph-client.0.47695.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.42250.log.gz 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47090.log 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.82180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31259.log 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.47090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68870.log 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.47695.log.gz 2024-09-19T11:28:07.525 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.31259.log: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76735.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.0.82180.log.gz 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.68870.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.31259.log.gz 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.47090.log.gz 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74315.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.76735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58585.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.74315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87595.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.58585.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.76735.log.gz 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.68870.log.gz 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61005.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.87595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72500.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.74315.log.gz 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.61005.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.58585.log.gz 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85792.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.87595.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.81575.log 2024-09-19T11:28:07.526 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.72500.log: /var/log/ceph/ceph-client.0.85792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39830.log 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.61005.log.gz 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.81575.log: 90.2% -- replaced with /var/log/ceph/ceph-client.0.85792.log.gz 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.39830.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.72500.log.gz 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.81575.log.gz 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65845.log 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.39830.log.gz 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45275.log 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.0.65845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52535.log 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.45275.log: /var/log/ceph/ceph-client.0.52535.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.65845.log.gz 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.45275.log.gz 2024-09-19T11:28:07.527 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.52535.log.gz 2024-09-19T11:28:07.559 INFO:teuthology.orchestra.run.smithi028.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-19T11:28:07.559 INFO:teuthology.orchestra.run.smithi028.stderr: 92.0% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-09-19T11:28:07.595 INFO:teuthology.orchestra.run.smithi022.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-19T11:28:08.331 INFO:teuthology.orchestra.run.smithi028.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-19T11:28:08.396 INFO:teuthology.orchestra.run.smithi022.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-19T11:28:08.397 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-19T11:28:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m1.002s 2024-09-19T11:28:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m1.157s 2024-09-19T11:28:08.398 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.082s 2024-09-19T11:28:08.470 INFO:teuthology.orchestra.run.smithi028.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-19T11:28:08.522 INFO:teuthology.orchestra.run.smithi028.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-19T11:28:08.523 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-19T11:28:08.523 INFO:teuthology.orchestra.run.smithi028.stderr:real 0m1.076s 2024-09-19T11:28:08.523 INFO:teuthology.orchestra.run.smithi028.stderr:user 0m3.022s 2024-09-19T11:28:08.524 INFO:teuthology.orchestra.run.smithi028.stderr:sys 0m0.222s 2024-09-19T11:28:08.524 INFO:tasks.ceph:Archiving logs... 2024-09-19T11:28:08.524 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434/remote/smithi022/log 2024-09-19T11:28:08.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T11:28:08.761 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/log/ceph to /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434/remote/smithi028/log 2024-09-19T11:28:08.762 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T11:28:09.286 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T11:28:09.296 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T11:28:09.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T11:28:09.298 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T11:28:09.383 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T11:28:09.393 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T11:28:09.393 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T11:28:09.395 DEBUG:teuthology.orchestra.run.smithi028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T11:28:09.410 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-19T11:28:09.410 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-09-19T11:28:09.414 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T11:28:09.414 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-19T11:28:09.414 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 30 +166us[ +166us] +/- 64ms 2024-09-19T11:28:09.414 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 +355us[ +362us] +/- 59ms 2024-09-19T11:28:09.414 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -673us[ -666us] +/- 87ms 2024-09-19T11:28:09.414 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:28:09.416 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T11:28:09.416 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-09-19T11:28:09.416 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 30 +161us[ +161us] +/- 64ms 2024-09-19T11:28:09.416 INFO:teuthology.orchestra.run.smithi028.stdout:^* hv02.front.sepia.ceph.com 3 6 377 32 +323us[ +328us] +/- 59ms 2024-09-19T11:28:09.416 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -668us[ -668us] +/- 87ms 2024-09-19T11:28:09.416 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T11:28:09.416 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T11:28:09.426 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T11:28:09.426 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T11:28:09.434 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T11:28:09.522 DEBUG:teuthology.orchestra.run.smithi028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T11:28:09.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-19T11:28:09.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:28:09.620 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-19T11:28:09.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:28:09.648 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T11:28:09.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T11:28:09.666 INFO:teuthology.task.internal:Duration was 720.577321 seconds 2024-09-19T11:28:09.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T11:28:09.675 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T11:28:09.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T11:28:09.678 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T11:28:09.718 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T11:28:09.728 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T11:28:09.994 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T11:28:09.994 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-09-19T11:28:09.994 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T11:28:10.056 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi028.front.sepia.ceph.com 2024-09-19T11:28:10.057 DEBUG:teuthology.orchestra.run.smithi028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T11:28:10.082 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T11:28:10.082 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T11:28:10.099 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T11:28:10.149 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T11:28:10.150 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T11:28:10.169 DEBUG:teuthology.orchestra.run.smithi028:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T11:28:10.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T11:28:10.301 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T11:28:10.301 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T11:28:10.328 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T11:28:10.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T11:28:10.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T11:28:10.370 DEBUG:teuthology.orchestra.run.smithi028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T11:28:10.392 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-09-19T11:28:10.418 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = core 2024-09-19T11:28:10.441 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T11:28:10.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:28:10.471 DEBUG:teuthology.orchestra.run.smithi028:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T11:28:10.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T11:28:10.496 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T11:28:10.506 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T11:28:10.507 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434/remote/smithi022 2024-09-19T11:28:10.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T11:28:10.546 DEBUG:teuthology.misc:Transferring archived files from smithi028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuvalif-2024-09-19_08:49:34-rgw-wip-yuval-test-distro-default-smithi/7912434/remote/smithi028 2024-09-19T11:28:10.546 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T11:28:10.580 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T11:28:10.580 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T11:28:10.582 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T11:28:10.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T11:28:10.650 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T11:28:10.651 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T11:28:10.661 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T11:28:10.661 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T11:28:10.664 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T11:28:10.677 INFO:teuthology.orchestra.run.smithi022.stdout: 265734 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 11:28 /home/ubuntu/cephtest 2024-09-19T11:28:10.687 INFO:teuthology.orchestra.run.smithi028.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 11:28 /home/ubuntu/cephtest 2024-09-19T11:28:10.688 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T11:28:10.699 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T11:28:10.721 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec supported-random-distro$/{centos_latest}} duration: 720.5773210525513 flavor: default owner: scheduled_yuvalif@teuthology success: true 2024-09-19T11:28:10.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T11:28:10.813 INFO:teuthology.run:pass